Ubuntu 18.04
Installed AWS sdk for Kinesis
No data is showing up in Kinesis and I have this error in the error log.
2021-12-09 18:37:53: netdata INFO : EXPORTING : thread created with task id 11386
2021-12-09 18:37:53: netdata INFO : EXPORTING : set name of thread 11386 to EXPORTING
2021-12-09 18:37:53: netdata INFO : EXPORTING : Processing connector instance (test_instance)
2021-12-09 18:37:53: netdata INFO : EXPORTING : Instance (test_instance) on connector (kinesis) is enabled and scheduled for activation
2021-12-09 18:37:53: netdata INFO : EXPORTING : Instance test_instance on kinesis
2021-12-09 18:37:53: netdata ERROR : EXPORTING : AWS Kinesis support isn’t compiled
During the compile I see
checking for AWS_CPP_SDK_KINESIS… yes. so it looks like the sdk is there but it is not using it?
and
checking if kinesis backend should be enabled… no
But I’m not sure if the backend matters with the new config, or even how to enable it if it does.
Any help in debugging this or getting it to work would be greatly appreciated, I’ve search till my eyes hurt and I can’t find the solution to this.
Is there some other compile option I need to use?
Please look at a PR that will be merged in a few days. Meanwhile, you can apply the changes manually to your configure.ac.
Thanks, I tried the changes.
I still get this error. I can’ figure out why it thinks the libs are not there.
checking for AWS_CPP_SDK_CORE… yes
checking for AWS_CPP_SDK_KINESIS… yes
checking for aws_checksums_crc32 in -laws-checksums… no
checking for aws_default_allocator in -laws-c-common… no
checking for aws_event_stream_library_init in -laws-c-event-stream… no
configure: error: AWS SKD third party dependencies required but not found
Maybe you installed the libs in /usr/local/lib. pkg-config can’t look for libs there. Try installing them into /usr/lib - use /usr as a prefix instead of /usr/local.
Still getting same error. It looks like the libs are installed in /usr/lib. though.
:/usr/lib$ ls libaws*
Please run
nm -D /usr/lib/libaws-checksums.so | grep aws_checksums_crc32
nm -D /usr/lib/libaws-c-common.so | grep aws_default_allocator
nm -D /usr/lib/libaws-c-event-stream.so | grep aws_event_stream_library_init
Do you see the symbols? Please also look into config.log. What errors do you see? Expected output:
configure:12541: checking for aws_checksums_crc32 in -laws-checksums
configure:12564: gcc -o conftest -Og -ggdb -Wall -Wextra -Wformat-signedness -fno-omit-frame-pointer -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTiFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1 -pthread conftest.c -laws-checksums >&5
configure:12564: $? = 0
configure:12574: result: yes
configure:12585: checking for aws_default_allocator in -laws-c-common
configure:12608: gcc -o conftest -Og -ggdb -Wall -Wextra -Wformat-signedness -fno-omit-frame-pointer -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTiFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1 -pthread conftest.c -laws-c-common >&5
configure:12608: $? = 0
configure:12618: result: yes
configure:12629: checking for aws_event_stream_library_init in -laws-c-event-stream
configure:12652: gcc -o conftest -Og -ggdb -Wall -Wextra -Wformat-signedness -fno-omit-frame-pointer -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTiFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1 -pthread conftest.c -laws-c-event-stream >&5
configure:12652: $? = 0
configure:12662: result: yes
There are no .so files in /usr/lib. only .a files for the libaws stuff. perhaps this is a problem?
First instance of aws_checksums_crc32 in config.log fails with this…
configure:11913: checking for aws_checksums_crc32 in -laws-checksums
configure:11938: gcc -o conftest -O2 -pthread conftest.c -laws-checksums >&5
/usr/lib/gcc/x86_64-linux-gnu/7/…/…/…/x86_64-linux-gnu/libaws-checksums.a(crc.c.o): In function `aws_checksums_crc32’:
crc.c:(.text+0x2e): undefined reference to `aws_cpu_has_feature’
/usr/lib/gcc/x86_64-linux-gnu/7/…/…/…/x86_64-linux-gnu/libaws-checksums.a(crc.c.o): In function `aws_checksums_crc32c’:
crc.c:(.text+0x8e): undefined reference to `aws_cpu_has_feature’
crc.c:(.text+0xbe): undefined reference to `aws_cpu_has_feature’
/usr/lib/gcc/x86_64-linux-gnu/7/…/…/…/x86_64-linux-gnu/libaws-checksums.a(crc32c_sse42_asm.c.o): In function `loop_small_220’:
crc32c_sse42_asm.c:(.text+0xacb): undefined reference to `aws_cpu_has_feature’
collect2: error: ld returned 1 exit status
configure:11938: $? = 1
configure: failed program was:
Could you add to configure.ac aws-c-common in a couple of AC_CHECK_LIB
AC_CHECK_LIB(
[aws-checksums],
[aws_checksums_crc32],
[have_libaws_checksums=yes],
[have_libaws_checksums=no],
[aws-c-common]
)
AC_CHECK_LIB(
[aws-c-common],
[aws_default_allocator],
[have_libaws_c_common=yes],
[have_libaws_c_common=no]
)
AC_CHECK_LIB(
[aws-c-event-stream],
[aws_event_stream_library_init],
[have_libaws_c_event_stream=yes],
[have_libaws_c_event_stream=no],
[aws-c-common]
)
Please also let me know what error do you get for aws-c-common itself in config.log.
Still having no luck with this.
Clean ubuntu 18.04 installation.
Followed these instructions for sdk installation.
aws_kinesis
Then attempt to install with…
bash <(curl -Ss https://my-netdata.io/kickstart.sh ) --enable-backend-kinesis
Still fails with libaws-cpp-sdk-kinesis required but not found. try installing aws C++ SDK
libaws-cpp-sdk-kinesis.a is in /usr/lib
Can anyone confirm these instructions work for them on Ubuntu 18.04?
I was able to reproduce the issue. I’ve updated the fix .
ilyam8
January 12, 2022, 8:07am
11
The fix has been merged. @Mark_Round can you try to install again? Should be fixed now.