Suddenly ErrUnsupportedAgent on a binpkg-deb agent

Help
1

Hi,

This morning I started to get system.swap (swap), used swap = 100%alerts on my Debian 11 node, so I wanted to check it.

I went to the cloud dashboard, and realised that the netdata agent was offline.

I tunneled in, and the memory seemed good:

MiB Mem : 32089.8 total, 4537.2 free, 4100.7 used, 23451.9 buff/cache
MiB Swap: 512.0 total, 0.4 free, 511.6 used. 27518.8 avail Mem

I checked the agent, it was active, tried restarting it, stopping, starting, etc, but it was still “Offline” according to the cloud. The alerts stopped after a while, only appeared only once per restart.

I checked ACLK status: sudo netdatacli aclk-state

ACLK Available: Yes
ACLK Version: 2
Protocols Supported: Protobuf
Protocol Used: Protobuf
MQTT Version: 5
Claimed: Yes
Claimed Id: 25718358-9c4a-11eb-9496-f23c91735f05
Cloud URL: https://app.netdata.cloud
Online: No
Reconnect count: 0
Banned By Cloud: Yes
Next Connection Attempt At: 2023-06-06 13:39:33
Last Backoff: 60.000

It was offline here as well, and it was always failed at connecting, reconnect count was always zero.

So I checked error.log:

2023-06-06 13:23:32: netdata INFO : ACLK_MAIN : HTTPS “GET” request to “app.netdata.cloud” finished with HTTP code: 406
2023-06-06 13:23:32: netdata ERROR : ACLK_MAIN : The HTTP code not 200 OK (Got 406)
2023-06-06 13:23:32: netdata ERROR : ACLK_MAIN : Cloud returned EC=“”, Msg-Key:“ErrUnsupportedAgent”, Msg:“do not retry: unsupported agent”, BlockRetry:true, Backoff:0s (-1 unset by cloud)
2023-06-06 13:23:32: netdata ERROR : ACLK_MAIN : Failed to Get ACLK environment
2023-06-06 13:23:32: netdata INFO : ACLK_MAIN : Wait before attempting to reconnect in 60.000 seconds
2023-06-06 13:23:33: netdata INFO : PD[node.d] : PLUGINSD: plugin called DISABLE. Disabling it.
2023-06-06 13:23:33: netdata INFO : PD[node.d] : PLUGINSD: ‘host:spacedesigner3d.com’, ‘/usr/libexec/netdata/plugins.d/node.d.plugin’ (pid 1006802) disconnected after 0 successful data collections (ENDs).
2023-06-06 13:23:33: netdata INFO : PD[node.d] : child pid 1006802 killed by signal 15.
2023-06-06 13:23:33: netdata INFO : PD[node.d] : PLUGINSD: ‘host:spacedesigner3d.com’, ‘/usr/libexec/netdata/plugins.d/node.d.plugin’ (pid 1006802) does not generate useful output but it reports success (exits with 0). Will not start it again - it is now disabled…
2023-06-06 13:23:33: netdata INFO : P[proc] : Using now_boottime_usec() for uptime (dt is 7 ms)
2023-06-06 13:23:33: netdata LOG FLOOD PROTECTION too many logs (201 logs in 1 seconds, threshold is set to 200 logs in 1200 seconds). Preventing more logs from process ‘netdata’ for 1199 seconds.
2023-06-06 13:43:33: netdata LOG FLOOD PROTECTION resuming logging from process ‘netdata’ (prevented 134 logs in the last 1200 seconds).
2023-06-06 13:43:33: netdata INFO : ACLK_MAIN : Attempting connection now
2023-06-06 13:43:33: netdata INFO : ACLK_MAIN : HTTPS “GET” request to “app.netdata.cloud” finished with HTTP code: 406
2023-06-06 13:43:33: netdata ERROR : ACLK_MAIN : The HTTP code not 200 OK (Got 406)
2023-06-06 13:43:33: netdata ERROR : ACLK_MAIN : Cloud returned EC=“”, Msg-Key:“ErrUnsupportedAgent”, Msg:“do not retry: unsupported agent”, BlockRetry:true, Backoff:0s (-1 unset by cloud)
2023-06-06 13:43:33: netdata ERROR : ACLK_MAIN : Failed to Get ACLK environment
2023-06-06 13:43:33: netdata INFO : ACLK_MAIN : Wait before attempting to reconnect in 60.000 seconds

It said ErrUnsupportedAgentso I checked buildinfo:

Version: netdata v1.39.0-97-nightly
Configure options: ‘–build=x86_64-linux-gnu’ ‘–includedir=${prefix}/include’ ‘–mandir=${prefix}/share/man’ ‘–infodir=${prefix}/share/info’ ‘–disable-option-checking’ ‘–disable-silent-rules’ ‘–libdir=${prefix}/lib/x86_64-linux-gnu’ ‘–libexecdir=${prefix}/lib/x86_64-linux-gnu’ ‘–disable-maintainer-mode’ ‘–prefix=/usr’ ‘–sysconfdir=/etc’ ‘–localstatedir=/var’ ‘–libdir=/usr/lib’ ‘–libexecdir=/usr/libexec’ ‘–with-user=netdata’ ‘–with-math’ ‘–with-zlib’ ‘–with-webdir=/var/lib/netdata/www’ ‘–disable-dependency-tracking’ ‘build_alias=x86_64-linux-gnu’ ‘CFLAGS=-g -O2 -ffile-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security’ ‘LDFLAGS=-Wl,-z,relro’ ‘CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2’ ‘CXXFLAGS=-g -O2 -ffile-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security’
Install type: binpkg-deb
Binary architecture: x86_64
Packaging distro:
Features:
dbengine: YES
Native HTTPS: YES
Netdata Cloud: YES
ACLK: YES
TLS Host Verification: YES
Machine Learning: YES
Stream Compression: YES
HTTPD (h2o): YES
Libraries:
protobuf: YES (system)
jemalloc: NO
JSON-C: YES
libcap: NO
libcrypto: YES
libm: YES
tcalloc: NO
zlib: YES
Plugins:
apps: YES
cgroup Network Tracking: YES
CUPS: YES
debugfs: YES
EBPF: YES
IPMI: YES
NFACCT: YES
perf: YES
slabinfo: YES
Xen: NO
Xen VBD Error Tracking: NO
Exporters:
AWS Kinesis: NO
GCP PubSub: NO
MongoDB: NO
Prometheus Remote Write: YES
Debug/Developer Features:
Trace Allocations: NO

Why is binpkg-deb unsupported now?

I had some issues with netdata before and maybe this node was also problematic. But we fixed it on the forum. Maybe some of the fix flagged this agent in a way that is unsupported now?

How could I fix it, what’s the source of the problem? :expressionless: Why am I getting swap alerts if there is plenty of free swap according to top?

Thanks in advance! :slight_smile:

2

Hi @Tudvari

We are sorry, but currently agents with version v1.39.0-97-nightly such as yours have an issue that forced us to prevent them from connecting to the cloud.

The issue will be resolved in the next nightly version. If you wish, you can manually run the netdata-updater script to update, or wait until it runs by itself.

We apologize for the trouble.

We can check though the issue of the swap usage alert, but from your post MiB Swap: 512.0 total, 0.4 free, 511.6 used indeed it seems that there is little swap left?

1 Like
3

Thanks, I updated it and now it works! :slight_smile:

Regarding the swap alert: shoot, I misread it, and indeed it is nearly full. I just have to find out why, I guess. :smiley:

4

Great! Thanks for the feedback!

1 Like