Updated the agent, Cloud UI still says that they need to be updated

Tudvari · February 22, 2022, 3:35pm

Environment

Debian 10

Problem/Question

I’ve noticed the alert on the Cloud UI that I need to update a few nodes’ agents:

So I clicked on the link and then looked up my install type:

INSTALL_TYPE='legacy-build'

So I went with this option:

And from this page I copied and ran this command:

wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh

I said yes on some “replace configure file with package’s version?” questions and then checked the install type again:

INSTALL_TYPE='binpkg-deb'
PREBUILT_ARCH='x86_64'
PREBUILT_DISTRO=' '

And also checked the version with “netdata -v” and it got updated:

netdata v1.33.1-42-g570a61e17

Also checked whether protobuf is installed or not:

 dpkg -l | grep protobuf
ii  libprotobuf-c1:amd64              1.3.1-1+b1                   amd64        Protocol Buffers C shared library (protobuf-c)
ii  libprotobuf17:amd64               3.6.1.3-2                    amd64        protocol buffers C++ library

Yet the Cloud UI still displays the same old version, the same info about the nodes as they were before the process described above. I also waited an hour, thinking that maybe it just needs time.

What I expected to happen

That the alerts are gone or at least the displayed version is updated.

Thanks in advance!

Tasos_Katsoulas · February 24, 2022, 10:34am

I can verify the same problem. This is a bug report from one of our users.

ekkuFWDU

In one the nodes. Agent installed with Docker. OS: Debian 10.8

The netdata -W buildinfo

# netdata -W buildinfo
Version: netdata v1.33.1
Configure options: '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/usr/libexec' '--libdir=/usr/lib' '--with-zlib' '--with-math' '--with-user=netdata' '--without-bundled-protobuf' '--with-bundled-libJudy' 'CFLAGS=' 'LDFLAGS='
Install type: unknown
Features:
dbengine: YES
Native HTTPS: YES
Netdata Cloud: YES
ACLK Next Generation: YES
ACLK-NG New Cloud Protocol: YES
ACLK Legacy: NO
TLS Host Verification: YES
Machine Learning: YES
Stream Compression: YES
Libraries:
protobuf: YES (system)
jemalloc: NO
JSON-C: YES
libcap: NO
libcrypto: YES
libm: YES
tcalloc: NO
zlib: YES
Plugins:
apps: YES
cgroup Network Tracking: YES
CUPS: NO
EBPF: NO
IPMI: YES
NFACCT: NO
perf: YES
slabinfo: YES
Xen: NO
Xen VBD Error Tracking: NO
Exporters:
AWS Kinesis: NO
GCP PubSub: NO
MongoDB: YES
Prometheus Remote Write: YES

And the netdatacli aclk-state

# netdatacli aclk-state
ACLK Available: Yes
ACLK Implementation: Next Generation
New Cloud Protocol Support: Yes
Claimed: Yes
Claimed Id: 802feb3a-bd2a-11eb-ac44-0242ac130002
Online: Yes
Used Cloud Protocol: Legacy

Tasos_Katsoulas · March 2, 2022, 12:18pm

Hello @Tudvari. You are using the nightly builds. If you haven’t specify the --no-updates flag in the kickstart script this issue should have been resolved by now. Otherwise try to Update the Netdata Agent | Learn Netdata. In any case please ACK that you are (or not) receiving this notification.

Cedric_Lemery · March 2, 2022, 7:15pm

Hi, I have the same issue here on various server.

# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian

and

# netdata -W buildinfo
Version: netdata v1.33.1-93-g23fab0772
Configure options:  '--build=x86_64-linux-gnu' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--with-user=netdata' '--with-math' '--with-zlib' '--with-webdir=/var/lib/netdata/www' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fdebug-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fdebug-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security'
Install type: binpkg-deb
    Binary architecture: x86_64
    Packaging distro:  
Features:
    dbengine:                   YES
    Native HTTPS:               YES
    Netdata Cloud:              YES 
    ACLK Next Generation:       YES
    ACLK-NG New Cloud Protocol: YES
    ACLK Legacy:                NO
    TLS Host Verification:      YES
    Machine Learning:           YES
    Stream Compression:         NO
Libraries:
    protobuf:                YES (system)
    jemalloc:                NO
    JSON-C:                  YES
    libcap:                  NO
    libcrypto:               YES
    libm:                    YES
    tcalloc:                 NO
    zlib:                    YES
Plugins:
    apps:                    YES
    cgroup Network Tracking: YES
    CUPS:                    YES
    EBPF:                    YES
    IPMI:                    YES
    NFACCT:                  YES
    perf:                    YES
    slabinfo:                YES
    Xen:                     NO
    Xen VBD Error Tracking:  NO
Exporters:
    AWS Kinesis:             NO
    GCP PubSub:              NO
    MongoDB:                 NO
    Prometheus Remote Write: YES

But on netdatacloud :
Screenshot (10)

Tudvari · March 3, 2022, 10:18am

I’ve tried the kickstart script again to update but it resulted in:

— Using /tmp/netdata-kickstart-w1h37vLPwS as a temporary directory. —
— Found an existing netdata install at /, with installation type ‘binpkg-deb’. —
WARNING Not updating existing install at /.

— Not attempting to claim existing install at / (no claiming token provided). —

So after that I tried it with the “–reinstall” flag because according to the update site:

If the above command fails, you can reinstall Netdata to get the latest version. This also preserves your configuration in netdata.conf or other files.

But that resulted in:

— Using /tmp/netdata-kickstart-JnCcTWrhrq as a temporary directory. —
— Found an existing netdata install at /, with installation type ‘binpkg-deb’. —
— Found an existing netdata install at /, but user requested reinstall, continuing. —
— Attempting to install using native packages… —
— Repository configuration is already present, attempting to install netdata. —
— Checking for usable Netdata package. —
— Installing Netdata package. —
[/tmp/netdata-kickstart-JnCcTWrhrq]$ sudo env apt-get install netdata
Reading package lists… Done
Building dependency tree
Reading state information… Done
netdata is already the newest version (1.33.1-99-nightly).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

So what should I do now? :\

Tasos_Katsoulas · March 4, 2022, 4:33pm

@Tudvari Do you still receive this notification for the node in question? (“Need update”)

Tudvari · March 4, 2022, 4:44pm

Yes, sadly. :\ (here are some extra characters to reach the minimum)

Tasos_Katsoulas · March 4, 2022, 5:32pm

Could you provide me with the netdata -W buildinfo and netdatacli aclk-state for this particular node?

Tudvari · March 7, 2022, 8:28am

Sure!

netdata -W buildinfo

Version: netdata v1.33.1-113-gb320d4d98
Configure options: ‘–build=x86_64-linux-gnu’ ‘–includedir=${prefix}/include’ ‘–mandir=${prefix}/share/man’ ‘–infodir=${prefix}/share/info’ ‘–disable-silent-rules’ ‘–libdir=${prefix}/lib/x86_64-linux-gnu’ ‘–libexecdir=${prefix}/lib/x86_64-linux-gnu’ ‘–disable-maintainer-mode’ ‘–disable-dependency-tracking’ ‘–prefix=/usr’ ‘–sysconfdir=/etc’ ‘–localstatedir=/var’ ‘–libdir=/usr/lib’ ‘–libexecdir=/usr/libexec’ ‘–with-user=netdata’ ‘–with-math’ ‘–with-zlib’ ‘–with-webdir=/var/lib/netdata/www’ ‘build_alias=x86_64-linux-gnu’ ‘CFLAGS=-g -O2 -fdebug-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security’ ‘LDFLAGS=-Wl,-z,relro’ ‘CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2’ ‘CXXFLAGS=-g -O2 -fdebug-prefix-map=/usr/src/netdata=. -fstack-protector-strong -Wformat -Werror=format-security’
Install type: binpkg-deb
Binary architecture: x86_64
Packaging distro:
Features:
dbengine: YES
Native HTTPS: YES
Netdata Cloud: YES
ACLK Next Generation: YES
ACLK-NG New Cloud Protocol: YES
ACLK Legacy: NO
TLS Host Verification: YES
Machine Learning: YES
Stream Compression: NO
Libraries:
protobuf: YES (system)
jemalloc: NO
JSON-C: YES
libcap: NO
libcrypto: YES
libm: YES
tcalloc: NO
zlib: YES
Plugins:
apps: YES
cgroup Network Tracking: YES
CUPS: YES
EBPF: NO
IPMI: YES
NFACCT: YES
perf: YES
slabinfo: YES
Xen: NO
Xen VBD Error Tracking: NO
Exporters:
AWS Kinesis: NO
GCP PubSub: NO
MongoDB: NO
Prometheus Remote Write: YES

netdatacli aclk-state

ACLK Available: Yes
ACLK Implementation: Next Generation
New Cloud Protocol Support: Yes
Claimed: Yes
Claimed Id: 884c04e0-9c58-11eb-b8d4-0e21c2e7f413
Online: Yes
Used Cloud Protocol: New

Tudvari · March 16, 2022, 9:45am

Now the time is up so they got removed from the Cloud. So I tried to reclaim them, but it always says:

Failed to claim node with the following error message:“already claimed”

I tried this: Can't reclaim node into netdata cloud · Issue #9346 · netdata/netdata · GitHub and removed the cloud.d directory, added the needed flag, but the error still persists.

Tudvari · March 25, 2022, 1:36pm

Bump! :\ I’m still having the issue sadly.

lars · March 25, 2022, 1:56pm

After some time and probably some scheduled maintenance of Netdata Cloud the problem “Need Update” disappeared for all our updated nodes. However, I did need to reconfigure dashboards again as these were showing “Error”.

Tasos_Katsoulas · March 28, 2022, 10:54am

@here sorry for the late response, could you bring me up to speed. What’s your status?

Do you still receive any notification about nodes that need your attention (need update, protobuf support) in the Netdata Cloud?
@Tudvari to reclaim the node you must follow this Connect Agent to Cloud | Learn Netdata section in our docs. If you find any problem, please let me know.

Tudvari · March 28, 2022, 11:16am

No, there are no notifications anymore because after March 15th or so the deadline was over and they got “unclaimed”. I can see them nowhere.

So after that I tried claiming them again, but as I said, it says
Failed to claim node with the following error message:“already claimed”

I tried this: Can’t reclaim node into netdata cloud · Issue #9346 · netdata/netdata · GitHub and removed the cloud.d directory, added the needed flag, but the error still persists.

Now I’ve read after the error message that

If this node was created by cloning a VM or as a container from a template, please remove the file //var/lib/netdata/registry/netdata.public.unique.id and restart the agent. Then try to claim it again with the same options.

This node wasn’t created by cloning a VM or as a container, but tried removing the file anyway… restarted the agent, and started the claming process again…

And it worked. And I could reclaim all of my lost nodes. I guess we can close this topic, thanks all!

Topic		Replies	Views
After some struggles to update netdata agent, finally it got updated, but now the service fails to start. Help agent , installation , update	21	3732	September 29, 2022
Up to date, but "Needs update" Help cloud	2	495	February 20, 2021
i have problem when i update new code update Help agent , update	1	523	March 31, 2022
Nightly update netdata agent says there is a new nightly, but not in repo Help agent	3	338	March 8, 2023
Reinstall the Netdata Agent Help agent	2	578	February 3, 2022

Updated the agent, Cloud UI still says that they need to be updated

Environment

Problem/Question

What I expected to happen

Related Topics