Unreachable needs update

We built and installed Netdata on AWS Linux 2. It installed without issue and is currently running; yet, the portal has “UNREACHABLE NEEDS UPDATE”. The needs update link says, “The agent on this node has an older version and an update is required. Please upgrade to agent version 1.26 or above.”; however, this was a new build done yesterday and is running 1.26.

$ netdata -V
netdata v1.26.0-375-nightly

We tried another EC2 instance and got the same error with nothing working on the portal.

Welcome to our community @wphowell,

I assume that the “needs update” message that you refer to is on Netdata Cloud, correct?

@Stelios_Fragkakis I think version number was used for composite instead of an ACLK version. Who is the right person to check what is the logic that recognizes agent version on cloud side? Perhaps it has an issue with nightly build versions?

Yes, this is on Netdata Cloud.

Is there a way to pull down 1.26.0 that isn’t the nightly 375?

Have upgraded to netdata v1.26.0-400-nightly and still have the same error in netdata.cloud. Have the agent installed and running on 2 servers, but both present the same error:

“UNREACHABLE” “NEEDS UPDATE”

" The agent on this node has an older version and an update is required. Please upgrade to agent version 1.26 or above."

@wphowell that doesn’t sound right. We are investigating internally and one of our engineers will jump back to this thread with more information.

Thank you for being patient :slight_smile:

If the agent is unreachable it means Netdata Cloud cannot connect to it so it doesn’t know its version yet. Currently Netdata Cloud has a known issue that it also shows “needs update” when it doesn’t know the agent version yet.
So this “needs update” is not true if you are using v1.26. The issue here is that you have an unreachable agent :sweat_smile:
We can help with that if you tell us a little bit more about your setup :wink:

Sure, these are AWS EC2 Micro and Small instances. We didn’t enable any ports in the AWS firewall because the assumption was that the agent would be connecting out, with nothing trying to connect inbound.

Do we need to make some firewall changes?

It looks like it’s a certificate issue:

2020-12-17 04:25:40: netdata ERROR : ACLK_Main : Libwebsockets: SSL error: unable to get local issuer certificate (preverify_ok=0;err=20;depth=1)
(errno 2, No such file or directory)

I’ve checked the openssl.cnf file for correct paths and updated the ca trust and still no joy.

I also pulled the certificates from apps.netdata.cloud to ensure they’re in the cacert file:

Certificate chain
 0 s:/CN=app.netdata.cloud
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
-----BEGIN CERTIFICATE-----
MIIFWzCCBEOgAwIBAgISA/8XE63uY+Mbq4XHAqQ45MZHMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDExMDcwMzQzMzVaFw0y
MTAyMDUwMzQzMzVaMBwxGjAYBgNVBAMTEWFwcC5uZXRkYXRhLmNsb3VkMIIBIjAN
BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArhfI1e3yOwnJCRwKIgC5dq3jwXsR
cPnB9JU27zlsCJRmtaBcZDXmcVtKQ/JrEFYZe2QQtKH0rtxyUAyl9OnySEoMW8SC
uZ7/kZ8EDpyk6QhZPg1ncVXLd2TQzv5i4VVK2yaTh2ZKJ0DFf6R1O/bcZZ5S9noq
eo6RiTw3AN1aQKdrriPU/37mfYT7dksM0pke+4iLIqJz5dDE9+R4ZaoH2p5Ooi+s
Z/+vXEufTLRnTBS1LYjcKCXwe3lJUV3s8N4LuG2AaK7eBXm5NEctwgNDI0tC+SUR
taLW1goMFDeoFo+XCTUv466X861XP3XwZA9HUOcV93zqpRSO7j0OUCM0ZwIDAQAB
o4ICZzCCAmMwDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr
BgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBQmkGQTIuDWnf62OfHwm0dX
4DJpkzAfBgNVHSMEGDAWgBSoSmpjBH3duubRObemRWXv86jsoTBvBggrBgEFBQcB
AQRjMGEwLgYIKwYBBQUHMAGGImh0dHA6Ly9vY3NwLmludC14My5sZXRzZW5jcnlw
dC5vcmcwLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlw
dC5vcmcvMBwGA1UdEQQVMBOCEWFwcC5uZXRkYXRhLmNsb3VkMEwGA1UdIARFMEMw
CAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9j
cHMubGV0c2VuY3J5cHQub3JnMIIBBQYKKwYBBAHWeQIEAgSB9gSB8wDxAHcARJRl
LrDuzq/EQAfYqP4owNrmgr7YyzG1P9MzlrW2gagAAAF1oQRN6gAABAMASDBGAiEA
i9yr3558pKx0bMLKIyyIVkB8UvnVMoCWJvxKA63V4h4CIQC4cYzP+CA294AjST+E
piRbBZPKeXbWQ/oyCAJrYD9EhAB2AH0+8viP/4hVaCTCwMqeUol5K8UOeAl/LmqX
aJl+IvDXAAABdaEETiIAAAQDAEcwRQIgTWB4VJsxRAavjDdBdzDIVeM3f7im5qsr
apBqjwGHYVICIQCv1UzIwC5a9D5vRpIDxWRilGA/WzS70KRSVq+YmjWevDANBgkq
hkiG9w0BAQsFAAOCAQEADCOrpaxIElufe9beuanrPSNfzSNtI1g1FT4kmG1O6GD0
hibC2Kt6CgcFRO8PS55Dls0dPN+n24YezXkHWkPzhGjzOpxaQVma0UI0hLkVVD8h
qndkpQqIbkqWaDRq9oxiHvlh5gx5x8t2wiKqyuqV9DMP48J7q7cXhzmzXrV8RS60
A2ix/ruogL68pApxoRAPgQPSJ9FBoG6WsBv0NHp8l/QN/WucDQxTEuiBPxVxVs7i
l8AsAYNj/u3NtwqHqTriJoeEzwyELDxlntzYebRuVYyNVbgSaO2Um1jEskPO2uvD
xFj5VWHc2yAUKAcLedJIzPR7ctsAAsYFQLeh14gg8A==
-----END CERTIFICATE-----
 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
-----BEGIN CERTIFICATE-----
MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow
SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT
GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF
q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8
SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0
Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA
a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj
/PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T
AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG
CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv
bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k
c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw
VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC
ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz
MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu
Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF
AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo
uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/
wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu
X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG
PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6
KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=app.netdata.cloud
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---

It hasn’t made any difference at all.

The issue was that the netdata client only works with OpenSSL v1.0.2k (the default on Amazon Linux 2) and we run v1.0.2u.

1 Like

Hey,

I am glad that you found the answer. So you couldn’t connect from Netdata Cloud to Netdata Agent because of the OpenSSL library. Can you please give me some more info in regards to your Netdata Agent? I think that the Agent should have signaled somehow that this is the error, instead if manually troubleshooting it.

  1. During Agent installation, if you use the kickstart script, did it showed any error?
  2. Can you send us a sample of the error.log? (docs)
  3. Did you claim your nodes with success using the netdata-claim.sh script?