Every day I lose my Ping integration charts

Help
1

I’ve been running Netdata for a few days now and becoming familiar with it. I bought a Homelab license.

I’ve got a primary node (running baremetal on a Proxmox host) and 4 other agents streaming to that host. I also set up netdata.cloud and am streaming to that. So far so good.

BUT, basically every day I am greeted with this empty Dashboard - my Ping integration has stopped working for some reason. I am not sure how to troubleshoot this!

# netdata -v
netdata v1.45.0-568-nightly

Here are some screenshots

Empty Dashboard charts

image
image3184×960 209 KB

Searching for “ping” fails to filter any metrics

image
image1192×678 39.8 KB

Collectors marked as “Failed” in Space Configurations

image
image2954×1302 199 KB

I tried the troubleshooting steps listed on the collector guide, which say to run

cd /usr/libexec/netdata/plugins.d/
sudo -u netdata -s
./go.d.plugin -d -m ping

And when I do that, it all seems to work fine, I see pings running, data is flying by on the console, no errors etc. But nothing makes it to the charts.

The only thing that “works” is restarting the netdata service:

systemctl restart netdata

…but obviously that’s not practical.

2

Moments after restarting the service…

image
image2824×828 264 KB

Also, not sure if this is the correct place to post this? Should I be opening an issue on Github instead?

3

Hi, @luckman212. Yes, we use GitHub for bugs.

I am not sure how to troubleshoot this!

Let’s check the logs. Run the following on the host that is pinging when you see ping jobs in a failed state:

journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata | grep ping | grep go.d
4

Thank you. So far this morning it hasn’t happened yet (of course). But I will keep checking later today and report back.

5

I tried to reproduce it and couldn’t but I didn’t try hard:

  • Install Netdata deb not the latest nightly.
  • Configure 3 ping data collection jobs. Restart Netdata. Check ping charts.
  • Run netdata-updater manually. Update Netdata to the latest nightly.
  • Check ping charts.

I did it once and all went well.

Are you using Discord? You can come to Netdata Discord. We respond there faster and can debug your problem in real-time (if I won’t be AFK at the time).

6

I do have Discord but am not there much. I realize that’s where the “action” is for this project, so when I can reproduce I will definitely stop by. Thanks!

7

Just an update. Not sure what changed but for the last week this has been working ok. Maybe it was a bug in one of the nightly builds? I am cautiously calling this “solved” for now.

8

Back again to confirm, this issue has not come back. Must have been a glitch in nightly. Onward!