Problem/Question
Hello, I use Netdata for 5 AWS accounts with managed kubernetes clusters.
I noticed that the network out for the EC2 are quite big when I installed Netdata.
Environment/Browser
What I expected to happen
What are the parameters to reduce this effect and all outgoing traffic to netdata cloud ?
I changed this :
[global]
update every = 60
Is it the good way to reduce all metrics send by netdata (parent?) to netdata cloud ?
Can we compress those? Or do we have more parameters?
Thanks in advance for your time.
Hello,
usage should differ in the time somebody is looking at the Cloud dashboard and in the time nobody is looking.
By default (when nobody actively looking at the cloud dashboard) only metadata is sent. Stuff like:
- charts list
- alarms
- node info
Therefore during this idle time, the traffic should depend mainly on how often new charts are created and deleted (e.g. when you have a lot of docker containers constantly started and stopped).
Those data are currently, unfortunately, being sent to the cloud uncompressed in JSON text form. We are working on an update which will rectify this (it will use binary protocol + “incremental diffs”/reduction in duplicated data being sent)
When somebody is looking at the cloud dashboard, data from visible nodes (only charts currently visible in browser of at least one Cloud user) are being streamed. This is done compressed. We do however have ideas on how to decrease the traffic in this case as well in the future.
2 Likes
Still, the outgoing traffic should be extremely low, unless there’s something netdata monitors that keeps appearing and disappearing. Metrics are not sent to the cloud, so the update_every doesn’t make a difference.
We had seen one other report from a user regarding sustained high egress and couldn’t reproduce it or identify the cause. @KubernetesLover would you be willing to do a call with our engineers so we can try to pinpoint the source? If so, please email me at chris@netdata.cloud and I will set it up.