Use SNMP Collector to track total ISP Bandwidth / Month

Help
1

New to Netdata. I really like the beautiful graphs. Looks really nice.

I set up an SNMP collector using the following go.d/snmp.conf:

jobs:

  • name: xxxxxxxxxxxx
    update_every: 3
    hostname: “xxx.xxx.xxx.xxx”
    community: public
    options:
    version: 1
    charts:
    • id: “bandwidth_port3”
      title: “Bandwidth for eth0 WAN”
      units: “kilobits/s”
      type: “area”
      family: “Ports”
      dimensions:
      • name: “Download”
        oid: “1.3.6.1.2.1.2.2.1.10.3”
        algorithm: “incremental”
        multiplier: 8
        divisor: 1024
      • name: “Upload”
        oid: “1.3.6.1.2.1.2.2.1.16.3”
        multiplier: 8
        divisor: 1024

It looks great but there are a couple of things I really need. I would like to combine in/ out or upload / download and I would like to see a total. SNMP is not providing those metrics but I should be able to get them with a little math if I knew how. Can anyone point me in the right direction?

2

Hi, @BigPines

:smile:

My understanding is that you want to see the amount of data transferred (in, out) for the last N days. While Netdata has all the data needed to calculate this number, it is currently not possible to get the number using the Agent or Cloud UI.

I would like to combine in/ out or upload / download and I would like to see a total.

Not possible either. I think we can extend the configuration by allowing to set multiple OIDs (which will be summed), e.g.:

dimensions:
name: “TotalTraffic”
oids: 
  - “1.3.6.1.2.1.2.2.1.10.3”
  - "1.3.6.1.2.1.2.2.1.16.3"

if you think that will do for you consider opening a feature request.

3

Oops on the community disclosure. Changed! I’ll have to be more careful about that! :rofl:

Yes, you understand my use case correctly. My ISP is limiting my bandwidth to 1TB per month. I believe this is a common situation. I want to monitor and verify my bandwidth usage per day, month, etc.

Summing multiple OIDs would be nice. That would get me part way there. I submitted a feature request.

Too bad you don’t have anything that would calculate the total of the data within the timeframe selected. Seems like it should be fairly straight forward to do. There must be a way I could roll my own by using the data Netdata is collecting?

I saw Bytes Consumed in this example and it looks a lot like what I want however, I’m not sure how it is being done:

4

Сorrection: that is possible when using Netdata Cloud UI. Can be done by changing the chart “group by” to “instance” (the default is “dimension”).

5

Thanks. I’ll try that.

Side note - I am having a hard time understanding why I need to use some cloud account to have this feature. I am not monitoring remote locations. I just want to monitor one gateway in one network from within that network and I don’t want to share the data I am monitoring outside of that network. Why is there such a push to get people to sign up for a cloud account for this kind of thing? It isn’t just Netdata. It is everyone. I just don’t get it. :man_shrugging:

6

That worked to aggregate the in / out together. Thanks!

Now I just need to figure out how to get the total bandwidth utilized during the selected timeframe. I am shocked this doesn’t seem possible. :thinking:

7

The total amount of transferred data (one value, sum of all values in some time period) is not possible.

8

Thank you for trying. You gave me a partial solution so I marked that as the solution.

Can I not even write a custom collector which would get the total amount of transferred data (one value, sum of all values in some time period)?

Maybe I should put in a feature request for what I am looking for.

Mike

9

To tell you the truth, we really should have a feature that lets someone aggregate across time, it’s a very common request, especially for network BW and number of requests.

Certain aggregations make sense for certain metric types, so it will need some logic, but we’ve seen this request often enough to warrant a proper feature to cover it.

You can certainly create your own collector to do the same thing, but there are two workarounds I can think of:

10

Actually, I remembered a better way, using the API’s group parameter. See below.

DESKTOP-C7OKV71:/mnt/c/Users/cakri# wget -O - "http://localhost:19999/api/v1/data?chart=wmi_DESKTOP-C7OKV71_local.nic_Realtek_RTL8822CE_802_11ac_PCIe_A
dapter_bandwidth&group=sum&gtime=86400&points=10&after=-604800"
Connecting to localhost:19999 (127.0.0.1:19999)
writing to stdout
{
 "labels": ["time","received","sent"],
    "data":
 [
 [ 1670371200,1842055.0049621,-1300639.9740386],
 [ 1670284800,4874666.4748265,-2011886.3119831],
 [ 1670198400,355296.5665291,-52249.549736],
 [ 1670112000,282946.6577606,-48195.7292564],
 [ 1670025600,1566943.7505509,-704567.6106298],
 [ 1669939200,1085566.5166644,-91179.0602226],
 [ 1669852800,null,null],
 [ 1669766400,null,null],
 [ 1669680000,null,null],
 [ 1669593600,null,null]
]

I only asked for 10 points, because I only have a few days of data on this instance. The after is the number of seconds in 7 days.

This gives me one point per day (gtime = 24 * 3600 = 86400), doing the appropriate calculations.

11

Thank you for the suggestions. I will look into exporting data in CSV via the REST API. I should be able to do whatever I want in Excel after I get that. I knew that there was a possibility to do this with Grafana but I wanted to avoid that if possible. Grafana is a great product but I didn’t want this to get too clunky and awkward just to see how much data is passing through the gateway. As a last resort, I will keep Grafana in mind. Truthfully, now that I have found Netdata, I just want to stay with Netdata if I can. I’m a fan.

I have created a feature request for aggregation across time: [Feat]: Ability to Aggregate Across Time in Chart · Issue #14102 · netdata/netdata · GitHub

I am very impressed with the responsiveness of the people here. I can’t thank you enough. You have been very helpful!

12

The feature request for multiple OIDs in the config is here: [Feat]: Extend the configuration to allow setting multiple OIDs · Issue #14096 · netdata/netdata · GitHub

14

I’ll check into this too. Thanks!

15

@BigPines hey! if you are pulling from local agent api and would rather use something like Pandas dataframes in Python that might help take some of the pain out of things (depending on your set up).

Basically you could have a little python script that pulls from agent api what you need into a nice Pandas DataFrame which you can easily then save as CSV or anything else if you want for visualizing it etc.

Have a look and would be happy to help you out at bit if needed.

16

Thanks! I will look at this too. I’ll check everything available.