prohibit netdata from scanning or binding to port range

Hello, general question

I have a fleet of servers w netdata agent, reporting to a netdata collector/master

all agents have the python and go plugins enabled

[plugins]
python.d = yes
go.d = yes

we have proprietary trading applications running on some of these rack servers, and I had ND running on them for at least a year, w no issues. but recently one of our app ports came up on port 27017 and i noticed netdata agent started trying to connect to it and query it for MongoDB data

it caused an issue due to heavy TCP load,

wondering if theres a way to prohibit ND from scanning and querying certain ports, like a range.

couldnt find anything in the documents.

Thanks

@perfecto25 This is due to port 27017 being the IANA-registered port for MongoDB. go.d.plugin has a service discovery component:

  • it finds that there is a service listens on 27107.
  • it doesn’t check anything else because that is a well-known port.
  • it creates a MongoDB data collection job.

Service discovery identification logic is go.d/sd/net_listeners.conf. If you want to change it - change or to and in this line.

What do you mean by that? It is just one query at the Netdata start.

yea i realized its a reserved port, dont think theres a way to tell netdata to not scan port range 27000-28000 for example

i generated plugin conf files for go.d and python.d and disabled all modules that i dont need, ie

vi /etc/netdata/go.d.conf


# Enable/disable specific g.d.plugin module
# If you want to change any value, you need to uncomment out it first.
# IMPORTANT: Do not remove all spaces, just remove # symbol. There should be a space before module name.
modules:
  activemq: no
  apache: no
  bind: yes
  chrony: no
  cockroachdb: no
  consul: no
  coredns: no
  couchbase: no
  couchdb: no
  dnsdist: no
  dnsmasq: no
  dnsmasq_dhcp: no
  dns_query: no
  docker: no
  docker_engine: no
  dockerhub: no
etc
mongodb: no

brought up a dummy 27017 port to simulate my custom app proc (same as mongodb port)

restarted netdata agent,

Im still seeing the mongodb module querying my dummy 27017 port

127.0.0.1 - - [16/Dec/2024 14:01:58] code 400, message Bad HTTP/0.9 request type ('æ\x00\x00\x00\x1b\x00\x00\x00\x00\x00\x00\x00Ô\x07\x00\x00\x04\x00\x00\x00admin.$cmd\x00\x00\x00\x00\x00ÿÿÿÿ¿\x00\x00\x00\x10isMaster\x00\x01\x00\x00\x00\x08helloOk\x00\x01\x04compression\x00\x05\x00\x00\x00\x00\x03client\x00\x88\x00\x00\x00\x03driver\x003\x00\x00\x00\x02name\x00\x10\x00\x00\x00mongo-go-driver\x00\x02version\x00\x07\x00\x00\x001.17.1\x00\x00\x03os\x00-\x00\x00\x00\x02type\x00\x06\x00\x00\x00linux\x00\x02architecture\x00\x06\x00\x00\x00amd64\x00\x00\x02platform\x00')
127.0.0.1 - - [16/Dec/2024 14:01:58] "æadmin.$cmdÿÿÿÿ¿isMastehelloOkcompressionclientdriver3namemongo-go-driverversion1.17.1os-typelinuxarchitectureamd64platform	go1.23.3" 400 -

ND quried my 27017 port ever 5 sec, causing our trading application to issue out tons of uncesseary requests and delay legitimate client TCP connections.

i turned off the ND agent and the TCP load went away right away. Just want to make sure that ND runs strictly on whitelisted ports and not surprise me by quering some service we dont even have if we spin up a business process on same port.

The problem isn’t with module enablement/disablement. Refer to my earlier message for the steps to disable 27017 discovery.

That was a bug. Fixed in 19232.

thanks, edited this file /usr/lib/netdata/conf.d/go.d/sd/net_listeners.conf, changed to ‘and’, seems it stopped querying on 27017

The next update will replace this file. Stable versions are unaffected (the bug will be fixed in the next stable release), but nightly builds will require you to manually copy the file to /etc/netdata/go.d/sd/ to preserve it.

awesome thanks for the update

sorry one more question, in the net listeners conf file, what is the point of having an OR statement here?

classify:
  - name: "Applications"
    selector: "unknown"
    tags: "-unknown app"
    match:
      - tags: "beanstalk"
        expr: '{{ or (eq .Port "11300") (eq .Comm "beanstalkd") }}'

wouldnt it be better to have AND everywhere to make sure its really querying the correct service?

I think the point is to create a data collection job for application X even if it doesn’t listen on the default port (it is always a configuration option and nothing stops you from starting multiple instances of an application).

Using an IANA-registered port for your custom app is a problem here (why would you do that? There are so many ports available), not discovery config.