cat /var/log/netdata/collector.log | grep "softirq"
Completely empty.
grep -w proc /var/log/netdata/collector.log
Returns
2023-02-09 17:39:31: ERROR : MAIN : Cannot open /proc/1325956/status
2023-02-09 17:40:25: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1327355/status' (errno 2, No such file or directory)
2023-02-09 17:40:25: ERROR : MAIN : Cannot open /proc/1327355/status
2023-02-09 17:41:20: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1330880/status' (errno 2, No such file or directory)
2023-02-09 17:41:20: ERROR : MAIN : Cannot open /proc/1330880/status
2023-02-09 17:42:14: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1338159/status' (errno 2, No such file or directory)
2023-02-09 17:42:14: ERROR : MAIN : Cannot open /proc/1338159/status
2023-02-09 17:43:08: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1340535/status' (errno 2, No such file or directory)
2023-02-09 17:43:08: ERROR : MAIN : Cannot open /proc/1340535/status
2023-02-09 17:44:03: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1345362/status' (errno 2, No such file or directory)
2023-02-09 17:44:03: ERROR : MAIN : Cannot open /proc/1345362/status
2023-02-09 17:44:57: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1349008/status' (errno 2, No such file or directory)
2023-02-09 17:44:57: ERROR : MAIN : Cannot open /proc/1349008/status
2023-02-09 17:45:51: ERROR : MAIN : PROCFILE: Cannot open file '/proc/1350483/status' (errno 2, No such file or directory)
2023-02-09 17:45:51: ERROR : MAIN : Cannot open /proc/1350483/status
....
For more information
cat /var/log/netdata/collector.log | grep "proc" | grep -v "No such file" | grep -v "/status"
2023-02-09 06:59:30: nfacct.plugin INFO : MAIN : NFACCT process exiting
2023-02-09 07:09:42: netdata INFO : P[proc netdev] : cleaning up...
2023-02-09 07:09:45: netdata INFO : P[proc] : cleaning up...
2023-02-09 07:10:06: go.d INFO: main[main] using config: enabled 'true', default_run 'true', max_procs '0'
2023-02-09 07:10:07: go.d ERROR: prometheus[proc_exporter_local] Get "http://127.0.0.1:9227/metrics": dial tcp 127.0.0.1:9227: connect: connection refused
2023-02-09 07:10:07: go.d ERROR: prometheus[proc_exporter_local] check failed
2023-02-09 07:10:07: go.d ERROR: prometheus[vector_packet_process_vpp_exporter_local] Get "http://127.0.0.1:9482/metrics": dial tcp 127.0.0.1:9482: connect: connection refused
2023-02-09 07:10:07: go.d ERROR: prometheus[vector_packet_process_vpp_exporter_local] check failed
2023-02-09 07:10:07: go.d ERROR: prometheus[opvizor_performance_analyzer_process_exporter_local] Get "http://127.0.0.1:9585/metrics": dial tcp 127.0.0.1:9585: connect: connection refused
2023-02-09 07:10:07: go.d ERROR: prometheus[opvizor_performance_analyzer_process_exporter_local] check failed
2023-02-09 07:10:07: go.d ERROR: prometheus[exporter_for_grouped_process_local] Get "http://127.0.0.1:9644/metrics": dial tcp 127.0.0.1:9644: connect: connection refused
2023-02-09 07:10:07: go.d ERROR: prometheus[exporter_for_grouped_process_local] check failed
2023-02-09 07:10:07: go.d ERROR: prometheus[processor_counter_monitor_exporter_local] Get "http://127.0.0.1:9738/metrics": dial tcp 127.0.0.1:9738: connect: connection refused
2023-02-09 07:10:07: go.d ERROR: prometheus[processor_counter_monitor_exporter_local] check failed
2023-02-09 11:10:10: nfacct.plugin INFO : MAIN : NFACCT process exiting
2023-02-09 15:10:20: nfacct.plugin INFO : MAIN : NFACCT process exiting
This is super strange!!! because [GONE] Firewall(netfilter) > netlink > connection tracker `errors`/`searches` - #14 by Austin_Hemmelgarn in this post I got answer that nfacct isn’t being installed on netdata with default installation? But I have it! on 2 nodes.(because I can see plenty of information in connection tracker like search restart and e.t.c) And both of these nodes have bug mentioned in this topic.