Netdata Community

EOF found in spawn pipe (Netdata fails to start)

Verify first that your question wasn’t asked before on the forum:

As far as I can tell, yes.

Briefly explain what is the problem you are having:

Two days ago a node entered an error state and now it refuses to bring up netdata. The issue appears to me to be because netdata is loading a stock config with an EOF. I’ve read the stock config file located at /usr/lib/netdata/conf.d/stream.conf and as far as I can tell there’s not an incorrect EOF in that file?

The service status is:
Active: activating (auto-restart) (Result: core-dump) since Mon 2021-09-20 16:48:31 UTC; 18s ago
Processing: error.log…
Process: 8209 ExecStart=/usr/sbin/netdata -D $EXTRA_OPTS (code=dumped, signal=SEGV)

Error.log (All this node does is repeat this on loop:

2021-09-20 16:31:48: netdata INFO : MAIN : resources control: allowed file descriptors: soft = 1024, max = 4096
2021-09-20 16:31:48: netdata INFO : MAIN : Out-Of-Memory (OOM) score is already set to the wanted value 1000
2021-09-20 16:31:48: netdata INFO : MAIN : Adjusted netdata scheduling policy to idle (5), with priority 0.
2021-09-20 16:31:48: netdata INFO : MAIN : Running with process scheduling policy ‘idle’
2021-09-20 16:31:48: netdata INFO : MAIN : netdata started on pid 1110.
2021-09-20 16:31:48: netdata INFO : MAIN : Initializing spawn client.
2021-09-20 16:31:48: netdata INFO : MAIN : Executing /usr/libexec/netdata/plugins.d/system-info.sh
Spawn server is up.
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_NAME=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID_LIKE=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION_ID=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_CONTAINER_OS_DETECTION=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_NAME=Ubuntu
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_ID=ubuntu
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_ID_LIKE=debian
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION=18.04.5 LTS (Bionic Beaver)
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION_ID=18.04
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_OS_DETECTION=/etc/os-release
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_HOST_IS_K8S_NODE=false
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_NAME=Linux
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_VERSION=4.15.0-156-generic
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_ARCHITECTURE=x86_64
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_VIRTUALIZATION=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_VIRT_DETECTION=systemd-detect-virt
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER=none
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER_DETECTION=systemd-detect-virt
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_LOGICAL_CPU_COUNT=16
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_VENDOR=GenuineIntel
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_MODEL=Intel(R) Core™ i7-10700T CPU @ 2.00GHz
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_FREQ=4500000000
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_DETECTION=lscpu
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_RAM=16547028992
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_RAM_DETECTION=procfs
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_DISK_SIZE=0
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA_SYSTEM_DISK_DETECTION=sysfs
2021-09-20 16:31:48: netdata INFO : MAIN : SQLite database /var/cache/netdata/netdata-meta.db initialization
2021-09-20 16:31:48: netdata INFO : MAIN : SQLite database initialization completed
2021-09-20 16:31:48: netdata INFO : MAIN : Cannot open the file /var/lib/netdata/health.silencers.json, so Netdata will work with the default health configuration.
2021-09-20 16:31:48: netdata INFO : MAIN : CONFIG: cannot load user config ‘/etc/netdata/stream.conf’. Will try stock config.
EOF found in spawn pipe.
Shutting down spawn server event loop.
Shutting down spawn server loop complete.

It has nothing to do with stream.conf, the error comes from the function in netdata/spawn_server.c at master · netdata/netdata · GitHub
Asking someone to take a look

1 Like