# EOF found in spawn pipe (Netdata fails to start)

**URL:** https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750
**Category:** Help
**Tags:** agent
**Created:** [September 20, 2021, 4:52pm UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750 "2021-09-20T16:52:11Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Mbrantley](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/mbrantley/32/983_2.png) [@Mbrantley](https://community.netdata.cloud/u/Mbrantley)
#### Post date: [September 20, 2021, 4:52pm UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/1 "2021-09-20T16:52:11Z")

</div>

Verify first that your question wasn’t asked before on the forum:

As far as I can tell, yes.

Briefly explain what is the problem you are having:

Two days ago a node entered an error state and now it refuses to bring up netdata. The issue appears to me to be because netdata is loading a stock config with an EOF. I’ve read the stock config file located at /usr/lib/netdata/conf.d/stream.conf and as far as I can tell there’s not an incorrect EOF in that file?

The service status is:  
Active: activating (auto-restart) (Result: core-dump) since Mon 2021-09-20 16:48:31 UTC; 18s ago  
Processing: error.log…  
Process: 8209 ExecStart=/usr/sbin/netdata -D $EXTRA\_OPTS (code=dumped, signal=SEGV)

Error.log (All this node does is repeat this on loop:

2021-09-20 16:31:48: netdata INFO : MAIN : resources control: allowed file descriptors: soft = 1024, max = 4096  
2021-09-20 16:31:48: netdata INFO : MAIN : Out-Of-Memory (OOM) score is already set to the wanted value 1000  
2021-09-20 16:31:48: netdata INFO : MAIN : Adjusted netdata scheduling policy to idle (5), with priority 0.  
2021-09-20 16:31:48: netdata INFO : MAIN : Running with process scheduling policy ‘idle’  
2021-09-20 16:31:48: netdata INFO : MAIN : netdata started on pid 1110.  
2021-09-20 16:31:48: netdata INFO : MAIN : Initializing spawn client.  
2021-09-20 16:31:48: netdata INFO : MAIN : Executing /usr/libexec/netdata/plugins.d/system-info.sh  
Spawn server is up.  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_NAME=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_ID=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_ID\_LIKE=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_VERSION=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_VERSION\_ID=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_CONTAINER\_OS\_DETECTION=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_NAME=Ubuntu  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_ID=ubuntu  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_ID\_LIKE=debian  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_VERSION=18.04.5 LTS (Bionic Beaver)  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_VERSION\_ID=18.04  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_OS\_DETECTION=/etc/os-release  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_HOST\_IS\_K8S\_NODE=false  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_KERNEL\_NAME=Linux  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_KERNEL\_VERSION=4.15.0-156-generic  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_ARCHITECTURE=x86\_64  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_VIRTUALIZATION=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_VIRT\_DETECTION=systemd-detect-virt  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CONTAINER=none  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CONTAINER\_DETECTION=systemd-detect-virt  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CPU\_LOGICAL\_CPU\_COUNT=16  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CPU\_VENDOR=GenuineIntel  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CPU\_MODEL=Intel(R) Core™ i7-10700T CPU @ 2.00GHz  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CPU\_FREQ=4500000000  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_CPU\_DETECTION=lscpu  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_TOTAL\_RAM=16547028992  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_RAM\_DETECTION=procfs  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_TOTAL\_DISK\_SIZE=0  
2021-09-20 16:31:48: netdata INFO : MAIN : NETDATA\_SYSTEM\_DISK\_DETECTION=sysfs  
2021-09-20 16:31:48: netdata INFO : MAIN : SQLite database /var/cache/netdata/netdata-meta.db initialization  
2021-09-20 16:31:48: netdata INFO : MAIN : SQLite database initialization completed  
2021-09-20 16:31:48: netdata INFO : MAIN : Cannot open the file /var/lib/netdata/health.silencers.json, so Netdata will work with the default health configuration.  
2021-09-20 16:31:48: netdata INFO : MAIN : CONFIG: cannot load user config ‘/etc/netdata/stream.conf’. Will try stock config.  
EOF found in spawn pipe.  
Shutting down spawn server event loop.  
Shutting down spawn server loop complete.

---

<div class="post-metadata">

### Author: ![Christopher\_Akritid1](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/christopher_akritid1/32/419_2.png) [@Christopher\_Akritid1](https://community.netdata.cloud/u/Christopher_Akritid1)
#### Post date: [September 20, 2021, 5:29pm UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/2 "2021-09-20T17:29:39Z")

</div>

It has nothing to do with stream.conf, the error comes from the function in [netdata/spawn\_server.c at master · netdata/netdata · GitHub](https://github.com/netdata/netdata/blob/master/spawn/spawn_server.c#L241)  
Asking someone to take a look

---

<div class="post-metadata">

### Author: ![gfay63](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/gfay63/32/1604_2.png) [@gfay63](https://community.netdata.cloud/u/gfay63)
#### Post date: [March 25, 2022, 10:07pm UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/3 "2022-03-25T22:07:46Z")

</div>

I now have this exact same issue. Everything was working fine before…

```auto
2022-03-25 17:05:01: netdata INFO : MAIN : TIMEZONE: using strftime(): 'CDT'

2022-03-25 17:05:01: netdata INFO : MAIN : TIMEZONE: fixed as 'CDT'

2022-03-25 17:05:01: netdata INFO : MAIN : SIGNAL: Not enabling reaper

2022-03-25 17:05:01: netdata INFO : MAIN : resources control: allowed file descriptors: soft = 256, max = 9223372036854775807

2022-03-25 17:05:01: netdata ERROR : MAIN : Out-Of-Memory (OOM) score setting is not supported on this system. (errno 2, No such file or directory)

2022-03-25 17:05:01: netdata INFO : MAIN : netdata started on pid 4560.

2022-03-25 17:05:01: netdata INFO : MAIN : Initializing spawn client.

2022-03-25 17:05:01: netdata INFO : MAIN : Executing /usr/local/netdata/usr/libexec/netdata/plugins.d/system-info.sh

Spawn server is up.

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_NAME=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID_LIKE=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION_ID=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_OS_DETECTION=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_CONTAINER_IS_OFFICIAL_IMAGE=false

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_NAME=mac

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_ID=macOS

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_ID_LIKE=mac

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION=12.2.1

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION_ID=unknown

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_OS_DETECTION=sw_vers

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_HOST_IS_K8S_NODE=false

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_NAME=Darwin

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_VERSION=21.3.0

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_ARCHITECTURE=x86_64

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_VIRTUALIZATION=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_VIRT_DETECTION=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER=unknown

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER_DETECTION=none

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_LOGICAL_CPU_COUNT=8

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_VENDOR=GenuineIntel

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_MODEL=Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_FREQ=3700000000

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_DETECTION=nproc sysctl

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_RAM=2147483648

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_RAM_DETECTION=sysctl

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_DISK_SIZE=3002231670652

2022-03-25 17:05:01: netdata INFO : MAIN : NETDATA_SYSTEM_DISK_DETECTION=df

2022-03-25 17:05:01: netdata INFO : MAIN : SQLite database /usr/local/netdata/var/cache/netdata/netdata-meta.db initialization

2022-03-25 17:05:01: netdata INFO : MAIN : SQLite database initialization completed

2022-03-25 17:05:01: netdata INFO : MAIN : Cannot open the file /usr/local/netdata/var/lib/netdata/health.silencers.json, so Netdata will work with the default health configuration.

2022-03-25 17:05:01: netdata INFO : MAIN : CONFIG: cannot load user config '/usr/local/netdata/etc/netdata/stream.conf'. Will try stock config.

EOF found in spawn pipe.

Shutting down spawn server event loop.

Shutting down spawn server loop complete.

```

---

<div class="post-metadata">

### Author: ![Manolis\_Vasilakis](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/manolis_vasilakis/32/820_2.png) [@Manolis\_Vasilakis](https://community.netdata.cloud/u/Manolis_Vasilakis)
#### Post date: [March 28, 2022, 9:44am UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/4 "2022-03-28T09:44:33Z")

</div>

Hi @gfay63 !

Can you please share more of the situation? Have you updated Netdata recently? Also, if it’s possible can you share the output of `netdata -W buildinfo` ?

Thanks!!

---

<div class="post-metadata">

### Author: ![mjtice](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/mjtice/32/510_2.png) [@mjtice](https://community.netdata.cloud/u/mjtice)
#### Post date: [May 17, 2022, 7:57pm UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/5 "2022-05-17T19:57:50Z")

</div>

Hi guys, we’re getting this as well on one of our nodes. Here’s my `netdata -W buildinfo`

```auto
Version: netdata v1.34.1
Configure options: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-plugin-freeipmi' '--with-bundled-protobuf' '--with-zlib' '--with-math' '--with-user=netdata' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' 'LDFLAGS=-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
Install type: custom
Features:
    dbengine: YES
    Native HTTPS: YES
    Netdata Cloud: YES
    ACLK Next Generation: YES
    ACLK-NG New Cloud Protocol: YES
    ACLK Legacy: NO
    TLS Host Verification: YES
    Machine Learning: YES
    Stream Compression: NO
Libraries:
    protobuf: YES (bundled)
    jemalloc: NO
    JSON-C: YES
    libcap: YES
    libcrypto: YES
    libm: YES
    tcalloc: NO
    zlib: YES
Plugins:
    apps: YES
    cgroup Network Tracking: YES
    CUPS: NO
    EBPF: NO
    IPMI: YES
    NFACCT: NO
    perf: YES
    slabinfo: YES
    Xen: NO
    Xen VBD Error Tracking: NO
Exporters:
    AWS Kinesis: NO
    GCP PubSub: NO
    MongoDB: NO
    Prometheus Remote Write: YES

```

This is on

```auto
# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

```

What’s interesting is that this is intermittent. Suddenly it (netdata) starts working again only to have it crash some time later.

I’ve definitely have had this issue pop up before but coincidentally this instance was upgraded to the netdata version above on 12-May.

---

<div class="post-metadata">

### Author: ![Manolis\_Vasilakis](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/manolis_vasilakis/32/820_2.png) [@Manolis\_Vasilakis](https://community.netdata.cloud/u/Manolis_Vasilakis)
#### Post date: [May 20, 2022, 6:38am UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/6 "2022-05-20T06:38:00Z")

</div>

Hi @mjtice !

Is it possible to share an `error.log` from a time when the crash occurred? What was the previous version of Netdata before the update?

Thanks!

---

<div class="post-metadata">

### Author: ![danesparza](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/danesparza/32/1849_2.png) [@danesparza](https://community.netdata.cloud/u/danesparza)
#### Post date: [May 26, 2022, 8:13am UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/7 "2022-05-26T08:13:40Z")

</div>

I’m also getting this issue on MacOS. Here is my netdata -W buildinfo output:

```auto
Version: netdata v1.34.1
Configure options: '--disable-dependency-tracking' '--disable-silent-rules' '--prefix=/usr/local/Cellar/netdata/1.34.1' '--sysconfdir=/usr/local/etc' '--localstatedir=/usr/local/var' '--libexecdir=/usr/local/Cellar/netdata/1.34.1/libexec' '--with-math' '--with-zlib' '--enable-dbengine' '--with-user=netdata' 'UUID_LIBS=-lc' 'UUID_CFLAGS=-I/usr/include' 'CC=clang' 'CFLAGS=-I/private/tmp/netdata-20220422-87515-1b18ofa/netdata-v1.34.1/resources/judy/include' 'LDFLAGS=-L/private/tmp/netdata-20220422-87515-1b18ofa/netdata-v1.34.1/resources/judy/lib' 'CXX=clang++' 'PKG_CONFIG_PATH=/usr/local/opt/json-c/lib/pkgconfig:/usr/local/opt/libuv/lib/pkgconfig:/usr/local/opt/lz4/lib/pkgconfig:/usr/local/opt/openssl@1.1/lib/pkgconfig:/usr/local/opt/protobuf/lib/pkgconfig:/usr/local/opt/protobuf-c/lib/pkgconfig' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/12'
Install type: custom
Features:
    dbengine: YES
    Native HTTPS: YES
    Netdata Cloud: YES
    ACLK Next Generation: YES
    ACLK-NG New Cloud Protocol: YES
    ACLK Legacy: NO
    TLS Host Verification: YES
    Machine Learning: YES
    Stream Compression: YES
Libraries:
    protobuf: YES (system)
    jemalloc: NO
    JSON-C: YES
    libcap: NO
    libcrypto: YES
    libm: YES
    tcalloc: NO
    zlib: YES
Plugins:
    apps: NO
    cgroup Network Tracking: NO
    CUPS: YES
    EBPF: NO
    IPMI: NO
    NFACCT: NO
    perf: NO
    slabinfo: NO
    Xen: NO
    Xen VBD Error Tracking: NO
Exporters:
    AWS Kinesis: NO
    GCP PubSub: NO
    MongoDB: NO
    Prometheus Remote Write: NO

```

---

<div class="post-metadata">

### Author: ![Manolis\_Vasilakis](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/manolis_vasilakis/32/820_2.png) [@Manolis\_Vasilakis](https://community.netdata.cloud/u/Manolis_Vasilakis)
#### Post date: [May 26, 2022, 8:28am UTC](https://community.netdata.cloud/t/eof-found-in-spawn-pipe-netdata-fails-to-start/1750/8 "2022-05-26T08:28:41Z")

</div>

Hi @danesparza ! Thank you for the report.

Can you please share the `error.log` preferably at the time of the crash? Is it happening on startup, or after a while of usage?

Thanks!
