# Mixed plain and SSLed data streaming issue

**URL:** <https://community.netdata.cloud/t/mixed-plain-and-ssled-data-streaming-issue/4209>\
**Category:** Help\
**Tags:** agent, streaming\
**Created:** [May 30, 2023, 12:05pm UTC](https://community.netdata.cloud/t/mixed-plain-and-ssled-data-streaming-issue/4209 "2023-05-30T12:05:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lex](https://avatars.discourse-cdn.com/v4/letter/l/c68b51/32.png) [@Lex](https://community.netdata.cloud/u/Lex)\
**Post date:** [May 30, 2023, 12:05pm UTC](https://community.netdata.cloud/t/mixed-plain-and-ssled-data-streaming-issue/4209/1 "2023-05-30T12:05:47Z")

</div>

I have a Netdata (v1.39.1) parent host collecting data from ~300 hosts (mostly v1.37.1).  
Currently data collection works well but via HTTP.  
Due to current security trends I would like to move to SSL.

As it is not that easy to reconfigure all 300 hosts in one moment I would like to apply “^SSL=optional”  
to have both HTTP and HTTPS streaming. As soon as all hosts are SSLed I switch to “^SSL=force”.

Here is HTTP config. All works well here.

```auto
[web]
    default port = 19999
    bind to = *:http=dashboard|management|netdata.conf netdata.mydomain.com:19999=streaming|registry
    web files owner = root
    web files group = netdata
    enable web responses gzip compression = no
# ssl key = /etc/netdata/ssl/mydomain.com.key
# ssl certificate = /etc/netdata/ssl/mydomain.com.crt

```

and here is new one. The only change is “^SSL=optional” and certs added.

```auto
[web]
    default port = 19999
    bind to = *:http=dashboard|management|netdata.conf netdata.mydomain.com:19999=streaming|registry^SSL=optional
    web files owner = root
    web files group = netdata
    enable web responses gzip compression = no
    ssl key = /etc/netdata/ssl/mydomain.com.key
    ssl certificate = /etc/netdata/ssl/mydomain.com.crt

```

As soon as I activate new config all streams to parent host stop working.  
Here is what child (UTC time zone) says about parent:

```auto
2023-05-30 09:29:25: netdata ERROR : STREAM_SENDER[systemAAc01.mydomain.com] : Clearing stream_collected_metrics flag in charts of host systemAAc01.mydomain.com
2023-05-30 09:29:25: netdata INFO : STREAM_SENDER[systemAAc01.mydomain.com] : STREAM systemAAc01.mydomain.com: attempting to connect to 'netdata.mydomain.com' (default port: 19999)...
2023-05-30 09:29:25: netdata INFO : STREAM_SENDER[systemAAc01.mydomain.com] : STREAM systemAAc01.mydomain.com [send to netdata.mydomain.com]: initializing communication...
2023-05-30 09:29:25: netdata INFO : STREAM_SENDER[systemAAc01.mydomain.com] : STREAM systemAAc01.mydomain.com [send to netdata.mydomain.com]: waiting response from remote netdata...
2023-05-30 09:29:26: netdata ERROR : STREAM_SENDER[systemAAc01.mydomain.com] : Clearing stream_collected_metrics flag in charts of host systemAAc01.mydomain.com
2023-05-30 09:29:26: netdata ERROR : STREAM_SENDER[systemAAc01.mydomain.com] : STREAM systemAAc01.mydomain.com [send to netdata.mydomain.com]: remote netdata does not respond.
2023-05-30 09:29:31: netdata ERROR : STREAM_SENDER[systemAAc01.mydomain.com] : Clearing stream_collected_metrics flag in charts of host systemAAc01.mydomain.com
2023-05-30 09:29:31: netdata INFO : STREAM_SENDER[systemAAc01.mydomain.com] : STREAM systemAAc01.mydomain.com: skipping destination 'netdata.mydomain.com' (default port: 19999) due to last error (code: -6, timeout while expecting first response), will retry it in 25 seconds

```

And here is parent log (CET time zone):

```auto
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : thread created with task id 83125
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : set name of thread 83125 to RCVR[systemAA
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : STREAM systemAAc01.mydomain.com [11.11.11.20]:43476: receive thread created (task id 83125)
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : Host systemAAc01.mydomain.com is not in archived mode anymore
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : STREAM systemAAc01.mydomain.com [receive from [11.11.11.20]:43476]: established link with negotiated capabilities: VCAPS HLABELS CLAIM CLABELS FUNCTIONS REPLICATION BINARY
2023-05-30 11:29:26: netdata ERROR : RCVR[systemAA : cannot write to SSL connection - connection is not ready.
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : STREAM 'systemAAc01.mydomain.com' [receive from [11.11.11.20]:43476]: cannot reply back. STATUS: CANT REPLY DROPPING CONNECTION
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : STREAM 'systemAAc01.mydomain.com' [receive from [11.11.11.20]:43476]: receive thread ended (task id 83125)
2023-05-30 11:29:26: netdata INFO : RCVR[systemAA : thread with task id 83125 finished

```

As you can see child and parent negotiated capabilities, thus I assume HTTP connection setup was completed successfully.  
But suddenly parent decides to talk back SSL to client: “cannot write to SSL connection - connection is not ready”.  
At this stage all falling apart and no streaming works.

Any hints what can be wrong with SSL configuration here?

**Environment/Browser/Agent’s version etc**

Parent streaming configuration:

```auto
$ cat /etc/netdata/stream.conf
[f1020749-6916-40fe-a425-ee2fc5fcaaf5]
    enabled = yes
    allow from = *
    default postpone alarms on connect seconds = 300
    enable compression = yes

```

```auto
$ netdata -W buildinfo
Version: netdata v1.39.1
Configure options: '--prefix=/usr' '--sbindir=/usr/bin' '--sysconfdir=/etc' '--libexecdir=/usr/lib' '--localstatedir=/var' '--with-zlib' '--with-math' '--with-user=netdata' 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/netdata/src=/usr/src/debug/netdata -flto=auto' 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto' 'CXXFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -g -ffile-prefix-map=/build/netdata/src=/usr/src/debug/netdata -flto=auto'
Install type: custom
Features:
    dbengine: YES
    Native HTTPS: YES
    Netdata Cloud: YES
    ACLK: YES
    TLS Host Verification: YES
    Machine Learning: YES
    Stream Compression: YES
Libraries:
    protobuf: YES (system)
    jemalloc: NO
    JSON-C: YES
    libcap: YES
    libcrypto: YES
    libm: YES
    tcalloc: NO
    zlib: YES
Plugins:
    apps: YES
    cgroup Network Tracking: YES
    CUPS: YES
    EBPF: NO
    IPMI: NO
    NFACCT: YES
    perf: YES
    slabinfo: YES
    Xen: NO
    Xen VBD Error Tracking: NO
Exporters:
    AWS Kinesis: NO
    GCP PubSub: NO
    MongoDB: YES
    Prometheus Remote Write: YES
Debug/Developer Features:
    Trace Allocations: NO

```

```auto
$ openssl version
OpenSSL 3.0.8 7 Feb 2023 (Library: OpenSSL 3.0.8 7 Feb 2023)

```

```auto
$ ldd /usr/bin/netdata | grep ssl
        libssl.so.3 => /usr/lib/libssl.so.3 (0x00007f3b53d38000)

```

Child configuration:

```auto
$ cat /etc/netdata/netdata.conf
[global]
        memory mode = none
        update every = 4

[web]
        mode = none

[registry]
        enabled = no
        registry to announce = http://netdata.mydomain.com:19999

[statsd]
        enabled = no

```

```auto
$ cat /etc/netdata/stream.conf
[stream]
    enabled = yes
    destination = netdata.mydomain.com
    api key = f1020749-6916-40fe-a425-ee2fc5fcaaf5

```

```auto
# netdata -W buildinfo
Version: netdata v1.37.1
Configure options: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-plugin-freeipmi' '--with-bundled-protobuf' '--with-zlib' '--with-math' '--with-user=netdata' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' 'LDFLAGS=-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
Install type: custom
Features:
    dbengine: YES
    Native HTTPS: YES
    Netdata Cloud: YES
    ACLK: YES
    TLS Host Verification: YES
    Machine Learning: YES
    Stream Compression: NO
Libraries:
    protobuf: YES (bundled)
    jemalloc: NO
    JSON-C: YES
    libcap: YES
    libcrypto: YES
    libm: YES
    tcalloc: NO
    zlib: YES
Plugins:
    apps: YES
    cgroup Network Tracking: YES
    CUPS: NO
    EBPF: NO
    IPMI: YES
    NFACCT: NO
    perf: YES
    slabinfo: YES
    Xen: NO
    Xen VBD Error Tracking: NO
Exporters:
    AWS Kinesis: NO
    GCP PubSub: NO
    MongoDB: NO
    Prometheus Remote Write: YES
Debug/Developer Features:
    Trace Allocations: NO

```

```auto
$ openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017

```

```auto
$ ldd /sbin/netdata | grep ssl
        libssl.so.10 => /lib64/libssl.so.10 (0x00007faa5d656000)

```

---

<div class="post-metadata">

**Author:** ![Manolis\_Vasilakis](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/manolis_vasilakis/32/820_2.png) [@Manolis\_Vasilakis](https://community.netdata.cloud/u/Manolis_Vasilakis)\
**Post date:** [May 31, 2023, 7:22am UTC](https://community.netdata.cloud/t/mixed-plain-and-ssled-data-streaming-issue/4209/2 "2023-05-31T07:22:37Z")

</div>

Hi @lex sorry for the late reply. Just to note that we are looking into it and will come back with more info. Thank you!

---

<div class="post-metadata">

**Author:** ![Lex](https://avatars.discourse-cdn.com/v4/letter/l/c68b51/32.png) [@Lex](https://community.netdata.cloud/u/Lex)\
**Post date:** [June 16, 2023, 10:37am UTC](https://community.netdata.cloud/t/mixed-plain-and-ssled-data-streaming-issue/4209/3 "2023-06-16T10:37:42Z")

</div>

Parent update to netdata v1.40.0 greatly improved situation.  
Streaming with SSL=optional works well with SSLed and plain children.  
No update on children was needed, still have most on 1.37.1.
