Netdata Community

Unable to see child pods/nodes in Netdata Cloud

Problem/Question

I am trying to setup Netdata on my kubernetes cluster. I am running into 2 separate issues when trying to access the node. When accessing from Netdata Cloud, I can only see metrics about the parent pod, no information about the child pods. This is similar to this issue, but I am curious if this has changed in the past few months. I was able to get it to work properly back in May of this year (the child pods showed up in Netdata Cloud).

The second issue I’m running into is that whenever I connect to the node directly, I do not see any charts or information show up on the page, including the information I am able to see in Netdata Cloud. I have checked the browser console for errors and do not see anything obvious in the logs for the child or parent pods.

Environment/Browser

Netdata is running in a single node, fresh kubernetes cluster (running with kubelet). Tested with Firefox 91.0.2 and Chromium 92.0.4515.159.

What I expected to happen

I expect the node to appear as is described in this guide for the Cloud installation, and I expect to be able to see at least the same metrics that are available in Netdata Cloud when I connect to the node directly.

Scratch the issue I was running into with not seeing any metrics when connecting directly. Turns out it was a configuration issue with my overrides. I am still experiencing the issue where I do not see the metrics on Netdata Cloud.

For another few weeks you still need to claim the children as well, to see their metrics. We have the params for the children documented in the helm chart, see token and rooms as the parent.

We are in the last stages of testing and fixes for the redesign that resolves this issue. We will roll it out gradually. If you’re interested in being in the first wave of users who will get the functionality, let us know here and we will contact you with more info as soon as we are ready.

I would love to be one of the first users to try the new method of claiming. I currently have the child set to claim with the same token/rooms as the parent and claiming is enabled. I noticed in the helm chart readme, it still lists the rooms to add the node to as child.claiming.room, but neither worked.

Wait, you mean you see the children and the parent in the general room but not in other rooms? If that’s the case, we need to fix it immediately. The changes I mentioned are so you don’t HAVE to claim the children, in order to see their data, i.e. to make that data accessible via the parent. Please provide some more info on your values.yaml (or the helm params), and describe exactly what you see, we need to investigate.

I see the parent in all the rooms it is configured to be in, but I do not see the children in any rooms. I am using ansible to deploy Netdata to my cluster, but I have tested using helm directly and end up with the same result. I have also just tried doing the same deployment on a different cluster, and that cluster showed up correctly in Netdata Cloud so it appears to be something specific with this cluster.
Kubernetes versions:
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.10", GitCommit:"8152330a2b6ca3621196e62966ef761b8f5a61bb", GitTreeState:"clean", BuildDate:"2021-08-11T18:00:37Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
CNI Plugin: Calico
Calico Version: v3.20.0

Let me know if there’s any more information you need.

If you don’t see the children listed in the “Nodes” tab of the “General” room, the most likely issue is that claiming fails, probably due to wrong settings in the following:

That’s why I asked for values.yaml or settings (not the specific ids of course). But if you can get them from another cluster and haven’t made any mistakes with the helm chart on this cluster, it’s something we haven’t seen before. Asked some more people to see how we can help you get this fixed, but can you definitively verify that the claiming info is properly provided for the children?

Oh sorry, I missed the values.yaml part. Here is what I’m overriding in the values.yaml file.

parent:
  alarms:
    storageclass: nfs-client
  database:
    storageclass: nfs-client
    volumesize: 12Gi
  claiming:
    enabled: true
    token: "{{ netdata_token }}"
    rooms: "{{ netdata_rooms }}"
child:
  claiming:
    enabled: true
    token: "{{ netdata_token }}"
    rooms: "{{ netdata_rooms }}"
  configs:
    netdata:
      data: |
        [global]
          memory mode = dbengine
          page cache size = 32
          dbengine multihost disk space = 9888
        [health]
          enabled = yes

Again, using this exact same configuration on a different cluster worked fine, so it seems to be specific to my environment. I have also verified that the token is set in the container (the value is set for the NETDATA_CLAIM_TOKEN env var).

Something I’d also like to mention while it’s relevant, the documentation for childclaiming.roomsis still listed aschild.claiming.rooms` in the values.yaml in the repo, and that is the value that works for configuring it with the overrides.

The only next step I can think of is to get the logs from the netdata child pods, as they get started. That’s when claiming happens.

I’m trying to imagine a way in which the parent can successfully connect and the child on the same exact node can’t and I’m coming up with a blank. Perhaps there’s something with k8s or docker that I don’t know though. See if you can see any errors during claiming at startup and I’ll ask a few more people for ideas.

Here’s the first 100 instances of ERROR in the logs of the child’s netdata container.

2021-09-06 07:40:32: netdata ERROR : MAIN : Cannot adjust netdata scheduling policy to idle (5), with priority 0. Falling back to nice. (errno 38, Function not implemented)
2021-09-06 07:40:32: netdata ERROR : MAIN : Cannot get my current process scheduling policy. (errno 38, Function not implemented)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : STREAM dallas-westegg-master [send]: not ready - discarding collected metrics.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/dev/termination-log' (disk '_dev_termination-log', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/containers/netdata/19b8a86b') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/hosts' (disk '_etc_hosts', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/etc-hosts') is not a directory. (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/hostname' (disk '_etc_hostname', filesystem 'ext4', root '/var/lib/docker/containers/50e95254806a8cc2ce9e53f827f273b021d4a7eaf9eaf8e676837ca58008dd5f/hostname') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/resolv.conf' (disk '_etc_resolv.conf', filesystem 'ext4', root '/var/lib/docker/containers/50e95254806a8cc2ce9e53f827f273b021d4a7eaf9eaf8e676837ca58008dd5f/resolv.conf') is not a directory. (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : STATSD : STREAM dallas-westegg-master [send]: not ready - discarding collected metrics.
2021-09-06 07:40:34: netdata ERROR : ACLK_Stats : STREAM dallas-westegg-master [send]: not ready - discarding collected metrics.
2021-09-06 07:40:34: netdata ERROR : GLOBAL_STATS : STREAM dallas-westegg-master [send]: not ready - discarding collected metrics.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/netdata/netdata.conf' (disk '_etc_netdata_netdata.conf', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/volumes/kubernetes.io~configmap/config/..2021_09_06_07_40_28.190633309/netdata') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/netdata/go.d.conf' (disk '_etc_netdata_go.d.conf', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/volumes/kubernetes.io~configmap/config/..2021_09_06_07_40_28.190633309/go.d') is not a directory. (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/run/docker.sock' (disk '_run_docker.sock', filesystem 'tmpfs', root '/docker.sock') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/netdata/stream.conf' (disk '_etc_netdata_stream.conf', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/volumes/kubernetes.io~configmap/config/..2021_09_06_07_40_28.190633309/stream') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/netdata/go.d/k8s_kubeproxy.conf' (disk '_etc_netdata_go.d_k8s_kubeproxy.conf', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/volumes/kubernetes.io~configmap/config/..2021_09_06_07_40_28.190633309/kubeproxy') is not a directory. (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[diskspace] : DISKSPACE: Mount point '/etc/netdata/go.d/k8s_kubelet.conf' (disk '_etc_netdata_go.d_k8s_kubelet.conf', filesystem 'ext4', root '/var/lib/kubelet/pods/bc5fae68-bbdb-4491-8aef-2273aff760d2/volumes/kubernetes.io~configmap/config/..2021_09_06_07_40_28.190633309/kubelet') is not a directory.
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[perf] : '/usr/libexec/netdata/plugins.d/perf.plugin' (pid 48083) disconnected after 0 successful data collections (ENDs). (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[perf] : child pid 48083 exited with code 1.
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[perf] : '/usr/libexec/netdata/plugins.d/perf.plugin' (pid 48083) exited with error code 1 and haven't collected any data. Disabling it. (errno 22, Invalid argument)
2021-09-06 07:40:34: freeipmi.plugin ERROR : MAIN : 2021-09-06 07:40:34: netdata INFO  : ACLK_Main : Using challenge-response: 9fb94afa-b43d-11eb-ada1-44a8421fa09d / BYE7rOnyfxrJs/Z6TcEQWKpixak=
2021-09-06 07:40:34: freeipmi.plugin ERROR : MAIN : ipmi_monitoring_sensor_readings_by_record_id(): internal system error (errno 2, No such file or directory)
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[freeipmi] : read failed: end of file (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[freeipmi] : '/usr/libexec/netdata/plugins.d/freeipmi.plugin' (pid 48108) disconnected after 0 successful data collections (ENDs). (errno 22, Invalid argument)
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[freeipmi] : child pid 48108 exited with code 1.
2021-09-06 07:40:34: netdata ERROR : PLUGINSD[freeipmi] : '/usr/libexec/netdata/plugins.d/freeipmi.plugin' (pid 48108) exited with error code 1 and haven't collected any data. Disabling it. (errno 22, Invalid argument)
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d.conf'.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d/stiebeleltron.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d/stiebeleltron.conf'.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read stock-configuration file /usr/lib/netdata/conf.d/node.d/stiebeleltron.conf: ENOENT: no such file or directory, open '/usr/lib/netdata/conf.d/node.d/stiebeleltron.conf', using internal defaults.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d/snmp.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d/snmp.conf'.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read stock-configuration file /usr/lib/netdata/conf.d/node.d/snmp.conf: ENOENT: no such file or directory, open '/usr/lib/netdata/conf.d/node.d/snmp.conf', using internal defaults.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d/sma_webbox.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d/sma_webbox.conf'.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read stock-configuration file /usr/lib/netdata/conf.d/node.d/sma_webbox.conf: ENOENT: no such file or directory, open '/usr/lib/netdata/conf.d/node.d/sma_webbox.conf', using internal defaults.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d/named.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d/named.conf'.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read stock-configuration file /usr/lib/netdata/conf.d/node.d/named.conf: ENOENT: no such file or directory, open '/usr/lib/netdata/conf.d/node.d/named.conf', using internal defaults.
2021-09-06 07:40:34: netdata INFO  : PLUGIN[proc] : 2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read user-configuration file /etc/netdata/node.d/fronius.conf: ENOENT: no such file or directory, open '/etc/netdata/node.d/fronius.conf'.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file '/host/proc/pressure/cpu' (errno 2, No such file or directory)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /host/proc/pressure/cpu.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file '/host/proc/pressure/memory' (errno 2, No such file or directory)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /host/proc/pressure/memory.
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file '/host/proc/pressure/io' (errno 2, No such file or directory)
2021-09-06 07:40:34: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /host/proc/pressure/io.
2021-09-06 07:40:34: node.d.plugin: ERROR: Cannot read stock-configuration file /usr/lib/netdata/conf.d/node.d/fronius.conf: ENOENT: no such file or directory, open '/usr/lib/netdata/conf.d/node.d/fronius.conf', using internal defaults.
2021-09-06 07:40:34: node.d.plugin: ERROR: named: local: Failed to make request, message: connect ECONNREFUSED 127.0.0.1:8888
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[fping] : '/usr/libexec/netdata/plugins.d/fping.plugin' (pid 48160) disconnected after 0 successful data collections (ENDs). (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[fping] : child pid 48160 exited with code 1.
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[fping] : '/usr/libexec/netdata/plugins.d/fping.plugin' (pid 48160) exited with error code 1 and haven't collected any data. Disabling it. (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : STREAM_SENDER[dallas-westegg-master] : Failed to connect to '10.101.104.122', port '19999' (errno 111, Connection refused)
2021-09-06 07:40:35: netdata ERROR : STREAM_SENDER[dallas-westegg-master] : STREAM dallas-westegg-master [send to netdata:19999]: failed to connect
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[ioping] : '/usr/libexec/netdata/plugins.d/ioping.plugin' (pid 48170) disconnected after 0 successful data collections (ENDs). (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[ioping] : child pid 48170 exited with code 1.
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[ioping] : '/usr/libexec/netdata/plugins.d/ioping.plugin' (pid 48170) exited with error code 1 and haven't collected any data. Disabling it. (errno 22, Invalid argument)
2021-09-06 07:40:35: apps.plugin ERROR : MAIN : PROCFILE: Cannot open file '/etc/netdata/apps_groups.conf' (errno 2, No such file or directory)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface eno2 duplex state by reading '/host/sys/class/net/eno2/duplex'. I will stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface eno2 carrier state by reading '/host/sys/class/net/eno2/carrier'. Stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: go.d ERROR: powerdns_recursor[local] error on HTTP request 'http://127.0.0.1:8081/api/v1/servers/localhost/statistics': Get "http://127.0.0.1:8081/api/v1/servers/localhost/statistics": dial tcp 127.0.0.1:8081: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: powerdns_recursor[local] check failed
2021-09-06 07:40:35: go.d ERROR: phpdaemon[local] error on request : Get "http://127.0.0.1:8509/FullStatus": dial tcp 127.0.0.1:8509: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: phpdaemon[local] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[nginx] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[nginx] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[apache] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[apache] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[apache] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[apache_vhosts] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[gunicorn] check failed
2021-09-06 07:40:35: go.d ERROR: web_log[gunicorn] check failed
2021-09-06 07:40:35: go.d ERROR: k8s_kubelet[k8s_kubelet] Get "http://127.0.0.1:10255/metrics": dial tcp 127.0.0.1:10255: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: k8s_kubelet[k8s_kubelet] check failed
2021-09-06 07:40:35: netdata ERROR : PLUGINSD[node.d] : '/usr/libexec/netdata/plugins.d/node.d.plugin' (pid 48110) disconnected after 0 successful data collections (ENDs). (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface enp10s0f0 duplex state by reading '/host/sys/class/net/enp10s0f0/duplex'. I will stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface enp10s0f0 carrier state by reading '/host/sys/class/net/enp10s0f0/carrier'. Stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface enp10s0f1 duplex state by reading '/host/sys/class/net/enp10s0f1/duplex'. I will stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: netdata ERROR : PLUGIN[proc] : Cannot refresh interface enp10s0f1 carrier state by reading '/host/sys/class/net/enp10s0f1/carrier'. Stop updating it. (errno 22, Invalid argument)
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] http://localhost/status?full&json returned HTTP status 404
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] check failed
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] http://127.0.0.1/status?full&json returned HTTP status 404
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] check failed
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] http://[::1]/status?full&json returned HTTP status 404
2021-09-06 07:40:35: go.d ERROR: phpfpm[local] check failed
2021-09-06 07:40:35: go.d ERROR: couchbase[local] error on scraping couchbase: error on HTTP request 'http://127.0.0.1:8091/pools/default/buckets?skipMap=true': Get "http://127.0.0.1:8091/pools/default/buckets?skipMap=true": dial tcp 127.0.0.1:8091: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: couchbase[local] check failed
2021-09-06 07:40:35: go.d ERROR: dnsdist[local] error on HTTP request 'http://127.0.0.1:8083/jsonstat?command=stats': Get "http://127.0.0.1:8083/jsonstat?command=stats": dial tcp 127.0.0.1:8083: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: dnsdist[local] check failed
2021-09-06 07:40:35: go.d ERROR: supervisord[local] error on 'supervisor.getAllProcessInfo' function call: Post "http://127.0.0.1:9001/RPC2": dial tcp 127.0.0.1:9001: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: supervisord[local] check failed
2021-09-06 07:40:35: netdata ERROR : PLUGIN[cgroups] : child pid 48281 exited with code 127.
2021-09-06 07:40:35: go.d ERROR: supervisord[local] error on 'supervisor.getAllProcessInfo' function call: Post "http://unix/RPC2": dial unix /run/supervisor.sock: connect: no such file or directory
2021-09-06 07:40:35: go.d ERROR: supervisord[local] check failed
2021-09-06 07:40:35: go.d ERROR: consul[local] error on request to http://localhost:8500/v1/agent/checks : Get "http://localhost:8500/v1/agent/checks": dial tcp [::1]:8500: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: consul[local] check failed
2021-09-06 07:40:35: go.d ERROR: consul[local] error on request to http://127.0.0.1:8500/v1/agent/checks : Get "http://127.0.0.1:8500/v1/agent/checks": dial tcp 127.0.0.1:8500: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: consul[local] check failed
2021-09-06 07:40:35: go.d ERROR: springboot2[local] check failed
2021-09-06 07:40:35: go.d ERROR: springboot2[local] check failed
2021-09-06 07:40:35: go.d ERROR: coredns[coredns] Get "http://127.0.0.1:9153/metrics": dial tcp 127.0.0.1:9153: connect: connection refused
2021-09-06 07:40:35: go.d ERROR: coredns[coredns] check failed
2021-09-06 07:40:35: go.d ERROR: dnsmasq_dhcp[dnsmasq_dhcp] open /var/lib/misc/dnsmasq.leases: no such file or directory
2021-09-06 07:40:35: go.d ERROR: dnsmasq_dhcp[dnsmasq_dhcp] init failed
2021-09-06 07:40:35: go.d ERROR: dnsmasq_dhcp[dnsmasq_dhcp] open /etc/pihole/dhcp.leases: no such file or directory
2021-09-06 07:40:35: go.d ERROR: dnsmasq_dhcp[dnsmasq_dhcp] init failed```

I don’t see anything interesting here. Can you grep for claim and then aclk? Case-insensitive.

We figured out a problem may be related to your experience as well. See Metrics missing for nodes in cloud - #4 by Christopher_Akritid1

Though the description of the issue isn’t the same. Please try with the updated helm chart (memory mode = ram for the children) and if you still have issues, send me the full error.log (captured from the output of the docker container via kubectl log) to chris@netdata.cloud.