# Unable to start Netdata child in k8s **URL:** \ **Category:** Help\ **Tags:** installation, agent\ **Created:** [June 12, 2022, 7:45pm UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988 "2022-06-12T19:45:40Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![neboman11](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/neboman11/32/1114_2.png) [@neboman11](https://community.netdata.cloud/u/neboman11)\ **Post date:** [June 12, 2022, 7:45pm UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/1 "2022-06-12T19:45:40Z")
**Suggested template:** ## Problem/Question I am unable to get the netdata child pod running on a specific node in my k8s cluster. It is stuck in a fail loop because of the health check that kills it. I’ve checked the logs trying to see any error messages there, but they are normally empty, except one time I got this error, but I couldn’t find anything relevant to my issue: ```Netdata Creating docker group 0 addgroup: gid '0' in use Assign netdata user to docker group 0 Could not add group docker with ID 0, its already there probably ``` I don’t have docker installed on any nodes in my cluster, so I’m assuming this is something that k8s is doing. The child is working on the other node in my cluster, and after redeploying multiple times, the issue is isolated to this node. The interesting part is I had netdata running on this node before, but I reset the cluster, and it stopped working. ## Relevant docs you followed/actions you took to solve the issue - Checked the logs, no output besides the previously mentioned error. - Updated helm repo (1.35 came out while I was troubleshooting this, but that update did not fix anything) - Cleared persistent volume contents ## Environment/Browser/Agent’s version etc `kubectl version --short` output: ```Client Kustomize Version: v4.5.4 Server Version: v1.24.1 ``` `helm show chart netdata/netdata` output: ```auto appVersion: v1.34.1 description: Real-time performance monitoring, done right! home: https://netdata.cloud/ icon: https://netdata.github.io/helmchart/logo.png keywords: - alerting - metric - monitoring maintainers: - email: ilya@netdata.cloud name: Ilya Mashchenko - email: knatsakis@netdata.cloud name: Konstantinos Natsakis - email: mansour@netdata.cloud name: Mansour Behabadi - email: vryumin@gmail.com name: Vladimir Ryumin - email: karuppiah7890@gmail.com name: Karuppiah Natarajan - email: kamikaze.ua@gmail.com name: Oleksii Kravchenko name: netdata sources: - https://github.com/netdata/helmchart - https://github.com/netdata/netdata type: application version: 3.7.18 ``` ## What I expected to happen Netdata child should start ---
**Author:** ![Christopher\_Akritid1](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/christopher_akritid1/32/419_2.png) [@Christopher\_Akritid1](https://community.netdata.cloud/u/Christopher_Akritid1)\ **Post date:** [June 13, 2022, 4:31am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/2 "2022-06-13T04:31:53Z")
I remember seeing sonething simular before, it was [Starting netdata official container fails with `addgroup: gid '999' in use` · Issue #6251 · netdata/netdata · GitHub](https://github.com/netdata/netdata/issues/6251) I don’t understand all of that though, so I will ask someone else to help. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [June 13, 2022, 8:49am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/3 "2022-06-13T08:49:59Z")
Hi, @neboman11. > Could not add group docker with ID 0, its already there probably That is not a fatal error. These logs come from [Netdata’s entrypoint script](https://github.com/netdata/netdata/blob/b5dcdbe7e87d0e770d07f3f0a155d1f8d145a91d/packaging/docker/run.sh#L40-L45). - Did you check `kubectl -n describe pod `? Usually the command provides more info and a good starting point to debug an issue with a pod. - Do you connect children nodes to Netdata Cloud? ---
**Author:** ![neboman11](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/neboman11/32/1114_2.png) [@neboman11](https://community.netdata.cloud/u/neboman11)\ **Post date:** [June 14, 2022, 1:19pm UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/4 "2022-06-14T13:19:07Z")
Describing the pod does give me some useful information, but not enough to know the root of the problem. It just tells me that the pod is restarting because it’s failing a health check, and that it’s happening only on a specific node. I would still think that there should be something in the logs that gives me a hint, but they are completely blank. I can include some of the output of the command if you want. I do have them setup to connect to Netdata Cloud, but neither of them show up in Cloud. The only thing I get is the parent pod, but it always shows as being offline. A weird thing about that though is the node that this is failing on that was originally working with Netdata still shows up in Cloud, but only as stale. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [June 14, 2022, 4:42pm UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/5 "2022-06-14T16:42:14Z")
Are you using default [liveness/readiness probs timeouts](https://github.com/netdata/helmchart/blob/128b8b7682cb41a81814c67303351648172b7996/charts/netdata/values.yaml#L94-L103)? If so, I’d try to increase both of them (e.g. `timeoutSeconds` 1-\>10). ---
**Author:** ![neboman11](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/neboman11/32/1114_2.png) [@neboman11](https://community.netdata.cloud/u/neboman11)\ **Post date:** [June 15, 2022, 12:53am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/6 "2022-06-15T00:53:28Z")
Yes, I was, but increasing them for both the parent and child did not change anything. I did notice in the output of `kubectl describe` of the child pod that it is exiting with code 143 if that helps any. I’m not able to find anything useful online related to that error code though. I definitely feel like this is somehow related to my node still appearing in Netdata Cloud as stale, since it’s the same node that is having this issue. I don’t have any idea how to troubleshoot that end of it though. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [June 15, 2022, 5:08pm UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/7 "2022-06-15T17:08:02Z")
I think you interpret the 143 exit code as 128+15. And 15 is [**SIGTERM**](https://man7.org/linux/man-pages/man7/signal.7.html). ---
**Author:** ![neboman11](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/neboman11/32/1114_2.png) [@neboman11](https://community.netdata.cloud/u/neboman11)\ **Post date:** [June 16, 2022, 12:44am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/8 "2022-06-16T00:44:12Z")
Is that sigterm just from it failing the liveliness check though? It still doesn’t explain why it fails to start to begin with or why the node is stale in Cloud. I also don’t see any documentation as to what the stale status means beyond a GitHub issue that isn’t very informative. Is there anything I can try to test or run that will help isolate the root or narrow down what might be causing this? ---
**Author:** ![neboman11](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/neboman11/32/1114_2.png) [@neboman11](https://community.netdata.cloud/u/neboman11)\ **Post date:** [July 14, 2022, 3:58am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/9 "2022-07-14T03:58:56Z")
Well, I managed to get netdata to finally run, but definitely not in a desirable configuration. I changed the liveliness probe and readiness probe periods to 30000s to see if allowing the container to run would do anything, and after waiting several minutes for it to start up, I managed to get it actually running (and populating Netdata Cloud). The only problem outside of having the set such a ridiculous configuration to get it to run is that it has at least 1 thread on the CPU constantly pegged at 100% and occasionally runs the whole CPU at 100% (across all 4 threads). No idea what is causing it to run like this, since I managed to start the container without issue when run outside the deployment. The logs also don’t immediately show anything that would signal a cause to this issue, but at least I got it running. It would be great if the netdata team can find a solution to the CPU usage since that is still a problem. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [July 14, 2022, 7:23am UTC](https://community.netdata.cloud/t/unable-to-start-netdata-child-in-k8s/2988/10 "2022-07-14T07:23:37Z")
@neboman11 - do you have SSH access to nodes? If so, you can use, for instance, `htop` and find out which Netdata thread uses 100% CPU. - you can try `netdata/netdata:latest` docker image, it has a lot of fixes. - [exporting an image](https://learn.netdata.cloud/docs/dashboard/import-export-print-snapshot#export-a-snapshot) from the child instance and sending it to `ilya@netdata.cloud` would also help. - did you check the child instance logs? Can you share them?