# How to debug container name resolution for the Netdata Agent **URL:** \ **Category:** General\ **Tags:** how-to\ **Created:** [June 3, 2021, 3:21pm UTC](https://community.netdata.cloud/t/how-to-debug-container-name-resolution-for-the-netdata-agent/1399 "2021-06-03T15:21:02Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![OdysLam](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/odyslam/32/98_2.png) [@OdysLam](https://community.netdata.cloud/u/OdysLam)\ **Post date:** [June 3, 2021, 3:21pm UTC](https://community.netdata.cloud/t/how-to-debug-container-name-resolution-for-the-netdata-agent/1399/1 "2021-06-03T15:21:03Z")
A couple of days ago I started a new container-based project and naturally, I installed Netdata to be able to have per-second metrics with minimal effort. This was crucial, as the project concerned [urbit](https://urbit.org/), a fascinating project that wants to re-invent the personal computer. We won’t go into detail, but it’s a sort of a VM that I deployed on my Raspberry pi using [balena](https://balena.io). Balena is a container-based platform to manage IoT devices and the lifecycle of their applications, so I deployed 2 containers on my Raspberry pi, a container running Urbit and a container running Netdata on docker. ![image](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/1X/9c1000d89f90f1b90bbe78e5b9afc7b7c103a85e.jpeg) To deploy Netdata, I defined a docker-compose file based on the [documentation](https://learn.netdata.cloud/docs/agent/packaging/docker) and a Dockerfile that used `netdata/netdata` as base. ![image](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/1X/6d83fc11d25f4611588505dd139bd53ccc2443ff.jpeg) ![image](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/1X/bf96dbca0d5bb2ee60423731528cbf046e2dbf3c.jpeg) ## Why Modify the Netdata Dockerfile? Netdata’s dockerfile is robust, created by our own @Austin_Hemmelgarn , but if you want to customize your Netdata installation, you will need to use our Dockerfile as base and create your own. Customing the Dockerfile is a great way to add custom configuration for Netdata, collectors, and alarms. This configuration will be copied every time you build the container, making it a great choice for automation. SSHing into the container to use `./edit-config` is far from ideal 😅 Another great reason to do this is the ability to load custom software into the container. For example, you might want to have a proper `ssh` server inside the container, so that you can ssh into it remotely. ## How to fix container name resolution It’s an issue that we see from time to time. Netdata fails to translate the `container_id` into the humanly readable container name, making the integration considerably harder to use. This boils down to a particular script that netdata uses, named `cgroup-name.sh`. This script is run by the Netdata Agent and communicates over `HTTP` or `Linux socket` with various container daemons, in order to find the name. In our case, it communicates with the docker daemon, over the docker socket. As per the documentation, there are a bunch of different ways to tackle this. I chose to go with an option that is both _somewhat_ secure and _somewhat easy_, that is to add the `netdata` user in the `docker group`, thus allowing netdata to use the socket and get the information it needs. The problem is that in order to do that, we need to define the `PGID` of the `docker group` in the `docker-compose.yml` file. (check the image above). And here lies the **gotcha**. In the Netdata docs, we use the `/etc/group` as the source of truth to find the PGID of the group `docker`. If we look at the startup script of the Netdata container, it created the docker group and proceeds to add the netdata user to it. ![image](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/1X/6be934fbdce0d716e129153e276d18a25caa793f.jpeg) Thus, when we read `/etc/group`, we will read the value that **we** in fact populated, by defining the `PGID` and then running the default `ENTRYPOINT` of the Netdata container. It’s a self-fulfilling prophecy. ## How Container Resolution is happening at Netdata? As we have said, Netdata needs access to the `docker socket` that lives in `/var/run/docker.sock`. This is the endpoint that the `cgroup-name.sh` script is using to translate IDs to names. **Thus, what we really really want is that the netdata user belongs to the same group to which that file belongs.** Thus, the best way to tackle container name resolution is: 1. Run netdata container without defining a PGID 2. SSH into the container (e.g `docker exec /bin/bash` ) 3. run `ls -l /var/run`. The second number in the columns is the PGID that we are searching for 4. Go back to our `docker run` or `docker-compose.yml` and modify the PGID In the image below, the PGID has been set correctly. This means that the PGID for the `docker` group and the PGID of the group to which `docker.sock` belongs to are the same. Thus, `ls -l` will not output a PGID, but instead the name of the group which has that PGID, in our case `docker`. ![image](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/1X/c71a2a8f3ae507ce0a2c77e1bf552b52e372ee36.jpeg) ## In conclusion 1. If name resolution doesn’t work, it’s probably because the `netdata` user can’t access the socket. 2. To fix this, run `ls -l /var/run/` and find the `docker.sock`. If the group is not properly set, it will have a number instead of `docker`. That number is the PGID. Note that alternative container solutions could have docker-compatible sockets with different names. For example, balena has `balena-engine.sock`. You will need to define this in the `docker-compose.yaml` file, the custom Dockerfile or `docker run`. 3. Go back in docker-compose/docker run and replace the PGID. 4. Build the container(s) again What do you think of **Netdata** + **Containers**? What would you like us to improve? Comment below 👇👇👇👇 ---
**Author:** ![r00nster](https://avatars.discourse-cdn.com/v4/letter/r/5f8ce5/32.png) [@r00nster](https://community.netdata.cloud/u/r00nster)\ **Post date:** [October 27, 2021, 3:46pm UTC](https://community.netdata.cloud/t/how-to-debug-container-name-resolution-for-the-netdata-agent/1399/2 "2021-10-27T15:46:28Z")
So I been running Netdata on my Alpine Linux docker host and its been excellent but had the “slightly annoying” container IDs displaying instead of the names. So I decided to try and fix this. I tried all 3 ways of configuring this in the documentation with the following results My setup Alpine linux docker no special configuration 1. Docker Socks Proxy I installed the “tecnativa/docker-socket-proxy” as suggested in the documentation and this “kinda” worked but with this flaw. The Netdata documentation suggests installing it to the default docker bridge network and then adding the “DOCKER\_HOST=proxy:237”. The problem I found with this is “proxy” (the container name for “tecnativa/docker-socket-proxy”) isn’t resolvable on the default bridge network hence the Netdata container fails as it can’t resolve it. In fact on the default/system bridge network you cannot resolve containers by container name as per this tutorial. I find the Netdata documentation slightly misleading for this reason. > **[Networking with standalone containers](https://docs.docker.com/network/network-tutorial-standalone/)** > > Tutorials for networking with standalone containers If I replace “proxy” with the internal IP address of the container it worked. Not a permanent fix but at least it proves things were working. The “tecnativa/docker-socket-proxy” documentation suggests placing its container on an isolated user-defined network and connecting containers to that network if they need to use it. Also for user-defined bridge networks the containers are resolvable by container name (as per the tutorial above). I went down this track, got the proxy container running and resolvable by name BUT when I start the Netdata container on a user-defined bridge network the container starts but I can’t access the webportal on http://:19999. In fact this was a strange issue as it took me an evening to discover I had to reboot my docker host before even my original container would work again back on the default system bridge network it had been working fine on for months. I therefore parked this option. 1. PGID environment variable (my preferred option) I just couldn’t get this to work even after finding and adding the group ID to the docker run command -e PGID=101 in my case. 2. Root access (volume binding to /var/run/docker.sock) This worked but I didn’t leave it enabled for the obvious security reasons. So… my questions. Why can I not get PGID working ? Why does running NetData container on a user defined bridge network cause its web portal to fail (if I try and curl I get “connection refused”)