# Netdata container fails to collect MySQL stats after MariaDB restart when connecting via UNIX socket **URL:** \ **Category:** Help\ **Tags:** agent, container-monitoring, agent-configuration\ **Created:** [October 9, 2021, 3:03am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819 "2021-10-09T03:03:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![John](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@John](https://community.netdata.cloud/u/John)\ **Post date:** [October 9, 2021, 3:03am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/1 "2021-10-09T03:03:36Z")
## Problem/Question Hi All, My netdata container collects MySQL stats as expected from the MariaDB database on the host, as long as netdata is started while MariaDB is running. The problem is when the database is restarted, netadata MySQL data collection stops. The logs show ‘connection refused’ when using MySQL collector in debug mode. [ERROR] mysql[local] mysql.go:93 error on pinging the mysql database [netdata:net01@@unix(/var/lib/mysql/mysql.sock)/]: dial unix /var/lib/mysql/mysql.sock: connect: connection refused Thanks. ## Environment netdata v1.31.0 Red Hat Enterprise Linux release 8.4 (Ootpa) MariaDb v10.4.21 ## What I expected to happen I expect the data collection to recommence and the MySQL graphs to render as per normal. [DEBUG] mysql[local] mysql.go:97 connected using DSN [netdata:net01@@unix(/var/lib/mysql/mysql.sock)/] ---
**Author:** ![andrewm4894](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/andrewm4894/32/125_2.png) [@andrewm4894](https://community.netdata.cloud/u/andrewm4894)\ **Post date:** [October 12, 2021, 11:16am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/2 "2021-10-12T11:16:24Z")
I do recall we need a `netdata` user to be created on the database: [MySQL monitoring with Netdata | Learn Netdata](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/mysql#requirements) Had you done that before and had it working? @ilyam8 have you come across this before? ---
**Author:** ![John](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@John](https://community.netdata.cloud/u/John)\ **Post date:** [October 13, 2021, 2:10am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/3 "2021-10-13T02:10:53Z")
Thanks for the reply @andrewm4894, Yes the netdata user has been setup and works as per mysql.conf extract below; # [JOBS] jobs: # my.cnf - name: local my.cnf: ‘/etc/my.cnf’ - name: local dsn: netdata:net01@@unix(/tmp/mysql.sock)/ The mysql collector works as expected as per extract below, however none of the mysql charts get updated; bash-5.0$ /usr/libexec/netdata/plugins.d/go.d.plugin -d -m mysql [DEBUG] main[main] main.go:111 plugin: name=go.d, version=v0.28.2 [DEBUG] main[main] main.go:113 current user: name=netdata, uid=201 [INFO] main[main] agent.go:106 instance is started [INFO] main[main] setup.go:39 loading config file [INFO] main[main] setup.go:47 looking for ‘go.d.conf’ in [/etc/netdata /usr/lib/netdata/conf.d] [INFO] main[main] setup.go:54 found ‘/usr/lib/netdata/conf.d/go.d.conf [INFO] main[main] setup.go:61 config successfully loaded [INFO] main[main] agent.go:110 using config: enabled ‘true’, default\_run ‘true’, max\_procs ‘0’ [INFO] main[main] setup.go:66 loading modules [INFO] main[main] setup.go:85 enabled/registered modules: 1/59 [INFO] main[main] setup.go:90 building discovery config [INFO] main[main] setup.go:116 looking for ‘mysql.conf’ in [/etc/netdata/go.d /usr/lib/netdata/conf.d/go.d] [INFO] main[main] setup.go:123 found ‘/etc/netdata/go.d/mysql.conf [INFO] main[main] setup.go:128 dummy/read/watch paths: 0/1/0 [INFO] discovery[manager] manager.go:90 registered discoverers: [file discovery: [file reader]] [INFO] discovery[manager] manager.go:95 instance is started [INFO] build[manager] build.go:106 instance is started [INFO] run[manager] run.go:30 instance is started [INFO] discovery[file manager] discovery.go:71 instance is started [INFO] discovery[file reader] read.go:39 instance is started [INFO] discovery[file reader] read.go:40 instance is stopped [DEBUG] build[manager] build.go:153 received config group (’/etc/netdata/go.d/mysql.conf’): 3 jobs (added: 3, removed: 0) [DEBUG] build[manager] build.go:302 building mysql[local] job, config: map[**provider** :file reader **source** :/etc/netdata/go.d/mysql.conf autodetection\_retry:0 module:mysql my.cnf:/etc/my.cnf name:local priority:70000 update\_every:1] [ERROR] mysql[local] mysql.go:81 key-value delimiter not found: !includedir /etc/my.cnf.d [ERROR] mysql[local] job.go:152 init failed [DEBUG] build[manager] build.go:302 building mysql[local] job, config: map[**provider** :file reader **source** :/etc/netdata/go.d/mysql.conf autodetection\_retry:0 dsn:netdata:net01@@unix(/tmp/mysql.sock)/ module:mysql name:local priority:70000 update\_every:1] [DEBUG] mysql[local] mysql.go:97 connected using DSN [netdata:net01@@unix(/tmp/mysql.sock)/] [DEBUG] mysql[local] collect\_version.go:17 executing query: ‘SELECT VERSION()’ [DEBUG] mysql[local] collect\_version.go:23 application version: 10.4.21-MariaDB [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [DEBUG] mysql[local] collect\_global\_vars.go:23 executing query: ‘SHOW GLOBAL VARIABLES WHERE Variable\_name LIKE ‘max\_connections’ OR Variable\_name LIKE ‘table\_open\_cache’’ [DEBUG] mysql[local] collect\_slave\_status.go:30 executing query: ‘SHOW ALL SLAVES STATUS’ [DEBUG] mysql[local] collect\_user\_statistics.go:25 executing query: ‘SHOW USER\_STATISTICS’ [INFO] mysql[local] job.go:160 check success [INFO] build[manager] build.go:208 mysql[local] job is being served by another job, skipping it [INFO] mysql[local] job.go:180 started, data collection interval 1s [DEBUG] run[manager] run.go:41 tick 0 [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [DEBUG] mysql[local] collect\_global\_vars.go:23 executing query: ‘SHOW GLOBAL VARIABLES WHERE Variable\_name LIKE ‘max\_connections’ OR Variable\_name LIKE ‘table\_open\_cache’’ [DEBUG] mysql[local] collect\_slave\_status.go:30 executing query: ‘SHOW ALL SLAVES STATUS’ [DEBUG] mysql[local] collect\_user\_statistics.go:25 executing query: ‘SHOW USER\_STATISTICS’ CHART ‘netdata.execution\_time\_of\_mysql\_local’ ‘’ ‘Execution Time for mysql\_local’ ‘ms’ ‘go.d’ ‘netdata.go\_plugin\_execution\_time’ ‘’ ‘145000’ ‘1’ ‘’ ‘go.d’ ‘mysql’ DIMENSION ‘time’ ‘’ ‘’ ‘1’ ‘1’ ‘’ CHART ‘mysql\_local.net’ ‘’ ‘Bandwidth’ ‘kilobits/s’ ‘bandwidth’ ‘[mysql.net](http://mysql.net)’ ‘area’ ‘70000’ ‘1’ ‘’ ‘go.d’ ‘mysql’ DIMENSION ‘bytes\_received’ ‘in’ ‘incremental’ ‘8’ ‘1000’ ‘’ DIMENSION ‘bytes\_sent’ ‘out’ ‘incremental’ ‘-8’ ‘1000’ ‘’ BEGIN ‘mysql\_local.net’ SET ‘bytes\_received’ = 1014140552 SET ‘bytes\_sent’ = 4910862481 END … … … Cheers ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [October 14, 2021, 7:39am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/4 "2021-10-14T07:39:32Z")
Hi, @John. You have edited your post several times, so let’s see if I understand the problem: - MySQL is **not running inside** a container. - Netdata is **running inside** a container. - Netdata is able to collect MySQL data when it starts after MySQL. - Netdata is connected via a UNIX socket. - Netdata fails to handle MySQL restart (the charts don’t get updated). Is that correct? If yes, let’s do the following: - run “go.d.plugin” in the debug mode. - restart MySQL service. - at this point, the debug output should tell us the problem. ---
**Author:** ![John](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@John](https://community.netdata.cloud/u/John)\ **Post date:** [October 15, 2021, 2:31am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/5 "2021-10-15T02:31:56Z")
Hi @ilyam8, Yes the above is correct. Here are the last few lines of output from debug mode; BEGIN ‘mysql\_local.userstats\_commands\_maxscale’ 999832 SET ‘userstats\_maxscale\_select\_commands’ = 1320 SET ‘userstats\_maxscale\_update\_commands’ = 0 SET ‘userstats\_maxscale\_other\_commands’ = 3 END BEGIN ‘netdata.execution\_time\_of\_mysql\_local’ 999832 SET ‘time’ = 3 END [DEBUG] run[manager] run.go:41 tick 4 [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [mysql] 2021/10/15 02:30:03 packets.go:122: closing bad idle connection: EOF [INFO] main[main] agent.go:191 received broken pipe signal (13). Terminating… bash-5.0$ Cheers. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [October 15, 2021, 5:20am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/6 "2021-10-15T05:20:10Z")
Thanks, @John. I see what is happening. A quick fix is to switch from Unix socket to TCP. ---
**Author:** ![ilyam8](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.netdata.cloud/ilyam8/32/134_2.png) [@ilyam8](https://community.netdata.cloud/u/ilyam8)\ **Post date:** [October 15, 2021, 9:35am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/7 "2021-10-15T09:35:20Z")
@John the PR with the fix is [agent: dont exit on SIGPIPE by ilyam8 · Pull Request #630 · netdata/go.d.plugin · GitHub](https://github.com/netdata/go.d.plugin/pull/630) ---
**Author:** ![John](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@John](https://community.netdata.cloud/u/John)\ **Post date:** [October 22, 2021, 9:35am UTC](https://community.netdata.cloud/t/netdata-container-fails-to-collect-mysql-stats-after-mariadb-restart-when-connecting-via-unix-socket/1819/8 "2021-10-22T09:35:52Z")
Thanks @ilyam8, The fix is working, FYI logs & graphs below; BEGIN ‘mysql\_local.userstats\_commands\_root’ 1000166 SET ‘userstats\_root\_select\_commands’ = 962 SET ‘userstats\_root\_update\_commands’ = 4 SET ‘userstats\_root\_other\_commands’ = 962 END BEGIN ‘netdata.execution\_time\_of\_mysql\_local’ 1000166 SET ‘time’ = 5 END [DEBUG] run[manager] run.go:41 tick 41 [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [mysql] 2021/10/22 09:23:39 packets.go:123: closing bad idle connection: EOF [ERROR] mysql[local] mysql.go:105 error on collecting global status: dial tcp 10.195.225.51:3306: connect: connection refused [DEBUG] run[manager] run.go:41 tick 42 [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [ERROR] mysql[local] mysql.go:105 error on collecting global status: dial tcp 10.195.225.51:3306: connect: connection refused [DEBUG] run[manager] run.go:41 tick 43 [DEBUG] mysql[local] collect\_global\_status.go:144 executing query: ‘SHOW GLOBAL STATUS’ [DEBUG] run[manager] run.go:41 tick 44 [DEBUG] mysql[local] job.go:174 skip the tick due to previous run hasn’t been finished [mysql] 2021/10/22 09:23:42 packets.go:37: read tcp 10.0.2.100:32920-\>10.195.225.51:3306: read: connection reset by peer [INFO] main[main] agent.go:191 received broken pipe signal (13). Terminating… bash-5.0$ ![PR630-01](https://canada1.discourse-cdn.com/flex029/uploads/netdata2/original/2X/2/2c08cc174f77436040dc1ac68a586068506e7d58.png) Cheers, John.