Postgres charts plugin is not Working

Hi,

I was able to download the latest stable on Debian 8.11

Distributor ID: Debian
Description:    Debian GNU/Linux 8.11 (jessie)
Release:        8.11
Codename:       jessie

My first problem is that Service installation didn’t went well instead netdata figured out to start himself by using the direct binary from /opt/netdata folder. In order to fix it i had to delete the service from /lib/systemd/system/netdata.service and then copy it from /opt/netdata systemd folder to /lib/systemd/system/netdata.service . I noticed a lot of differences but the main one was ExecStart=/usr/sbin/netdata that was wrong or at least wasn’t existed on my Server.

My biggest problem now is with postgresql
First postgrsql is installed under /usr/local/pgsql , second problem is that I already have user not assigned with netdata that I’d like to use the .pgpass method to connect to server.


.....usr/libexec/netdata/plugins.d$ ./go.d.plugin -d -m postgres
[ DEBUG ] main[main] main.go:113 plugin: name=go.d, version=v0.45.0
[ DEBUG ] main[main] main.go:115 current user: name=netdata, uid=999
[ INFO  ] main[main] agent.go:136 instance is started
[ INFO  ] main[main] setup.go:42 loading config file
[ INFO  ] main[main] setup.go:50 looking for 'go.d.conf' in [/opt/netdata/etc/netdata /opt/netdata/usr/lib/netdata/conf.d]
[ INFO  ] main[main] setup.go:57 found '/opt/netdata/usr/lib/netdata/conf.d/go.d.conf
[ INFO  ] main[main] setup.go:64 config successfully loaded
[ INFO  ] main[main] agent.go:140 using config: enabled 'true', default_run 'true', max_procs '0'
[ INFO  ] main[main] setup.go:69 loading modules
[ INFO  ] main[main] setup.go:88 enabled/registered modules: 1/78
[ INFO  ] main[main] setup.go:93 building discovery config
[ INFO  ] main[main] setup.go:123 looking for 'postgres.conf' in [/opt/netdata/etc/netdata/go.d /opt/netdata/usr/lib/netdata/conf.d/go.d]
[ INFO  ] main[main] setup.go:139 found '/opt/netdata/etc/netdata/go.d/postgres.conf
[ INFO  ] main[main] setup.go:144 dummy/read/watch paths: 0/1/0
[ INFO  ] discovery[manager] manager.go:92 registered discoverers: [file discovery: [file reader]]
[ INFO  ] discovery[manager] manager.go:97 instance is started
[ INFO  ] run[manager] run.go:32 instance is started
[ INFO  ] build[manager] build.go:107 instance is started
[ INFO  ] discovery[file manager] discovery.go:73 instance is started
[ INFO  ] discovery[file reader] read.go:41 instance is started
[ INFO  ] discovery[file reader] read.go:42 instance is stopped
[ DEBUG ] build[manager] build.go:154 received config group ('/opt/netdata/etc/netdata/go.d/postgres.conf'): 1 jobs (added: 1, removed: 0)
[ DEBUG ] build[manager] build.go:303 building postgres[local] job, config: map[__provider__:file reader __source__:/opt/netdata/etc/netdata/go.d/postgres.conf autodetection_retry:0 dsn:postgres://periodic@localhost:5432/postgres module:postgres name:local priority:70000 update_every:1]
[ ERROR ] postgres[local] postgres.go:119 error on pinging the Postgres database [postgres://periodic@localhost:5432/postgres]: failed to connect to `host=localhost user=periodic database=postgres`: server error (FATAL: no pg_hba.conf entry for host "127.0.0.1", user "periodic", database "postgres" (SQLSTATE 28000))
[ ERROR ] postgres[local] job.go:191 check failed
[ DEBUG ] run[manager] run.go:43 tick 0
[ DEBUG ] run[manager] run.go:43 tick 1
[ DEBUG ] run[manager] run.go:43 tick 2
^C[ INFO  ] main[main] agent.go:104 received interrupt signal (2). Terminating...
[ INFO  ] run[manager] run.go:33 instance is stopped
[ INFO  ] build[manager] build.go:108 instance is stopped
[ INFO  ] discovery[manager] manager.go:98 instance is stopped
[ INFO  ] discovery[file manager] discovery.go:74 instance is stopped
[ INFO  ] main[main] agent.go:137 instance is stopped

Ok there is also an other pkg installed onthe system which runs under /usr/bin/psql , but this is NOT my main installation. I guess that somehow netdata trying to execute the process by using the default psql client. So if this is the case, how I can configure the psql path?

Also my postgresql.conf containing the below entries

jobs:
  - name: local
    #dsn: 'postgres://postgres:postgres@127.0.0.1:5432/postgres'
    dsn: 'postgres://periodic@localhost:5432/postgres'
    #collect_databases_matching: '*'

Thank you

I managed to solve this by using the below with .pgpass

dsn: 'host=/tmp dbname=postgres user=periodic'

But while debug is running just fine, charts still not coming up. Any idea?

:~/usr/libexec/netdata/plugins.d$ ./go.d.plugin -d -m postgres                                                                                                                                  
[ DEBUG ] main[main] main.go:113 plugin: name=go.d, version=v0.45.0                                                                                                                                               
[ DEBUG ] main[main] main.go:115 current user: name=netdata, uid=999                                                                                                                                              
[ INFO  ] main[main] agent.go:136 instance is started                                                                                                                                                             
[ INFO  ] main[main] setup.go:42 loading config file                                                                                                                                                              
[ INFO  ] main[main] setup.go:50 looking for 'go.d.conf' in [/opt/netdata/etc/netdata /opt/netdata/usr/lib/netdata/conf.d]                                                                                        
[ INFO  ] main[main] setup.go:57 found '/opt/netdata/etc/netdata/go.d.conf                                                                                                                                        
[ INFO  ] main[main] setup.go:64 config successfully loaded                                                                                                                                                       
[ INFO  ] main[main] agent.go:140 using config: enabled 'true', default_run 'true', max_procs '0'                                                                                                                 
[ INFO  ] main[main] setup.go:69 loading modules                                                                                                                                                                  
[ INFO  ] main[main] setup.go:88 enabled/registered modules: 1/78                                                                                                                                                 
[ INFO  ] main[main] setup.go:93 building discovery config                                                                                                                                                        
[ INFO  ] main[main] setup.go:123 looking for 'postgres.conf' in [/opt/netdata/etc/netdata/go.d /opt/netdata/usr/lib/netdata/conf.d/go.d]                                                                         
[ INFO  ] main[main] setup.go:139 found '/opt/netdata/etc/netdata/go.d/postgres.conf                                                                                                                              
[ INFO  ] main[main] setup.go:144 dummy/read/watch paths: 0/1/0                                                                                                                                                   
[ INFO  ] discovery[manager] manager.go:92 registered discoverers: [file discovery: [file reader]]                                                                                                                
[ INFO  ] discovery[manager] manager.go:97 instance is started                                                                                                                                                    
[ INFO  ] discovery[file manager] discovery.go:73 instance is started                                                                                                                                             
[ INFO  ] build[manager] build.go:107 instance is started                                                                                                                                                         
[ INFO  ] run[manager] run.go:32 instance is started                                                                                                                                                              
[ INFO  ] discovery[file reader] read.go:41 instance is started                                                                                                                                                   
[ INFO  ] discovery[file reader] read.go:42 instance is stopped                                                                                                                                                   
[ DEBUG ] build[manager] build.go:154 received config group ('/opt/netdata/etc/netdata/go.d/postgres.conf'): 1 jobs (added: 1, removed: 0)                                                                        
[ DEBUG ] build[manager] build.go:303 building postgres[local] job, config: map[__provider__:file reader __source__:/opt/netdata/etc/netdata/go.d/postgres.conf autodetection_retry:0 dsn:host=/tmp dbname=postgre
s user=periodic module:postgres name:local priority:70000 update_every:1]                                                                                                                                         
[ DEBUG ] postgres[local] collect.go:37 connected to PostgreSQL v100019                                                                                                                                           
[ DEBUG ] postgres[local] collect.go:46 connected as super user: false                                                                                                                                            
[ DEBUG ] postgres[local] collect.go:55 the instance is in recovery mode: false                                                                                                                                   
[ INFO  ] postgres[local] job.go:194 check success                                                                                                                                                                
[ INFO  ] postgres[local] job.go:214 started, data collection interval 1s                                                                                                                                         
[ DEBUG ] run[manager] run.go:43 tick 0                                                                                                                                                                           
CHART 'netdata.execution_time_of_postgres_local' '' 'Execution time' 'ms' 'go.d' 'netdata.go_plugin_execution_time' 'line' '145000' '1' '' 'go.d' 'postgres'                                                      
CLABEL '_collect_job' 'local' '1'                                                                                                                                                                                 
CLABEL_COMMIT                                                                                                                                                                                                     
DIMENSION 'time' '' 'absolute' '1' '1' ''                                                                                                                                                                         
                                                                                                                                                                                                                  
CHART 'postgres_local.connections_utilization' '' 'Connections utilization' 'percentage' 'connections' 'postgres.connections_utilization' 'line' '70000' '1' '' 'go.d' 'postgres'                                 
CLABEL '_collect_job' 'local' '1'                                                                                                                                                                                 
CLABEL_COMMIT                                                                                                                                                                                                     
DIMENSION 'used' 'used' 'absolute' '1' '1' ''                       
...snip...

But Charts is missing?

~/usr/libexec/netdata/plugins.d$ ./python.d.plugin debug postgres
2023-01-25 17:11:00: python.d INFO: plugin[main] : using python v2
2023-01-25 17:11:00: python.d ERROR: plugin[main] : unknown modules : ['postgres']
2023-01-25 17:11:00: python.d INFO: plugin[main] : probably you meant : 
['postfix']
:/usr/libexec/netdata/plugins.d$ 

Some additional data for NetData

/usr/libexec/netdata/plugins.d$ /opt/netdata/bin/srv/netdata -W buildinfo
Version: netdata v1.37.1
Configure options:  '--prefix=/opt/netdata/usr' '--sysconfdir=/opt/netdata/etc' '--localstatedir=/opt/netdata/var' '--libexecdir=/opt/netdata/usr/libexec' '--libdir=/opt/netdata/usr/lib' '--with-zlib' '--with-math' '--with-user=netdata' '--enable-cloud' '--without-bundled-protobuf' '--disable-dependency-tracking' 'CFLAGS=-static -O2 -I/openssl-static/include -pipe' 'LDFLAGS=-static -L/openssl-static/lib' 'PKG_CONFIG_PATH=/openssl-static/lib/pkgconfig'
Install type: kickstart-static
    Binary architecture: x86_64
Features:
    dbengine:                   YES
    Native HTTPS:               YES
    Netdata Cloud:              YES 
    ACLK:                       YES
    TLS Host Verification:      YES
    Machine Learning:           YES
    Stream Compression:         YES
Libraries:
    protobuf:                YES (system)
    jemalloc:                NO
    JSON-C:                  YES
    libcap:                  NO
    libcrypto:               YES
    libm:                    YES
    tcalloc:                 NO
    zlib:                    YES
Plugins:
    apps:                    YES
    cgroup Network Tracking: YES
    CUPS:                    NO
    EBPF:                    YES
    IPMI:                    NO
    NFACCT:                  NO
    perf:                    YES
    slabinfo:                YES
    Xen:                     NO
    Xen VBD Error Tracking:  NO
Exporters:
    AWS Kinesis:             NO
    GCP PubSub:              NO
    MongoDB:                 NO
    Prometheus Remote Write: YES
Debug/Developer Features:
    Trace Allocations:       NO

Hi, @pgro.

I managed to solve this by using the below with .pgpass

dsn: 'host=/tmp dbname=postgres user=periodic'

Did you restart netdata service after? If yes and it didn’t help - check the error log

grep postgres /opt/netdata/var/log/netdata/error.log

It was worked for a few minutes with a few graphs but now dissapears again.

error log with a lot of permission denied , but from which user?


2023-01-30 10:08:37: apps.plugin ERROR : MAIN : Cannot process entries in /proc/43018/fd (command 'postgres') (errno 13, Permission denied)
2023-01-30 12:56:11: apps.plugin ERROR : MAIN : Cannot process /proc/56632/io (command 'postgres') (errno 13, Permission denied)
2023-01-30 12:56:11: apps.plugin ERROR : MAIN : Cannot process entries in /proc/56632/fd (command 'postgres') (errno 13, Permission denied)
2023-01-30 14:05:43: apps.plugin ERROR : MAIN : Cannot process /proc/62213/io (command 'postgres') (errno 13, Permission denied)`Preformatted text`
2023-01-30 14:05:43: apps.plugin ERROR : MAIN : Cannot process entries in /proc/62213/fd (command 'postgres') (errno 13, Permission denied)

.
.
.

2023-01-30 14:15:39: apps.plugin ERROR : MAIN : Cannot process /proc/63012/io (command 'tail') (errno 13, Permission denied)
2023-01-30 14:15:45: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63021/fd (command 'tail') (errno 13, Permission denied)
2023-01-30 14:15:45: apps.plugin ERROR : MAIN : Cannot process /proc/63021/io (command 'tail') (errno 13, Permission denied)
2023-01-30 14:15:49: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63025/fd (command 'tail') (errno 13, Permission denied)
2023-01-30 14:15:49: apps.plugin ERROR : MAIN : Cannot process /proc/63025/io (command 'tail') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63036/fd (command 'sh') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63037/fd (command 'pgbackrest') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63038/fd (command 'ssh') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process /proc/63036/io (command 'sh') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process /proc/63037/io (command 'pgbackrest') (errno 13, Permission denied)
2023-01-30 14:15:54: apps.plugin ERROR : MAIN : Cannot process /proc/63038/io (command 'ssh') (errno 13, Permission denied)
2023-01-30 14:16:30: apps.plugin ERROR : MAIN : Cannot process entries in /proc/63108/fd (command 'cat') (errno 13, Permission denied)

Ok I found an older thread under Permission denied accessing /proc and /sys (Debian 7 and 8) · Issue #280 · netdata/netdata · GitHub.

I remembered that I manually installed the service as SystemD so I switched it now to Init.d. All ok now

Thanx