Retrieve web_log in netdata cloud

[laupi]

Hi,

I successfully setup nginx web_log in the agent but i don’t see it in the netdata cloud ?

So, how i can retrieve the stats from this web_log parsing ?

Thanks for you help

web_log.conf

jobs:
  # NGINX
  # debian, arch
  - name: nginx
    path: /var/log/nginx/api.savaoo.fr.access.log
    log_type: csv
    csv_config:
      format: '$remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time'

./go.d.plugin -d -m web_log is OK

bash-5.1# ./go.d.plugin -d -m web_log
[ DEBUG ] main[main] main.go:113 plugin: name=go.d, version=v0.37.2
[ DEBUG ] main[main] main.go:115 current user: name=root, uid=0
[ INFO  ] main[main] agent.go:136 instance is started
[ INFO  ] main[main] setup.go:42 loading config file
[ INFO  ] main[main] setup.go:50 looking for 'go.d.conf' in [/etc/netdata /usr/lib/netdata/conf.d]
[ INFO  ] main[main] setup.go:57 found '/etc/netdata/go.d.conf
[ INFO  ] main[main] setup.go:64 config successfully loaded
[ INFO  ] main[main] agent.go:140 using config: enabled 'true', default_run 'true', max_procs '0'
[ INFO  ] main[main] setup.go:69 loading modules
[ INFO  ] main[main] setup.go:88 enabled/registered modules: 1/71
[ INFO  ] main[main] setup.go:93 building discovery config
[ INFO  ] main[main] setup.go:123 looking for 'web_log.conf' in [/etc/netdata/go.d /usr/lib/netdata/conf.d/go.d]
[ INFO  ] main[main] setup.go:139 found '/etc/netdata/go.d/web_log.conf
[ INFO  ] main[main] setup.go:144 dummy/read/watch paths: 0/1/0
[ INFO  ] discovery[manager] manager.go:92 registered discoverers: [file discovery: [file reader]]
[ INFO  ] discovery[manager] manager.go:97 instance is started
[ INFO  ] run[manager] run.go:32 instance is started
[ INFO  ] build[manager] build.go:107 instance is started
[ INFO  ] discovery[file manager] discovery.go:73 instance is started
[ INFO  ] discovery[file reader] read.go:41 instance is started
[ INFO  ] discovery[file reader] read.go:42 instance is stopped
[ DEBUG ] build[manager] build.go:154 received config group ('/etc/netdata/go.d/web_log.conf'): 1 jobs (added: 1, removed: 0)
[ DEBUG ] build[manager] build.go:303 building web_log[nginx] job, config: map[__provider__:file reader __source__:/etc/netdata/go.d/web_log.conf autodetection_retry:0 csv_config:map[format:$remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time] log_type:csv module:web_log name:nginx path:/var/log/nginx/api.savaoo.fr.access.log priority:70000 update_every:1]
[ DEBUG ] web_log[nginx] init.go:33 skipping URL patterns creating, no patterns provided
[ DEBUG ] web_log[nginx] init.go:51 skipping custom fields creating, no custom fields provided
[ DEBUG ] web_log[nginx] init.go:76 skipping custom time fields creating, no custom time fields provided
[ DEBUG ] web_log[nginx] init.go:105 starting log reader creating
[ DEBUG ] web_log[nginx] reader.go:73 open log file: /var/log/nginx/api.savaoo.fr.access.log
[ DEBUG ] web_log[nginx] init.go:110 created log reader, current file '/var/log/nginx/api.foobar.fr.access.log'
[ DEBUG ] web_log[nginx] init.go:116 starting parser creating
[ DEBUG ] web_log[nginx] init.go:122 last line: 'x.x.x.x - - [31/Aug/2022:11:51:10 +0200] "GET /api/v1/foobar HTTP/1.1" 200 2447 "-" "xxx" "xxx" rt=0.018 ut=0.016'
[ DEBUG ] web_log[nginx] parser.go:85 log_type is csv, skipping auto-detection
[ DEBUG ] web_log[nginx] parser.go:88 config: {FieldsPerRecord:-1 Delimiter:  TrimLeadingSpace:false Format:$remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time CheckField:0x1d27b80}
[ DEBUG ] web_log[nginx] init.go:128 created parser: csv: $remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time
[ INFO  ] web_log[nginx] job.go:188 check success
[ INFO  ] web_log[nginx] job.go:208 started, data collection interval 1s
[ DEBUG ] run[manager] run.go:43 tick 0

[andrewm4894]

Welcome @laupi !!

How long was it between seeing on agent dashboard and checking in cloud? Sometimes the metadata can be a little delayed when syncing for various reasons.

Although if its been longer than like 5 minutes or so that’s not right and i can try get someone from BE to help us maybe.

A netdata restart may also help if possible since on restart i think it would sync the metadata to cloud potentially a bit quicker.

Start, stop, or restart the Netdata Agent | Learn Netdata

[Manolis_Vasilakis]

Also @laupi can you please your netdata version (netdata -W buildinfo). Thanks!

[laupi]

The job is running for 2 days.
The agent is running inside docker (i follow the docker installation documentation)

@Manolis_Vasilakis

netdata -W buildinfo

bash-5.1# netdata -W buildinfo
Version: netdata v1.36.0-61-nightly
Configure options:  '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/usr/libexec' '--libdir=/usr/lib' '--with-zlib' '--with-math' '--with-user=netdata' '--without-bundled-protobuf' '--disable-dependency-tracking' '--disable-ebpf' 'CFLAGS=-O2 -pipe' 'LDFLAGS='
Install type: oci
    Binary architecture: x86_64
Features:
    dbengine:                   YES
    Native HTTPS:               YES
    Netdata Cloud:              YES 
    ACLK:                       YES
    TLS Host Verification:      YES
    Machine Learning:           YES
    Stream Compression:         YES
Libraries:
    protobuf:                YES (system)
    jemalloc:                NO
    JSON-C:                  YES
    libcap:                  NO
    libcrypto:               YES
    libm:                    YES
    tcalloc:                 NO
    zlib:                    YES
Plugins:
    apps:                    YES
    cgroup Network Tracking: YES
    CUPS:                    NO
    EBPF:                    NO
    IPMI:                    YES
    NFACCT:                  NO
    perf:                    YES
    slabinfo:                YES
    Xen:                     NO
    Xen VBD Error Tracking:  NO
Exporters:
    AWS Kinesis:             NO
    GCP PubSub:              NO
    MongoDB:                 YES
    Prometheus Remote Write: YES
bash-5.1# 

[laupi]

any update or someone who has already successfully parse and retrieve web_log ?

[andrewm4894]

@ilyam8 does the web_log config above all look ok to you?

@laupi can you share outpul of

grep 'web_log' /var/log/netdata/error/log

Just in case is something in the log that might be relevant but not visible when running it in debug mode.

[laupi]

I’m running netdata inside the provided docker container, so error.log is redirect to /dev/sdterr.

However, when inspecting docker logs, i see nothing about a web_log error

[Christopher_Akritid1]

In your original post I don’t actually see data getting collected from the collector. After the last line, do you actually see thing like the following (the specific ones are for apache)?
If you don’t, then I expect that the lines aren’t matching the filter’s regexp.

[ DEBUG ] run[manager] run.go:43 tick 0
CHART 'netdata.execution_time_of_web_log_apache' '' 'Execution time' 'ms' 'go.d' 'netdata.go_plugin_execution_time' 'line' '145000' '1' '' 'go.d' 'web_log'
DIMENSION 'time' '' 'absolute' '1' '1' ''

CHART 'web_log_apache.requests' '' 'Total Requests' 'requests/s' 'requests' 'web_log.requests' 'line' '70000' '1' '' 'go.d' 'web_log'
DIMENSION 'requests' '' 'incremental' '1' '1' ''

BEGIN 'web_log_apache.requests'
SET 'requests' = 0
END

CHART 'web_log_apache.excluded_requests' '' 'Excluded Requests' 'requests/s' 'requests' 'web_log.excluded_requests' 'stacked' '70001' '1' '' 'go.d' 'web_log'
DIMENSION 'unmatched' 'unmatched' 'incremental' '1' '1' ''

BEGIN 'web_log_apache.excluded_requests'
SET 'unmatched' = 0
END

CHART 'web_log_apache.requests_by_ip_proto' '' 'Requests By IP Protocol' 'requests/s' 'ip proto' 'web_log.ip_proto_requests' 'stacked' '70021' '1' '' 'go.d' 'web_log'
DIMENSION 'ipv4' 'ipv4' 'incremental' '1' '1' ''
DIMENSION 'ipv6' 'ipv6' 'incremental' '1' '1' ''

[laupi]

Thanks for your help.

So more log from the collector

bash-5.1# ./go.d.plugin -d -m web_log

[ DEBUG ] main[main] main.go:113 plugin: name=go.d, version=v0.37.2

[ DEBUG ] main[main] main.go:115 current user: name=root, uid=0

[ INFO ] main[main] agent.go:136 instance is started

[ INFO ] main[main] setup.go:42 loading config file

[ INFO ] main[main] setup.go:50 looking for 'go.d.conf' in [/etc/netdata /usr/lib/netdata/conf.d]

[ INFO ] main[main] setup.go:57 found '/etc/netdata/go.d.conf

[ INFO ] main[main] setup.go:64 config successfully loaded

[ INFO ] main[main] agent.go:140 using config: enabled 'true', default_run 'true', max_procs '0'

[ INFO ] main[main] setup.go:69 loading modules

[ INFO ] main[main] setup.go:88 enabled/registered modules: 1/71

[ INFO ] main[main] setup.go:93 building discovery config

[ INFO ] main[main] setup.go:123 looking for 'web_log.conf' in [/etc/netdata/go.d /usr/lib/netdata/conf.d/go.d]

[ INFO ] main[main] setup.go:139 found '/etc/netdata/go.d/web_log.conf

[ INFO ] main[main] setup.go:144 dummy/read/watch paths: 0/1/0

[ INFO ] discovery[manager] manager.go:92 registered discoverers: [file discovery: [file reader]]

[ INFO ] discovery[manager] manager.go:97 instance is started

[ INFO ] run[manager] run.go:32 instance is started

[ INFO ] build[manager] build.go:107 instance is started

[ INFO ] discovery[file manager] discovery.go:73 instance is started

[ INFO ] discovery[file reader] read.go:41 instance is started

[ INFO ] discovery[file reader] read.go:42 instance is stopped

[ DEBUG ] build[manager] build.go:154 received config group ('/etc/netdata/go.d/web_log.conf'): 1 jobs (added: 1, removed: 0)

[ DEBUG ] build[manager] build.go:303 building web_log[nginx] job, config: map[__provider__:file reader __source__:/etc/netdata/go.d/web_log.conf autodetection_retry:0 csv_config:map[format:$remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time] log_type:csv module:web_log name:nginx path:/var/log/nginx/api.xxxxx.fr.access.log priority:70000 update_every:1]

[ DEBUG ] web_log[nginx] init.go:33 skipping URL patterns creating, no patterns provided

[ DEBUG ] web_log[nginx] init.go:51 skipping custom fields creating, no custom fields provided

[ DEBUG ] web_log[nginx] init.go:76 skipping custom time fields creating, no custom time fields provided

[ DEBUG ] web_log[nginx] init.go:105 starting log reader creating

[ DEBUG ] web_log[nginx] reader.go:73 open log file: /var/log/nginx/api.xxxxx.fr.access.log

[ DEBUG ] web_log[nginx] init.go:110 created log reader, current file '/var/log/nginx/api.xxxxx.fr.access.log'

[ DEBUG ] web_log[nginx] init.go:116 starting parser creating

[ DEBUG ] web_log[nginx] init.go:122 last line: '80.11.x - - [07/Sep/2022:16:15:25 +0200] "GET /api/v1/t HTTP/1.1" 200 2354 "-" "xxxxx-v1.6.b8-" "api.xxxxx.fr" rt=0.008 ut=0.008'

[ DEBUG ] web_log[nginx] parser.go:85 log_type is csv, skipping auto-detection

[ DEBUG ] web_log[nginx] parser.go:88 config: {FieldsPerRecord:-1 Delimiter: TrimLeadingSpace:false Format:$remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time CheckField:0x1d27b80}

[ DEBUG ] web_log[nginx] init.go:128 created parser: csv: $remote_addr - - [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$host" rt=$request_time ut=$upstream_response_time

[ INFO ] web_log[nginx] job.go:188 check success

[ INFO ] web_log[nginx] job.go:208 started, data collection interval 1s

[ DEBUG ] run[manager] run.go:43 tick 0

CHART 'netdata.execution_time_of_web_log_nginx' '' 'Execution time' 'ms' 'go.d' 'netdata.go_plugin_execution_time' 'line' '145000' '1' '' 'go.d' 'web_log'

DIMENSION 'time' '' 'absolute' '1' '1' ''

CHART 'web_log_nginx.requests' '' 'Total Requests' 'requests/s' 'requests' 'web_log.requests' 'line' '70000' '1' '' 'go.d' 'web_log'

DIMENSION 'requests' '' 'incremental' '1' '1' ''

BEGIN 'web_log_nginx.requests'

SET 'requests' = 0

END

CHART 'web_log_nginx.excluded_requests' '' 'Excluded Requests' 'requests/s' 'requests' 'web_log.excluded_requests' 'stacked' '70001' '1' '' 'go.d' 'web_log'

DIMENSION 'unmatched' 'unmatched' 'incremental' '1' '1' ''

BEGIN 'web_log_nginx.excluded_requests'

SET 'unmatched' = 0

END

CHART 'web_log_nginx.requests_by_vhost' '' 'Requests By Vhost' 'requests/s' 'vhost' 'web_log.vhost_requests' 'stacked' '70016' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.requests_by_vhost'

END

CHART 'web_log_nginx.requests_by_ip_proto' '' 'Requests By IP Protocol' 'requests/s' 'ip proto' 'web_log.ip_proto_requests' 'stacked' '70021' '1' '' 'go.d' 'web_log'

DIMENSION 'ipv4' 'ipv4' 'incremental' '1' '1' ''

DIMENSION 'ipv6' 'ipv6' 'incremental' '1' '1' ''

BEGIN 'web_log_nginx.requests_by_ip_proto'

SET 'ipv4' = 0

SET 'ipv6' = 0

END

CHART 'web_log_nginx.current_poll_uniq_clients' '' 'Current Poll Unique Clients' 'clients' 'client' 'web_log.current_poll_uniq_clients' 'stacked' '70015' '1' '' 'go.d' 'web_log'

DIMENSION 'ipv4' 'ipv4' 'absolute' '1' '1' ''

DIMENSION 'ipv6' 'ipv6' 'absolute' '1' '1' ''

BEGIN 'web_log_nginx.current_poll_uniq_clients'

SET 'ipv4' = 0

SET 'ipv6' = 0

END

CHART 'web_log_nginx.requests_by_http_method' '' 'Requests By HTTP Method' 'requests/s' 'http method' 'web_log.http_method_requests' 'stacked' '70019' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.requests_by_http_method'

END

CHART 'web_log_nginx.requests_by_http_version' '' 'Requests By HTTP Version' 'requests/s' 'http version' 'web_log.http_version_requests' 'stacked' '70020' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.requests_by_http_version'

END

CHART 'web_log_nginx.requests_by_type' '' 'Requests By Type' 'requests/s' 'requests' 'web_log.type_requests' 'stacked' '70002' '1' '' 'go.d' 'web_log'

DIMENSION 'success' 'success' 'incremental' '1' '1' ''

DIMENSION 'bad' 'bad' 'incremental' '1' '1' ''

DIMENSION 'redirect' 'redirect' 'incremental' '1' '1' ''

DIMENSION 'error' 'error' 'incremental' '1' '1' ''

BEGIN 'web_log_nginx.requests_by_type'

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

CHART 'web_log_nginx.responses_by_status_code_class' '' 'Responses By Status Code Class' 'responses/s' 'responses' 'web_log.status_code_class_responses' 'stacked' '70003' '1' '' 'go.d' 'web_log'

DIMENSION '2xx' '2xx' 'incremental' '1' '1' ''

DIMENSION '5xx' '5xx' 'incremental' '1' '1' ''

DIMENSION '3xx' '3xx' 'incremental' '1' '1' ''

DIMENSION '4xx' '4xx' 'incremental' '1' '1' ''

DIMENSION '1xx' '1xx' 'incremental' '1' '1' ''

BEGIN 'web_log_nginx.responses_by_status_code_class'

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

CHART 'web_log_nginx.status_code_class_1xx_responses' '' 'Informational Responses By Status Code' 'responses/s' 'responses' 'web_log.status_code_class_1xx_responses' 'stacked' '70005' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

CHART 'web_log_nginx.status_code_class_2xx_responses' '' 'Successful Responses By Status Code' 'responses/s' 'responses' 'web_log.status_code_class_2xx_responses' 'stacked' '70006' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

CHART 'web_log_nginx.status_code_class_3xx_responses' '' 'Redirects Responses By Status Code' 'responses/s' 'responses' 'web_log.status_code_class_3xx_responses' 'stacked' '70007' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

CHART 'web_log_nginx.status_code_class_4xx_responses' '' 'Client Errors Responses By Status Code' 'responses/s' 'responses' 'web_log.status_code_class_4xx_responses' 'stacked' '70008' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

CHART 'web_log_nginx.status_code_class_5xx_responses' '' 'Server Errors Responses By Status Code' 'responses/s' 'responses' 'web_log.status_code_class_5xx_responses' 'stacked' '70009' '1' '' 'go.d' 'web_log'

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

CHART 'web_log_nginx.bandwidth' '' 'Bandwidth' 'kilobits/s' 'bandwidth' 'web_log.bandwidth' 'area' '70010' '1' '' 'go.d' 'web_log'

DIMENSION 'received' 'received' 'incremental' '8' '1000' ''

DIMENSION 'sent' 'sent' 'incremental' '-8' '1000' ''

BEGIN 'web_log_nginx.bandwidth'

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx'

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 1

BEGIN 'web_log_nginx.requests' 999202

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 999202

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 999202

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 999202

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 999202

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 999202

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 999202

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 999202

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 2

BEGIN 'web_log_nginx.requests' 999766

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 999766

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 999766

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 999766

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 999766

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 999766

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 999766

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 999766

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 3

BEGIN 'web_log_nginx.requests' 1000319

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 1000319

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 1000319

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 1000319

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 1000319

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 1000319

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 1000319

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 1000319

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 4

BEGIN 'web_log_nginx.requests' 1000595

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 1000595

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 1000595

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 1000595

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 1000595

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 1000595

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 1000595

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 1000595

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 5

BEGIN 'web_log_nginx.requests' 999286

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 999286

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 999286

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 999286

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 999286

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 999286

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 999286

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 999286

SET 'time' = 0

END

[ DEBUG ] run[manager] run.go:43 tick 6

BEGIN 'web_log_nginx.requests' 1000511

SET 'requests' = 0

END

BEGIN 'web_log_nginx.excluded_requests' 1000511

SET 'unmatched' = 0

END

BEGIN 'web_log_nginx.requests_by_vhost'

END

BEGIN 'web_log_nginx.requests_by_ip_proto' 1000511

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.current_poll_uniq_clients' 1000511

SET 'ipv4' = 0

SET 'ipv6' = 0

END

BEGIN 'web_log_nginx.requests_by_http_method'

END

BEGIN 'web_log_nginx.requests_by_http_version'

END

BEGIN 'web_log_nginx.requests_by_type' 1000511

SET 'success' = 0

SET 'bad' = 0

SET 'redirect' = 0

SET 'error' = 0

END

BEGIN 'web_log_nginx.responses_by_status_code_class' 1000511

SET '2xx' = 0

SET '5xx' = 0

SET '3xx' = 0

SET '4xx' = 0

SET '1xx' = 0

END

BEGIN 'web_log_nginx.status_code_class_1xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_2xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_3xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_4xx_responses'

END

BEGIN 'web_log_nginx.status_code_class_5xx_responses'

END

BEGIN 'web_log_nginx.bandwidth' 1000511

SET 'received' = 0

SET 'sent' = 0

END

BEGIN 'netdata.execution_time_of_web_log_nginx' 1000511

SET 'time' = 0

END

^C[ INFO ] main[main] agent.go:104 received interrupt signal (2). Terminating...

[ INFO ] run[manager] run.go:33 instance is stopped

[ INFO ] build[manager] build.go:108 instance is stopped

[ INFO ] discovery[file manager] discovery.go:74 instance is stopped

[ INFO ] discovery[manager] manager.go:98 instance is stopped

[ DEBUG ] web_log[nginx] reader.go:129 close log file: /var/log/nginx/api.xxxxx.fr.access.log

[ INFO ] web_log[nginx] job.go:209 stopped

[ INFO ] main[main] agent.go:137 instance is stopped

bash-5.1#

It seems quite similar to yours.

I see some error/warning in the docker container log, maybe it can help found the issue

2022-09-07 13:58:25: 12299: 148 '[localhost]:48532' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.57/0.03/0.61 ms) 200 '/api/v1/info'

2022-09-07 13:59:25: 12300: 155 '[localhost]:47652' 'CONNECTED'

2022-09-07 13:59:25: 12300: 155 '[localhost]:47652' 'DISCONNECTED'

2022-09-07 13:59:25: 12300: 155 '[localhost]:47652' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.55/0.03/0.58 ms) 200 '/api/v1/info'

2022-09-07 14:00:26: 12301: 155 '[localhost]:58962' 'CONNECTED'

2022-09-07 14:00:26: 12301: 155 '[localhost]:58962' 'DISCONNECTED'

2022-09-07 14:00:26: 12301: 155 '[localhost]:58962' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.60/0.04/0.64 ms) 200 '/api/v1/info'

2022-09-07 14:01:26: 12302: 153 '[localhost]:56794' 'CONNECTED'

2022-09-07 14:01:26: 12302: 153 '[localhost]:56794' 'DISCONNECTED'

2022-09-07 14:01:26: 12302: 153 '[localhost]:56794' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.59/0.04/0.63 ms) 200 '/api/v1/info'

2022-09-07 14:02:15: cgroup-name.sh: INFO: Running API command: curl "/var/run/docker.sock/containers/516f43xdtbhddthdth068106a9f58b3e5c88f032f/json"

curl: (3) URL using bad/illegal format or missing URL

2022-09-07 14:02:15: cgroup-name.sh: WARNING: cannot find the name of docker container '516f43xdtbhddthdth068106a9f58b3e5c88f032f'

2022-09-07 14:02:15: cgroup-name.sh: INFO: cgroup 'system.slice_docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope' is called '516f43c9bad0'

2022-09-07 14:02:15: netdata ERROR : MAIN : child pid 361795 exited with code 2.

2022-09-07 14:02:16: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1222 to 1222 batch=1

2022-09-07 14:02:18: cgroup-name.sh: INFO: Running API command: curl "/var/run/docker.sock/containers/516f43xdtbhddthdth068106a9f58b3e5c88f032f/json"

curl: (3) URL using bad/illegal format or missing URL

2022-09-07 14:02:18: cgroup-name.sh: WARNING: cannot find the name of docker container '516f43xdtbhddthdth068106a9f58b3e5c88f032f'

2022-09-07 14:02:18: cgroup-name.sh: INFO: cgroup 'system.slice_docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope' is called '516f43c9bad0'

2022-09-07 14:02:18: netdata ERROR : MAIN : child pid 361808 exited with code 2.

2022-09-07 14:02:18: /usr/libexec/netdata/plugins.d/cgroup-network INFO : MAIN : Using host prefix directory '/host'

2022-09-07 14:02:18: /usr/libexec/netdata/plugins.d/cgroup-network ERROR : MAIN : Cannot open pid_from_cgroup() file '/host/sys/fs/cgroup/system.slice/docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope/tasks'. (errno 2, No such file or directory)

2022-09-07 14:02:18: /usr/libexec/netdata/plugins.d/cgroup-network INFO : MAIN : running: exec /usr/libexec/netdata/plugins.d/cgroup-network-helper.sh --cgroup '/host/sys/fs/cgroup/system.slice/docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope'

2022-09-07 14:02:18: cgroup-network-helper.sh: INFO: searching for network interfaces of cgroup '/host/sys/fs/cgroup/system.slice/docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope'

2022-09-07 14:02:18: /usr/libexec/netdata/plugins.d/cgroup-network ERROR : MAIN : child pid 361821 exited with code 1.

2022-09-07 14:02:18: /usr/libexec/netdata/plugins.d/cgroup-network ERROR : MAIN : Cannot find a cgroup PID from cgroup '/host/sys/fs/cgroup/system.slice/docker-516f43xdtbhddthdth068106a9f58b3e5c88f032f.scope'

2022-09-07 14:02:18: netdata ERROR : MAIN : child pid 361820 exited with code 1.

2022-09-07 14:02:23: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1223 to 1223 batch=1

2022-09-07 14:02:26: 12303: 153 '[localhost]:35336' 'CONNECTED'

2022-09-07 14:02:26: 12303: 153 '[localhost]:35336' 'DISCONNECTED'

2022-09-07 14:02:26: 12303: 153 '[localhost]:35336' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.54/0.03/0.57 ms) 200 '/api/v1/info'

2022-09-07 14:02:26: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1224 to 1224 batch=1

2022-09-07 14:03:13: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1225 to 1236 batch=1

2022-09-07 14:03:26: 12304: 154 '[localhost]:57704' 'CONNECTED'

2022-09-07 14:03:26: 12304: 154 '[localhost]:57704' 'DISCONNECTED'

2022-09-07 14:03:26: 12304: 154 '[localhost]:57704' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.56/0.03/0.59 ms) 200 '/api/v1/info'

2022-09-07 14:04:26: 12305: 132 '[localhost]:37648' 'CONNECTED'

2022-09-07 14:04:26: 12305: 132 '[localhost]:37648' 'DISCONNECTED'

2022-09-07 14:04:26: 12305: 132 '[localhost]:37648' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.55/0.03/0.58 ms) 200 '/api/v1/info'

2022-09-07 14:05:26: 12306: 148 '[localhost]:49924' 'CONNECTED'

2022-09-07 14:05:26: 12306: 148 '[localhost]:49924' 'DISCONNECTED'

2022-09-07 14:05:26: 12306: 148 '[localhost]:49924' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.61/0.04/0.65 ms) 200 '/api/v1/info'

2022-09-07 14:06:26: 12307: 153 '[localhost]:36090' 'CONNECTED'

2022-09-07 14:06:26: 12307: 153 '[localhost]:36090' 'DISCONNECTED'

2022-09-07 14:06:26: 12307: 153 '[localhost]:36090' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.55/0.03/0.58 ms) 200 '/api/v1/info'

2022-09-07 14:07:26: 12308: 148 '[localhost]:42432' 'CONNECTED'

2022-09-07 14:07:26: 12308: 148 '[localhost]:42432' 'DISCONNECTED'

2022-09-07 14:07:26: 12308: 148 '[localhost]:42432' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.55/0.03/0.58 ms) 200 '/api/v1/info'

2022-09-07 14:07:59: tc-qos-helper.sh: WARNING: FireQOS is not installed on this system. Use FireQOS to apply traffic QoS and expose the class names to netdata. Check https://github.com/netdata/netdata/tree/master/collectors/tc.plugin#tcplugin

2022-09-07 14:07:59: tc-qos-helper.sh: WARNING: Cannot find file '/usr/lib/netdata/conf.d/tc-qos-helper.conf'.

2022-09-07 14:07:59: tc-qos-helper.sh: WARNING: Cannot find file '/etc/netdata/tc-qos-helper.conf'.

2022-09-07 14:08:26: 12309: 132 '[localhost]:48690' 'CONNECTED'

2022-09-07 14:08:26: 12309: 132 '[localhost]:48690' 'DISCONNECTED'

2022-09-07 14:08:26: 12309: 132 '[localhost]:48690' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.60/0.04/0.64 ms) 200 '/api/v1/info'

2022-09-07 14:09:26: 12310: 154 '[localhost]:37990' 'CONNECTED'

2022-09-07 14:09:26: 12310: 154 '[localhost]:37990' 'DISCONNECTED'

2022-09-07 14:09:26: 12310: 154 '[localhost]:37990' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.61/0.04/0.65 ms) 200 '/api/v1/info'

2022-09-07 14:10:27: 12311: 155 '[localhost]:46250' 'CONNECTED'

2022-09-07 14:10:27: 12311: 155 '[localhost]:46250' 'DISCONNECTED'

2022-09-07 14:10:27: 12311: 155 '[localhost]:46250' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.59/0.04/0.63 ms) 200 '/api/v1/info'

2022-09-07 14:11:27: 12312: 154 '[localhost]:58596' 'CONNECTED'

2022-09-07 14:11:27: 12312: 154 '[localhost]:58596' 'DISCONNECTED'

2022-09-07 14:11:27: 12312: 154 '[localhost]:58596' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.53/0.03/0.56 ms) 200 '/api/v1/info'

2022-09-07 14:12:14: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1237 to 1238 batch=1

2022-09-07 14:12:27: 12313: 154 '[localhost]:33412' 'CONNECTED'

2022-09-07 14:12:27: 12313: 154 '[localhost]:33412' 'DISCONNECTED'

2022-09-07 14:12:27: 12313: 154 '[localhost]:33412' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.59/0.04/0.63 ms) 200 '/api/v1/info'

2022-09-07 14:12:27: ACLK RES [2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth (xxxxx.fr)]: ALERTS SENT from 1239 to 1239 batch=1

2022-09-07 14:13:27: 12314: 153 '[localhost]:52342' 'CONNECTED'

2022-09-07 14:13:27: 12314: 153 '[localhost]:52342' 'DISCONNECTED'

2022-09-07 14:13:27: 12314: 153 '[localhost]:52342' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.54/0.03/0.57 ms) 200 '/api/v1/info'

2022-09-07 14:14:27: 12315: 154 '[localhost]:34222' 'CONNECTED'

2022-09-07 14:14:27: 12315: 154 '[localhost]:34222' 'DISCONNECTED'

2022-09-07 14:14:27: 12315: 154 '[localhost]:34222' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.57/0.05/0.61 ms) 200 '/api/v1/info'

2022-09-07 14:15:27: 12316: 155 '[localhost]:41544' 'CONNECTED'

2022-09-07 14:15:27: 12316: 155 '[localhost]:41544' 'DISCONNECTED'

2022-09-07 14:15:27: 12316: 155 '[localhost]:41544' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.60/0.04/0.64 ms) 200 '/api/v1/info'

2022-09-07 14:16:27: 12317: 132 '[localhost]:35274' 'CONNECTED'

2022-09-07 14:16:27: 12317: 132 '[localhost]:35274' 'DISCONNECTED'

2022-09-07 14:16:27: 12317: 132 '[localhost]:35274' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.58/0.04/0.61 ms) 200 '/api/v1/info'

2022-09-07 14:17:27: 12318: 132 '[localhost]:60240' 'CONNECTED'

2022-09-07 14:17:27: 12318: 132 '[localhost]:60240' 'DISCONNECTED'

2022-09-07 14:17:27: 12318: 132 '[localhost]:60240' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.57/0.04/0.61 ms) 200 '/api/v1/info'

2022-09-07 14:18:27: 12319: 153 '[localhost]:49750' 'CONNECTED'

2022-09-07 14:18:27: 12319: 153 '[localhost]:49750' 'DISCONNECTED'

2022-09-07 14:18:27: 12319: 153 '[localhost]:49750' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.57/0.04/0.61 ms) 200 '/api/v1/info'

2022-09-07 14:19:06: 0: 358 '[ACLK]:5' 'DATA' (sent/all = 44260/609733 bytes -93%, prep/sent/total = 0.18/12.72/12.90 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/charts'

2022-09-07 14:19:06: 0: 354 '[ACLK]:1' 'DATA' (sent/all = 1543/4270 bytes -64%, prep/sent/total = 0.04/1.49/1.53 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=system.cpu&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:06: 0: 356 '[ACLK]:3' 'DATA' (sent/all = 946/2498 bytes -62%, prep/sent/total = 0.34/1.31/1.64 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=system.load&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:06: 0: 357 '[ACLK]:4' 'DATA' (sent/all = 616/2115 bytes -71%, prep/sent/total = 0.18/1.41/1.59 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=system.swap&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:06: 0: 353 '[ACLK]:0' 'DATA' (sent/all = 792/2065 bytes -62%, prep/sent/total = 0.17/1.31/1.48 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=mem.available&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:06: 0: 357 '[ACLK]:4' 'DATA' (sent/all = 1117/2506 bytes -55%, prep/sent/total = 0.09/2.98/3.07 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=system.io&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:06: 0: 358 '[ACLK]:5' 'DATA' (sent/all = 1117/2529 bytes -56%, prep/sent/total = 0.20/2.49/2.70 ms) 200 '/node/2ffe9fe7-b409-496f-8cf1-dthdthdthdstdth/api/v1/data?after=1662559445&before=1662560346&context=system.net&dimensions=&format=json&group=average&gtime=0&options=allow_past%7Cflip%7Cjsonwrap%7Cms&points=37&show_dimensions=1&timeout=9000'

2022-09-07 14:19:27: 12320: 153 '[localhost]:41806' 'CONNECTED'

2022-09-07 14:19:27: 12320: 153 '[localhost]:41806' 'DISCONNECTED'

2022-09-07 14:19:27: 12320: 153 '[localhost]:41806' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.53/0.03/0.56 ms) 200 '/api/v1/info'

2022-09-07 14:20:28: 12321: 132 '[localhost]:50548' 'CONNECTED'

2022-09-07 14:20:28: 12321: 132 '[localhost]:50548' 'DISCONNECTED'

2022-09-07 14:20:28: 12321: 132 '[localhost]:50548' 'DATA' (sent/all = 5484/5484 bytes -0%, prep/sent/total = 0.53/0.03/0.56 ms) 200 '/api/v1/info'

[Christopher_Akritid1]

I don’t see how the cgroup naming errors can have anything to do with missing web_log charts and I’m not sure what else could have gone wrong. @Manolis_Vasilakis what do you think?

[laupi]

I don’t think so for cgroup but posts the log anyway

Just to be sure, i should retrieve the charts from my web_log parsing in this list ?