Today I’ve created a new account and installed Netdata on a Debian server. Basically everything worked out of the box, but I am not seeing my systemd logs. When I go to my Configurations page, I only see two tabs: Collectors and Health.
On another Debian server that I setup quite a long time ago, I see three tabs on the Configurations page: Collectors, Health and Logs. Under logs is systems-journal, and I do indeed have access to my logs on this server.
I have no idea why the logs are not added to this new server, and I also don’t know how to add them. To be honest I can’t remember what steps (if any) I did all that time ago on the older server. Any ideas why I am not seeing this on the new server, and how to add it?
ilyam8
April 4, 2024, 1:17pm
2
Hi. Show Netdata build info, it is
/usr/sbin/netdata -W buildinfo
# or
/opt/netdata/usr/sbin/netdata -W buildinfo
/usr/sbin/netdata -W buildinfo
Packaging:
Netdata Version ____________________________________________ : v1.45.2
Installation Type __________________________________________ : binpkg-deb
Package Architecture _______________________________________ : x86_64
Package Distro _____________________________________________ :
Configure Options __________________________________________ : dummy-configure-command
Default Directories:
User Configurations ________________________________________ : /etc/netdata
Stock Configurations _______________________________________ : /usr/lib/netdata/conf.d
Ephemeral Databases (metrics data, metadata) _______________ : /var/cache/netdata
Permanent Databases ________________________________________ : /var/lib/netdata
Plugins ____________________________________________________ : /usr/libexec/netdata/plugins.d
Static Web Files ___________________________________________ : /var/lib/netdata/www
Log Files __________________________________________________ : /var/log/netdata
Lock Files _________________________________________________ : /var/lib/netdata/lock
Home _______________________________________________________ : /var/lib/netdata
Operating System:
Kernel _____________________________________________________ : Linux
Kernel Version _____________________________________________ : 5.10.0-21-amd64
Operating System ___________________________________________ : Debian GNU/Linux
Operating System ID ________________________________________ : debian
Operating System ID Like ___________________________________ : unknown
Operating System Version ___________________________________ : 11 (bullseye)
Operating System Version ID ________________________________ : none
Detection __________________________________________________ : /etc/os-release
Hardware:
CPU Cores __________________________________________________ : 8
CPU Frequency ______________________________________________ : 2445000000
RAM Bytes __________________________________________________ : 32879128576
Disk Capacity ______________________________________________ : 163842097152
CPU Architecture ___________________________________________ : x86_64
Virtualization Technology __________________________________ : kvm
Virtualization Detection ___________________________________ : systemd-detect-virt
Container:
Container __________________________________________________ : none
Container Detection ________________________________________ : systemd-detect-virt
Container Orchestrator _____________________________________ : none
Container Operating System _________________________________ : none
Container Operating System ID ______________________________ : none
Container Operating System ID Like _________________________ : none
Container Operating System Version _________________________ : none
Container Operating System Version ID ______________________ : none
Container Operating System Detection _______________________ : none
Features:
Built For __________________________________________________ : Linux
Netdata Cloud ______________________________________________ : YES
Health (trigger alerts and send notifications) _____________ : YES
Streaming (stream metrics to parent Netdata servers) _______ : YES
Back-filling (of higher database tiers) ____________________ : YES
Replication (fill the gaps of parent Netdata servers) ______ : YES
Streaming and Replication Compression ______________________ : YES (zstd lz4 gzip)
Contexts (index all active and archived metrics) ___________ : YES
Tiering (multiple dbs with different metrics resolution) ___ : YES (5)
Machine Learning ___________________________________________ : YES
Database Engines:
dbengine ___________________________________________________ : YES
alloc ______________________________________________________ : YES
ram ________________________________________________________ : YES
none _______________________________________________________ : YES
Connectivity Capabilities:
ACLK (Agent-Cloud Link: MQTT over WebSockets over TLS) _____ : YES
static (Netdata internal web server) _______________________ : YES
h2o (web server) ___________________________________________ : YES
WebRTC (experimental) ______________________________________ : NO
Native HTTPS (TLS Support) _________________________________ : YES
TLS Host Verification ______________________________________ : YES
Libraries:
LZ4 (extremely fast lossless compression algorithm) ________ : YES
ZSTD (fast, lossless compression algorithm) ________________ : YES
zlib (lossless data-compression library) ___________________ : YES
Brotli (generic-purpose lossless compression algorithm) ____ : NO
protobuf (platform-neutral data serialization protocol) ____ : YES (system)
OpenSSL (cryptography) _____________________________________ : YES
libdatachannel (stand-alone WebRTC data channels) __________ : NO
JSON-C (lightweight JSON manipulation) _____________________ : YES
libcap (Linux capabilities system operations) ______________ : NO
libcrypto (cryptographic functions) ________________________ : YES
libyaml (library for parsing and emitting YAML) ____________ : YES
Plugins:
apps (monitor processes) ___________________________________ : YES
cgroups (monitor containers and VMs) _______________________ : YES
cgroup-network (associate interfaces to CGROUPS) ___________ : YES
proc (monitor Linux systems) _______________________________ : YES
tc (monitor Linux network QoS) _____________________________ : YES
diskspace (monitor Linux mount points) _____________________ : YES
freebsd (monitor FreeBSD systems) __________________________ : NO
macos (monitor MacOS systems) ______________________________ : NO
statsd (collect custom application metrics) ________________ : YES
timex (check system clock synchronization) _________________ : YES
idlejitter (check system latency and jitter) _______________ : YES
bash (support shell data collection jobs - charts.d) _______ : YES
debugfs (kernel debugging metrics) _________________________ : YES
cups (monitor printers and print jobs) _____________________ : YES
ebpf (monitor system calls) ________________________________ : YES
freeipmi (monitor enterprise server H/W) ___________________ : YES
nfacct (gather netfilter accounting) _______________________ : YES
perf (collect kernel performance events) ___________________ : YES
slabinfo (monitor kernel object caching) ___________________ : YES
Xen ________________________________________________________ : YES
Xen VBD Error Tracking _____________________________________ : NO
Logs Management ____________________________________________ : YES
Exporters:
AWS Kinesis ________________________________________________ : NO
GCP PubSub _________________________________________________ : NO
MongoDB ____________________________________________________ : YES
Prometheus (OpenMetrics) Exporter __________________________ : YES
Prometheus Remote Write ____________________________________ : YES
Graphite ___________________________________________________ : YES
Graphite HTTP / HTTPS ______________________________________ : YES
JSON _______________________________________________________ : YES
JSON HTTP / HTTPS __________________________________________ : YES
OpenTSDB ___________________________________________________ : YES
OpenTSDB HTTP / HTTPS ______________________________________ : YES
All Metrics API ____________________________________________ : YES
Shell (use metrics in shell scripts) _______________________ : YES
Debug/Developer Features:
Trace All Netdata Allocations (with charts) ________________ : NO
Developer Mode (more runtime checks, slower) _______________ : NO
ps faxu | grep [n]etdata
postgres 2964655 0.8 0.0 220940 31656 ? Ss 13:25 0:02 \_ postgres: 13/main: netdata postgres [local] idle
root 2954277 0.0 0.0 31768 11984 ? Ss 11:50 0:00 /lib/systemd/systemd-journald netdata
netdata 2959504 1.2 0.2 891120 70696 ? SNsl 12:35 0:39 /usr/sbin/netdata -P /run/netdata/netdata.pid -D
netdata 2959533 0.0 0.0 63344 13748 ? SNl 12:35 0:00 \_ /usr/sbin/netdata --special-spawn-server
netdata 2959765 1.1 0.1 210988 33808 ? SNl 12:35 0:37 \_ /usr/libexec/netdata/plugins.d/apps.plugin 1
netdata 2959788 0.0 0.0 4080 2912 ? SN 12:35 0:01 \_ bash /usr/libexec/netdata/plugins.d/tc-qos-helper.sh 1
netdata 2959790 0.0 0.0 42040 29072 ? SNl 12:35 0:01 \_ /usr/bin/python3 /usr/libexec/netdata/plugins.d/python.d.plugin 1
netdata 2959792 0.7 0.2 1295792 65880 ? SNl 12:35 0:24 \_ /usr/libexec/netdata/plugins.d/go.d.plugin 1
root 2959798 0.0 0.0 11316 2456 ? SN 12:35 0:00 \_ /usr/libexec/netdata/plugins.d/nfacct.plugin 1
netdata 2959810 0.0 0.0 11264 2360 ? SN 12:35 0:00 \_ /usr/libexec/netdata/plugins.d/debugfs.plugin 1
root 2959812 0.0 0.0 432328 29604 ? SNl 12:35 0:01 \_ /usr/libexec/netdata/plugins.d/ebpf.plugin 1
The logs tab shows a never-ending loading indicator:
ilyam8
April 4, 2024, 1:36pm
6
System-journal plugin is not running.
Try restarting Netdata and if the problem still exists, show:
# installed plugin packages
dpkg -l | grep netdata-plugin
# plugins/permissions in the plugins.d directory
ls -l /usr/libexec/netdata/plugins.d/
# logs since the last restart
journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata
It wasn’t installed, apt-get install netdata-plugin-systemd-journal and service netdata restart seems to have fixed it. The Logs tab show up as expected now.
Not sure why the install script didn’t install this plugin?
ilyam8
April 4, 2024, 2:30pm
8
Not sure why the install script didn’t install this plugin?
I’m not sure either - installation output would be helpful, but I understand you didn’t save it. How did you install Netdata? Using the kickstart.sh script, right?
Yup, kickstart.sh. And no I didn’t keep the output for that.
ilyam8
April 4, 2024, 2:50pm
10
Probably your system is configured not to install recommended packages by default.
grep "APT::Install-Recommends" /etc/apt/apt.conf.d/*
Ah, true.
It’s weird that nowhere in the integrations info or even on systemd journal plugin | Learn Netdata is any info about getting this installed. The script doesn’t warn about recommended packages not getting installed.
Maybe docs and/or the script can be improved? But at least my problem is solved. Thanks!
