Graphing / Alerting on systemd service status

Help
1

Hi Everyone,

Sorry for the beginner question but loving the platform so far. I’ve been scratching my head trying to get this working for a while. I’m looking to get the status of a service into the dashboard.

I’ve enabled systemd monitoring on the go.d file. Configured updated the go.d/systemdunits.conf and verified with the debug that this is now tracked and set to 1 indicating active which is perfect.

[ DEBUG ] systemdunits[myservice] collect.go:68 got total/loaded 1/1 units
BEGIN 'systemdunits_myservice.service_unit_state' 9999861
SET 'myservice.service' = 1
END

My question is, how can I get a graph into the dashboard, I couldn’t find the relevant metric when searching the drop down box.

Also, which .conf file under /usr/lib/netdata/conf.d/health.d/ would I use to track its state and alert if it ever is stopped?

Thanks!

2

Will do, thanks again mate!

3

I am very happy with these news @belcast

Please, let us know if you need any help. :slight_smile:

2 Likes
4

Hey @Thiago-Marques-0 ,

Ah, I didn’t realise the lack of a space in go.d.conf would invalidate the attribute. I had totally missed it when I was troubleshooting but after you prompted me to look at the log and the spacing of the config file I noticed it still hadn’t initialised. Thanks for that
I changed
systemdunits:yes to
systemdunits: yes
Made a reload and it worked!

I got alerting working as well by creating a new health.d/abc.conf

 alarm: abc_status
 on: systemdunits_abc.service_unit_state
 lookup: max -1
 every: 5m
 green: $this = 1
 warn: $this > 1
 crit: $this > 1
 info: The status of the abc service.

I tuned it down to 1 second and tested and its working perfectly :slight_smile: it was literally just the missing space in the attribute. Thanks so much for your help Thiago!

5

Hello @belcast ,

You do not need to say sorry, it is always a pleasure to help our users.

When the chart is enabled, you will see a chart like this:

!

1605460183757-units.png
1605460183757-units.png1573×468 83.9 KB

To have this chart, I executed the following steps:

$ cd /etc/netdata/
$ ./edit-config go.d.conf

When the editor is open, I remove the comment from the line systemdunits: yes, please, pay attention that I only removed the channel (#), I kept all the spaces that were in front the name.

After the first step, I enabled the chart with the following steps:

cd /etc/netdata
./edit-config go.d/systemdunits.conf

And I removed the comments of the lines I wanted, my final configuration file is:

jobs:
  - name: service-units
    include:
      - '*.service'

Again I only removed the comments without to remove the spaces.

If a necessary space was removed or some wrong character is added, you will have error messages like this inside your /var/log/netdata/error.log:

$ grep systemdunits /var/log/netdata/error.log
2020-11-15 13:37:14: go.d WARNING: discovery[file reader] parse '/etc/netdata/go.d/systemdunits.conf': yaml: unmarshal errors:

Finally, for you create an alarm for these charts, you can follow our guide, but before to go deep on this, my suggestion is we fix the chart load. Please, take a look in your error log as I showed and adjust the config files. After we fix this first problem, I will also help you with the alarm.

Best regards!

2 Likes