Dear Forum,
I run Netdata on a couple of FreeBSD systems. FreeBSD version is 12.3. I am running Netdata version 1.37.
Problem I have: Netdata does not send an alert when disk usage threshold is reached.
On one of the systems I have a /var mountpoint with the following disk usage:
root@system:~ # df -m
Filesystem 1M-blocks Used Avail Capacity Mounted on
/dev/vtbd0p4 7916 5982 1300 82% /var
Netdata actually reports also 82.1% percentage of disk usage for this system on the /var mountpoint.
The health.d/disks.conf has:
template: disk_space_usage
on: disk.space
class: Utilization
type: System
component: Disk
os: freebsd
hosts: *
families: !/dev !/dev/* !/run !/run/* *
calc: $used * 100 / ($avail + $used)
units: %
every: 5m
warn: $this > (($status >= $WARNING ) ? (75) : (90))
crit: $this > (($status == $CRITICAL) ? (90) : (98))
delay: up 1m down 15m multiplier 1.5 max 1h
info: disk $family space utilization
to: sysadmin
And my health_alarm_notify config has:
role_recipients_email[sysadmin]=“${DEFAULT_RECIPIENT_EMAIL}”
I do receieve alert notiications for other checks. As the usage is between 75 and 90% I should receieve an alert, but I do not.
How can I troubleshoot further? What commands can I run to solve this? Where can I see why Netdata does not send an alert, on what grounds?
Thank you in advance for any pointers!
With kind regards,
Lars
Hi,
I am now using the standard disks.conf file for the health check on this system (from /usr/local/lib/netdata/conf.d/health.d). See also the Netdata blog post How to monitor the disk usage on your infrastructure | Netdata Blog.
I still do not get any alert although the disk usage of the /var mountpoint is still 82%.
The health.log shows:
2023-02-10 11:39:59: [FQDN]: Alert event for [disk_space./var.disk_space_usage], value [82.2%], status [CLEAR].
With kind regards,
Lars
For some reason does the standard health.d/disks.conf not work. Or I do not understand it. I expect an alarm when a mountpoint hits a uasga of > 80%, but it does not do that.
When I configure:
warn: $this > 80
crit: $this > 90
repeat: warning 240m critical 120m
I get the results I want and expect.
So this is SOLVED NOW!
welcome @Lars_Wittebrood !
great to hear is solved - i think maybe it could have been the delay? i know sometimes that can be a little confusing
Hi @Lars_Wittebrood !
It is indeed not really clear what our stock alert is doing with the following:
warn: $this > (($status >= $WARNING ) ? (75) : (90))
crit: $this > (($status == $CRITICAL) ? (90) : (98))
But basically what it does is that it takes into account whether the alert is already in a WARNING or CRITICAL state, and if so, the values are different to trigger the event.
So WARNING will be raised on 2 occasions:
If it’s already WARNING or CRITICAL, it will do so if the value is over 75.
If not, it will raise when the value is over 90.
In other words, if the alert is clear, you will get a warning when the value reaches 90 and it will stay in warning until it drops <75.
But yes, you can customize the alert as it suits your needs!