active_processes
OS: Any
This alert presents the percentage of used PIDs. If this alert gets raised it means that your system is experiencing high system process IDs (PID) space utilization.
If this value is 100% then the system can not start new processes.
This alert is raised on a warning state when the percentage of used PIDs exceeds the value of 90% and in critical state when the metric exceeds 95% used PIDs.
Note also that zombie processes could be responsible for high used PID percentage.
What are "PIDs"
A “PID” (i.e., process identification number) is an identification number that is automatically assigned to
eachprocess
when it is created on a Unix-like operating system.
Aprocess
is an executing (i.e., running) instance of a program. Each process is guaranteed a unique PID, which is always a non-negative integer. 1
As you can understand, a system has a finite number of PIDs, as it can run a limited number of processes at any given time.
Zombie Processes
A zombie process is one that never received a signal from the parent process that created it, a child process is one that has its origin in a higher level process known as the parent process that is in charge of sending the signals to the child processes generated by it to indicate that their life span has ended. So, if the parent process isn’t programmed properly it can spawn zombie processes. 2
So if the system has a lot of zombie processes, it can’t repurpose the PIDs used on those processes.
On Linux, the max PID is runtime configurable and by using
sysctl /kernel/pid_max
you can see that max value.
Furthermore, large numbers of processes are not optimal for reasons other than PID exhaustion. Each process on the system has some fixed overhead in the form of some memory usage (ir reaches up to a few hundred bytes on Linux) and scheduling overhead. Even if you are not exhausting available PIDs, you generally do not want any more processes running than you absolutely need, though this only truly matters in an HPC environment. (HPC stands for “high-performance computing”)
References and Sources
Troubleshooting Section
Terminate processes you don't need
Linux
Use the ps
command;
root@netdata~ # ps
it will display all the active processes
in your system alongside with their PIDs
.
From there you can determine which processes you don’t need and use the kill
command to terminate them;
root@netdata~ # kill <the pid number you want to terminate>
Note: It would be helpful to close any unneeded processes, but Netdata strongly suggests knowing exactly what
processes you are closing and being certain that they are not necessary.
Check for zombie processes
As we have seen, a system can report high percentage of used PIDs due to zombie processes (among other things).
You can’t simply kill zombie processes. So to clean your system up from these processes, you should locate the parent of that process. Killing the parent would result in the process getting inherited by pid 1
which will wait on it and eventually clear it from the process table.
- Use
ps
;
Freebsd and MacOS
root@netdata~ # ps -al | grep Z
Linux
root@netdata~ # ps -elf | grep Z
This command will show you the processes currently at a zombie state.
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
...
S
stands for state
, on this column the zombie processes have a Z
.
PID
is the PID of the process
PPID
is the PID of the parent of that process
CMD
Should give you more info about the process and help you determine if it is needed or not
After you have decided that you want to clean up a zombie process, you can;
kill -s SIGCHLD <pid>
where<pid>
is thePPID
of the zombie process.
This will send a signal to the parent process telling it to clean up its zombie processes. If the parent isn’t
programmed properly, it might be ignoring the SIGCHILD
signal we would be sending. In this case, we would need to kill
the parent, so the zombie can get inherited by pid 1
which will wait on it and eventually clear it from the process table.
To do so:
root@netdata~ # kill <PID>
Where <PID>
is the PPID
(parent process PID) of the zombie process.
Note: Netdata strongly suggests knowing exactly what processes you are closing and being certain that they are not necessary.