Netdata Community

Monitor shell script and all its processes

Hello,

I’m developing a pipeline for research and am exited to use Netdata to monitor the resources that it is taking. I’m very new to Netdata, and I can’t seem to find the monitoring I’m looking for in the documentation, hence my post here.

The pipeline I have will run multiple scripts and tools, each of which could spawn sub processes. Here is an example of a moment in time where my pipeline is running using the pstree command

|-sshd-+-sshd---sshd---bash---seq2mgs.sh---R---sh---reformat.sh---java-+-pigz---3*[{pigz}]
        |      |                                                               |-pigz---5*[{pigz}]
        |      |                                                               `-20*[{java}]
        |      |-2*[sshd---sshd---sftp-server]
        |      `-sshd---sshd---bash---pstree

The seq2mgs.sh script is the main script, which invokes R scripts that then start other processes / scripts. What I’d like to do is to monitor / export each of the sub-processes’ CPU and memory usage at set intervals (e.g. every 5 sec) and be able to calculate the afterwards the parts of the pipeline which high / low resource usage.

I have figured out how to add processes to the apps.groups.conf (like R or my own script), but I’m not sure how I can keep track of the process tree at any point in time so I know how to aggregate the data in the end.

Ideally I’d like a table / json with the CPU / Memory per process, and the parent process that it is linked to (if any) with seq2mgs.sh being the root.

I’m looking forward to any suggestions
PJ