Whenever I try to build Netdata on a Pi 4, it crashes the entire machine. My hunch is that it’s that Netdata always builds using 4 threads, breaks, and then leaves no time for anything else on the machine
Console output looks like the following:
/tmp/netdata-protobuf-sd4N7P/protobuf-3.17.3# env CFLAGS=-fPIC CXXFLAGS= LDFLAGS=make -j4
make all-recursive
[...]
CXX google/protobuf/util/message_differencer.lo
INFO: task kworker/2:2:122 blocked for more than 120 seconds.
Tainted: G C E 5.11.0-1016-raspi #17-Ubuntu
Indeed, using 4 threads might peg the machine… Is it a 4 or 8GB memory model? Can you check dmesg or journalctl if there is more info on what else might be going on ? On my 8GB Pi4, it builds without some noticeable problem…
It could be memory exhustion or perhaps a problem with the filesystem/sd card that could be causing it.
If you’re feeling a bit adventurous, can you try to manually issue the compile command inside the /tmp/netdata-protobuf-sd4N7P/protobuf-3.17.3 folder using a smaller value of -j to see if it completes ok?
In this case, it’s probably memory exhaustion causing thrashing due to excessive swapping (the system in question has only 1GB of RAM per the OP).
I’m working on a PR right now to add support for passing arbitrary options to make during the install process, which will let users override the default number of jobs, though with only 1GB of RAM, I’m not sure how much that may help. protobuf is C++, which is already rather resource-intensive to compile, and on top of that it does things that result in non-trivial memory usage while building it.
The long term solution will be to use native binary packages once we merge https://github.com/netdata/netdata/pull/10769 (though currently I have no idea when that may get merged).
We’ve now got a partial fix for this in the master branch of the agent repository. Starting from tonight’s nightly (or the next stable release if you are not using nightlies), you can set MAKEOPTS="-j1" in the environment when running the installer to force it to serialize the build process, which may enable an install to work successfully in the described scenario.