I have been developing a custom plugin to collect monitoring data from one of our application to netdata.
The conceptual issue is the number of dimensions in some of the charts are dynamic. The examples and plugins that are based on the Framework classes (e.g. SimpleService, URLService, etc) all appear to have a static chart and dimensions definition, and to output it once when the plugin is started.
Great thing: it works, and I have the charts as I want them.
Bad thing: when I put my plugin in the /usr/libexec/netdata/python.d, all the other python plugins don’t run, as if my plugin was hijacking the internal scheduling.
Questions:
is there any signal/character/command that I need to output on the standard output that tells netdata “I am done for this iteration”
what is the proper way to force reoutput of CHART/DIMENSION when using the framework classes such as SimpleService?
Yes, I forgot to mention that I did run in debug mode. It runs fine; no error show up, the expected set of CHART/DIMENSION/BEGIN/SET/END commands are issued. Actually, the resulting charts are OK and perfect in every way; except from the fact that I can’t seem to run more than just my own plugin.
When debugging all modules, it correctly shows the debug info for reading the configuration file, and “built XX job config”. Then when it comes to the custom plugin, just prints the stdout of the module. I paste here a cropped version that shows the idea (to avoid hundreds of lines ;):
I have left two modules for which I have an explicit configuration: postgres and weblog. The log progresses nicely in alphabetical order through the modules, until it reaches mine:
Yes… to go back to the beginning of the thread, I attempted to approach things this way because the number of dimensions in my charts may be dynamic (e.g. when a new stream shows up)
The examples of plugin implementation use a static array for storing charts/dimensions which are only transmitted to netdata once on plugin startup.
If you know how to force the refresh of dimensions, I’d be happy to move my collector to a child of the SimpleService class
At the same time, I do believe I followed the documentation to the letter, so it is a bit misleading…
I come to resolve my own issue and share if anyone has the same issue of:
Having a dynamic number of charts
Having dynamic dimensions inside those charts.
The way I got it to work is to implement a child of SimpleService.
Then, in the get_data() function:
Redefineself.order = [], and self.definitions = { }
Clear self.charts: you need to remake the call that is done (for you) in the SimpleService constructor, i.e. self.charts = Charts(job_name=self.actual_name, priority=configuration["priority"], cleanup=configuration["chart_cleanup"], get_update_every=self.get_update_every, module_name=self.module_name). You will need to have deep copied the configuration OrderedDict that netdata has given you in your own class constructor.
Fill in your new self.order and self.definitions, based on the templates, dynamically constructed from the data that you have gathered.
Explicitly call SimpleService.create(self) from within get_data(). This will regenerate the chart content and transmit the commands to the backend.
If any netdata team member is watching this thread, I would like to kindly point out that:
the claim that “Any program that can print a few values to its standard output can become a Netdata external plugin” is a bit misleading as the plugin scheduling basically makes this impossible to just output data and expect it to work alongside other things (on this page → External plugins overview | Learn Netdata )
I have already stated in another thread there is next to no documentation in the source code… I don’t know if you have an internal codebase with comments that you just strip out for release… or if you just remember everything… but it makes it unnecessarily complex to figure out the inner workings…
This is invaluable feedback. I can’t thank you enough! We are actually working on a python collector guide, so your feedback will probably be incorporated into the guide, so you are actively contributing in making this easier for everyone .
If you are interested, I would love your feedback ahead of Guide’s release. Pinging our documentation guru for extra visibility @joel .
Just chiming in in case others find useful - i think the alarms collector also has a sort of example of dynamically creating charts and/or dims on the fly as needed.
Using the add_dimension() and del_dimension() methods of self.charts
I’m not sure if thats quite the same as the use cases needed here but felt like it was maybe close enough to share some more info in case anyone in future finds it useful.