Combine multiple prometheus endpoints

I have a bunch of processes on the same host each producing the same groups of metrics for prometheus. I’m configuring them as just “name, url” pairs. The name is being used as a metric prefix by the collector.

I would like to see the prometheus metrics grouped by app on the same chart, with the prometheus name as the dimension. Is this possible?

If not, what can I do instead?

You use the prometheus collector, right? What exactly do you put in the configuration for each job?

We have a sort of hidden feature which allows users to add custom labels to any job, which should do the trick, but first we’ll need to check why we’re not already putting them under the same “context”, which would organize them under the same chart in Netdata.

The configuration is very simply - {name: "x", url: "http://localhost:123/metrics"} with a different name for each endpoint, and the metrics are just metric_total 123

Ok, so apparently we have an undocumented “app: xxx” option that you can add to each job, which should treat all these endpoints as part of the same app and give them the same context. Add it to a couple of them and let’s see if you start seeing them grouped together in a chart with a common context. The job name should still appear as an option you can “group by” on the composite chart, let us know how it goes.

We’re not quite sure why we decided to include the job name in the context by default, but we’ll need to experiment a bit first, before we can go ahead and remove it.

1 Like