dbengine high availability

  1. On the recent call for the v1.37 release, mention was made of having a HA setup for the db engine. If I understood it correctly, the idea was that two parent nodes could stream their child-data to one another (horizontal streaming, if you will).
    Is this in the v1.37 release and if so what do we need to do to enable it?

2a. Is there any plan to provide a virtual networking component (or plugin to one) so that Netdata in a HA configuration can automatically register things like DNS names and or Virtual IP addresses?

2b. Assuming HTTPS is disabled, does the parent or child node actually care about things like DNS mismatches and the such-like that you’d get if you configured child agents to stream to, say, the domain-name/FQDN for a DNS ‘load-balancer’ in active-passive configuration?

Thanks in advance! :slight_smile:

You don’t really need DNS or Virtual IPs, because of the way our HA works. It’s not well documented yet, so I’ll try to summarize and we’ll improve the documentation a lot, in the next few days.

On a HA setup, you have N children and M parents. Every single one of those N+M nodes must have in its stream.conf a destination that lists ALL the M parents. i.e. every child and every parent must say destination = parent1 parent2 ... parentM.

This setup guarantees that every one of the M parents will have data for all N children (the way it works is not important right now).

The DNS names, virtual IPs etc. are important when you want to ensure that you’ll get back a response, regardless of whether a number of parents are currently live or not. We support this at the moment only via Netdata Cloud, which maintains the state of each parent (and each claimed child) and routes requests as needed. If children are also directly claimed and connected to the cloud, requests would actually be routed to the children themselves, if all parents were unavailable.

We haven’t implemented any other way of getting responses from the parents via a hostname or virtual IP, without the cloud. It’s entirely possible of course, but not something we have at the moment.

Please let us know what you think and if this covers you.

1 Like

Hi,

this doesn’t work for me - i have 2 parents and both of them are in the stream.conf. Only one of them (the first one where the connection is successful). I added a streaming from parent 1 to parent 2 and vice versa so they forward all streams they receive to the other one. This works but i have some warnings in the log when parent 2 then tries to stream the metrics back to Parent 1 and so on :wink:

So do each of the entries in destination get the metrics or only one of them? The Documentation [1] states the following: “A space-separated list of parent nodes to attempt to stream to, with the first available parent receiving metrics”

1: Streaming reference | Learn Netdata