I have configured web_log to parse logs from HAProxy (which is configured to log in its httplog format) using a custom regexp log type.
Everything is working fine, except that I am frequently receiving web_log_1m_unmatched alerts because of how HAProxy logs various “error” events.
Even though my regexp pattern does parse this error-line variant properly:
192.168.1.1:61942 [27/Oct/2023:01:02:03.227] frontend backend/<NOSRV> -1/-1/-1/-1/0 400 0 - - CR-- 29/28/0/0/0 0/0 "<BADREQ>"
web_log is unhappy because it is unable to parse the “request” portion:
[ INFO ] web_log[haproxy] collect.go:63 unmatched line: regexp parse: assign 'request': assign 'BADREQ': bad request
The other problematic log lines are ones like this:
192.168.1.1:59481 [27/Oct/2023:01:02:04.466] web/2: Connection closed during SSL handshake
192.168.1.1:59483 [27/Oct/2023:01:02:04.446] web/2: SSL handshake failure
Is there some kind of ignore_pattern configuration option that would simply ignore lines that matched the given Matcher pattern, instead of reporting them as “mismatched”? (Such lines could even be reported as a separate ignored_requests metric and not included in the excluded_requests metric.)
Note that this isn’t simply about erroneous “mismatched alerts” — the bigger problem is that if the last log line just happens to be an error when Netdata starts, the HAProxy web_log job doesn’t start at all:
[ WARN ] web_log[haproxy] weblog.go:129 check failed: parse last line: regexp parse: assign 'request': assign 'BADREQ': bad request (...)
Any ideas/suggestions welcome.