logstash-plugins / logstash-patterns-core

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

HTTPD24_ERRORLOG fails in if enabling loglevel between trace1 and trace8, due to number in loglevel

arberg opened this issue · comments

Logstash information:

$> apache2 -v
Server version: Apache/2.4.41 (Ubuntu)
Server built:   2023-03-08T17:32:54

I'm using Logstash 8.2.3, but problematic grok-pattern is still in master as seen here:
https://github.com/logstash-plugins/logstash-patterns-core/blob/main/patterns/ecs-v1/httpd

Log example (from apache2)

[Thu Jan 25 12:07:44.355300 2024] [mpm_event:trace1] [pid 1633595:tid 140648225171200] event.c(1573): All workers are busy or dying, will close 0 keep-alive connections

Apache2 config to enable this log
LogLevel warn mpm_event:trace8
or in the general case, regardless of which plugins you use
LogLevel trace1

Problem Cause
The problem is caused by usage of this LOGLEVEL definition from grok-patterns

LOGLEVEL ([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo?(?:rmation)?|INFO?(?:RMATION)?|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)

Possible solution
These patterns work, instead of modifying LOGLEVEL, I have added a new copy with a new name, to avoid changing others usage of LOGLEVEL

HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[(?:%{WORD:[apache][error][module]})?:%{LOGLEVEL_HTTP:[log][level]}\] \[pid %{POSINT:[process][pid]:int}(:tid %{INT:[process][thread][id]:int})?\](?: \(%{POSINT:[apache][error][proxy][error][code]?}\)%{DATA:[apache][error][proxy][error][message]}:)?(?: \[client %{IPORHOST:[source][address]}(?::%{POSINT:[source][port]:int})?\])?(?: %{DATA:[error][code]}:)? %{GREEDYDATA:message}
LOGLEVEL_HTTP ([Aa]lert|ALERT|[Tt]race(\d*)|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo?(?:rmation)?|INFO?(?:RMATION)?|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)

Above patterns work when tested in https://grokconstructor.appspot.com/do/match#result