sensu-plugins / sensu-plugins-logs

This plugin provides native instrumentation for monitoring log files or system logs via journald for regular expressions, and a Sensu handler for logging Sensu events to log files.

Home Page:http://sensu-plugins.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

check_log configuration issues

opened this issue · comments

Hi,

I'm using the check_log to check my logfiles on certain keywords, for example "error". If a new error is written to the log file, sensu gives me an error. After a couple of seconds, the critical error disappears. How should I configure my check in order to check the log file on a word like ERROR and that the error stays as long the error exists in the log file?

My check looks a bit like this:

{
  "checks": {
    "check_log": {
      "command": "/etc/sensu/plugins/check-log.rb -f /opt/test/logs/testDocker.log -q 'ERROR'",
      "interval": 10,
      "subscribers": ["subscriber1"]
    }
  }
}

Thanks in advance

@romariolodder

So if I understand this:

  1. an error gets written to the log file
  2. sensu detects the error
  3. sensu fires off an alert through a handler
  4. sensu clears the error after a short length of time

If this is the case it is because the check ran again and did not detect and error message, therefore sending a clear to sensu which cleared the alert. If you don't want this then you can modify your handler to not auto clear if an error is still in the stash.

@romariolodder

Are you still fighting this or are you good?

Any update on this one? I have same issue. At the moment I have to modify the script to create a new alert internally and submit it to local sensu client whenever it is not OK so there will never be an auto clear. This works but I end up with two alerts on the dashboard.

You can set "auto_resolve: false" in you check configuration for that alert. However, this means that it will stay triggered. You need to manually clear the alert via dashboard/uchiwa or API when feel that the issue is properly resolved.

@romariolodder @gk642 Does this work for you? #1 (comment)

This is intended behavior and there is a valid way to handle the use case. Closing due to inactivity, feel free to comment back or re-open if you would like to discuss further.