Nenzyz / lager_logstash_backend

Send lager logs to logstash

Lager Logstash Backend

Backend for lager data into log stash. This a modification of mhald repo, this fork was done because we need to correctly handle other scenarios not contemplated in the original lib to being able to integrate lager with the EKL Stack. This repo adds UTC handling for logs in the format ISO8601 required by Logstash, because the old one was sent with UTC added at the end, making it invalid. Also this backend sends the function where the log comes from, and other minor improvements.

A sample project that shows how to use it can be found here.

Configure erlang with Rebar3

You need to add this project and lager as dependency to your rebar.config,

{deps, [
        lager,
        {lager_logstash_backend,
           {git,"https://github.com/lambdaclass/lager_logstash_backend",
              {tag, "0.1"}}}
]}.

then also in rebar.config set your sys_config to configure the backend.

{relx,
 [
  {sys_config, "./conf/sys.config"},
  ...

Then in sys.config do specify sasl to work in UTC, this step is required because logstash needs UTC timestamps.

{sasl, [{utc_log, true}]}

Finally configure the lager backends in sys.config too:

{lager,
  [
   {handlers,
    [
     {lager_logstash_backend,
      [
        {level,             info},
        {logstash_host,     "logstash_host"},
        {logstash_port,     9125},
        {node_role,         "erlang"},
        {node_version,      "0.0.1"},
        {metadata, [
                   {account_token,  [{encoding, string}]},
                   {client_os,      [{encoding, string}]},
                   {client_version, [{encoding, string}]}
                  ]}
      ]}
     ]}
  ]}

Testing

On the erlang shell use

$ rebar3 shell
1> lager:log(error, self(), "Error notice").

Configure Logstash

Install logstash and setup the sample.config with information about your logstash server.

Sample logstash config:

input {
    udp  {
        codec => "json"
        port  => 9125
        type  => "erlang"
    }
}

filter {
  mutate {
    add_field => { "env" => "debug" }
    replace => { "host" => "ip_address" }
  }
}

output {
    elasticsearch { hosts => ["elasticsearch:9200"] }
}