mozilla-services / hindsight

Hindsight - light weight data processing skeleton

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Issue while decoding syslog

KalaiarasiAyyadurai opened this issue · comments

I had installed hindsight to parse the syslogs using lua modules(lpeg.sub_decoder_util and decoders.syslog).It reads the log and writes the decoded log in the output.Decoded log contains some junk characters as shown below.

Configuration:
filename = "file.lua"
input_filename = "syslog.log"
send_decode_failures = true
decoder_module = "decoders.syslog"

decoders_syslog = {
template = "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%",
}

Input(syslog.log):
Feb 13 14:25:19 ubuntu sshd[7192]: Accepted publickey for foobar from 216.160.83.56 port 4242 ssh2

Output:
▒d▒e5J▒▒\H▒▒Mi▒▒▒▒▒Ѽ▒"
input.file2?Accepted publickey for foobar from 216.160.83.56 port 4242 ssh2@▒8JubuntuR

Please, help to fix.

Where is that output being captured from? If it is from an output plugin, it looks like it is using a Heka protobuf encoder you can switch to heka_debug for human readable output. You can also examine the internal queue using lsb_heka_cat <output_path>/input/0.log

Thanks for your direction Trink.The output is being captured from input plugin.I got your message on switching the encoder.I will try this and revert back of I have any issues.