Problem UDP as input

[sinaei]

Hi,

i am using logstash but I face a problem that is related to UDP input

it can not split the line as input, so my message contains of multiline messages not one line message.

[David Butler]

Are you using the correct logstash configuration?

input {
  udp {
    port => 5228
    codec => json_lines
  }
}

[Ivan Sotnikov]

Guys, I use the same settings (udp with json_lines codec) and very often I see strange things - looks like logstash can't parse the following:

Two json objects in one line.

[David Butler]

@isotnikov Take a look at the code in formatter.rb:21:

      if data.is_a?(String) && data.start_with?('{')
        data = (JSON.parse(message) rescue nil) || message
      end

This means that if JSON.parse had trouble parsing your JSON, it will fall back to using the raw string as the message. It's possible you have some invalid encoding somewhere in your string. You might try replacing the stdlib JSON library with Oj via oj_mimic_json.

[David Butler]

@isotnikov After a closer look, I see what you meant about two JSON strings on one line. JSON.parse would definitely fall over on that. Are you using a gem such as Lograge to send JSON strings to LogStashLogger?

[David Butler]

@sinaei @isotnikov The LogstashLogger::Formatter appends a newline to every log message it receives before sending it to the underlying log device. I can't think of any reason UDP would be randomly dropping newlines, but it's not impossible either. Is it possible that your application is sending two JSON strings in one line to the logger?

[David Butler]

After some more research, I think this may be a bug in logstash. The logstash UDP listener reads a fixed number of bytes at a time into a buffer, queues up the buffers, and decodes each chunk of bytes one at a time. If the end of one log message just happens to coincide with the end of a buffer, it's possible that logstash codec is combining two log messages into one.

The LogStash JsonLines codec also has a fallback when JSON fails to parse:

def decode(data)

    @lines.decode(data) do |event|
      begin
        yield LogStash::Event.new(JSON.parse(event["message"]))
      rescue JSON::ParserError => e
        @logger.info("JSON parse failure. Falling back to plain-text", :error => e, :data => data)
        yield LogStash::Event.new("message" => event["message"])
      end
    end
  end

You might try checking the log file for your logstash listener.

[sinaei]

Yes, also I have read about it, and think that UDP reads a fixed number of bytes, TCP is not like this,

[David Butler]

Closing, since this appears to be a bug in the logstash listener implementation, and it's not possible to fix this from LogStashLogger.

[David Butler]

I added a note to the readme about this problem in the troubleshooting section.