Giters

logstash-plugins / logstash-input-s3

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error: can't convert nil into String

runningman84 opened this issue · comments

commented

Our setup contains two logstash nodes, node 1 writes all logs to an s3 bucket, node 2 should read this bucket.

This was working fine with logstash 2.4.x

But logstash 5.x shows an error message.

  • Version: 5.1.1
  • Operating System: Ubuntu 14.04
  • Config File:

logstash 1 output:

output {
   s3 {
     id => "output_s3_1"
      region => 'eu-central-1'
      bucket => 'prod-log-loggingbucket-XXXXXX'
      temporary_directory => '/data/logstash/s3_out'
      prefix => 'logstash/'
      time_file => 5
      size_file => 4096000
      codec => 'json_lines'
   }
}

logstash 2 input

input {
  s3 {
    id => "input_s3_1"
    region => 'eu-central-1'
    bucket => 'dev-log-loggingbucket-XXXXXX'
    temporary_directory => '/data/logstash/s3_in'
    delete => true
    codec => 'json_lines'
    prefix => 'logstash/'
    tags => ["input_s3"]
  }
}

Error on logstash 2:

[2016-12-29T11:24:19,545][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::S3 id=>"input_s3_1", region=>"eu-central-1", bucket=>"dev-log-loggingbucket-XXXXXX", temporary_directory=>"/data/logstash/s3_in", delete=>true, codec=><LogStash::Codecs::JSONLines id=>"json_lines_f7c89b1e-dd7f-43e7-8823-a4dbfc393842", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, prefix=>"logstash/", tags=>["input_s3"], enable_metric=>true, interval=>60>
  Error: can't convert nil into String
commented

Did you get this resolved? i'm hitting the same thing with 5.1.1

commented

I have the same problem with Logstash 5.1.2.- with debug logging enabled, I saw that the s3 input didn't have sincedb_path defined.

The documentation says this is not mandatory (although it defaults to 'nil'), and indeed with logstash 2.2.4, the same s3 input configuration auto-created the file /var/lib/logstash/.sincedb_6c32b368013344df55137b2d340ed524.

If I manually set sincedb_path, logstash works fine.

So looks like the behaviour of sincedb_path has changed in V5.

commented

@ph please advise or update the docs.

commented

AndrewStickler's comment seems to be the answer. I had the same issue and defining this resolved it.

commented

Yep, @AndrewStickler's solution worked for me also, set sincedb_path.

Logs show as;
Using the provided sincedb_path {:sincedb_path=>"/var/lib/logstash/.sincedb_6c32b368013344df55137b2d340ed524"}"