Failing test on Logstash_Default_Plugins_21

Question

Failing test on Logstash_Default_Plugins_21

ph opened this issue 9 years ago · comments

Pier-Hugues Pellerin commented 9 years ago

http://build-eu-00.elastic.co/view/LS%202.1/job/Logstash_Default_Plugins_21/8/console

  1) LogStash::Inputs::S3 when working with logs when event doesn't have a `message` field deletes the temporary file
     Failure/Error: events = fetch_events(config)
     ArgumentError:
       no time information in ""
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:413:in `read'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:408:in `newer?'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:114:in `list_new_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:110:in `list_new_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:144:in `process_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/spec/support/helpers.rb:5:in `fetch_events'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/spec/inputs/s3_spec.rb:247:in `(root)'
     # ./vendor/bundle/jruby/1.9/gems/rspec-wait-0.0.8/lib/rspec/wait.rb:46:in `(root)'
     # ./rakelib/test.rake:54:in `(root)'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:240:in `execute'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:179:in `invoke_with_call_chain'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:106:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:106:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:78:in `run'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:75:in `run'

  2) LogStash::Inputs::S3 when working with logs when event doesn't have a `message` field should process events
     Failure/Error: events = fetch_events(config)
     ArgumentError:
       no time information in ""
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:413:in `read'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:408:in `newer?'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:114:in `list_new_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:110:in `list_new_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/lib/logstash/inputs/s3.rb:144:in `process_files'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/spec/support/helpers.rb:5:in `fetch_events'
     # ./vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.2/spec/inputs/s3_spec.rb:242:in `(root)'
     # ./vendor/bundle/jruby/1.9/gems/rspec-wait-0.0.8/lib/rspec/wait.rb:46:in `(root)'
     # ./rakelib/test.rake:54:in `(root)'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:240:in `execute'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:179:in `invoke_with_call_chain'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:106:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:106:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:78:in `run'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
     # ./vendor/bundle/jruby/1.9/gems/rake-10.4.2/lib/rake/application.rb:75:in `run'

Pier-Hugues Pellerin · Answer 1 · Thu Nov 19 2015 04:30:27 GMT+0800 (China Standard Time)

Seem to be related to the sincedb

Pier-Hugues Pellerin · Answer 2 · Thu Nov 19 2015 05:00:03 GMT+0800 (China Standard Time)

This can happen if the file is created but we didn't have the time to flush the buffer to disk and this make the next executions fails.

The fix is to make sure there is content in the file before actually trying to parse a time. If there is no content we will return the current time, this is the same behavior when the file doesn't exist.