Add compatibility for pipeline.ecs_compatibility: v8

[Toby Sutor]

Logstash in version 8.x by default uses pipeline.ecs_compatibility: v8. However, the Dead Letter Queue Input Plugin does not seem to be adapted to the latest ECS compatibility mode.

It fails with the following stack trace on Logstash 8.x:

Dec 22 05:23:45 HOSTNAME logstash[1362879]: [2023-01-10T09:02:13,453][ERROR][logstash.javapipeline ][dlq] Pipeline error {:pipeline_id=>"PIPELINE_NAME", :exception=>#<Java::OrgLogstash::LockException: Existing `dlg_reader.lock` file in [/var/lib/logstash-dlq/dead_letter_queue/elastic_agent]. Only one DeadLetterQueueReader with `cleanConsumed` set is allowed per Dead Letter Queue.>, :backtrace=>["org.logstash.common.io.DeadLetterQueueReader.<init>(org/logstash/common/io/DeadLetterQueueReader.java:117)", "org.logstash.input.DeadLetterQueueInputPlugin.lazyInitQueueReader(org/logstash/input/DeadLetterQueueInputPlugin.java:90)", "org.logstash.input.DeadLetterQueueInputPlugin.register(org/logstash/input/DeadLetterQueueInputPlugin.java:114)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:77)", "jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:568)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:442)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:306)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_input_minus_dead_letter_queue_minus_2_dot_0_dot_0.lib.logstash.inputs.dead_letter_queue.register(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-input-dead_letter_queue-2.0.0/lib/logstash/inputs/dead_letter_queue.rb:77)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:234)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1865)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:233)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_inputs(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:392)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:317)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.run(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:143)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:309)", "java.lang.Thread.run(java/lang/Thread.java:833)"], "pipeline.sources"=>["/etc/logstash/conf.d/dead_letter_queue/01-input.conf", "/etc/logstash/conf.d/dead_letter_queue/50-filter.conf", "/etc/logstash/conf.d/dead_letter_queue/99-output.conf"], :thread=>"#<Thread:0x72ab0b8d run>"}
Dec 22 05:23:45 HOSTNAME logstash[1362879]: [2023-01-10T09:02:13,455][INFO ][logstash.javapipeline ][dlq] Pipeline terminated {"pipeline.id"=>"PIPELINE_NAME"}
Dec 22 05:23:45 HOSTNAME logstash[1362879]: [2023-01-10T09:02:13,459][ERROR][logstash.agent ] Failed to execute action {:id=>:dlq, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Reload<dlq>, action_result: false", :backtrace=>nil}

It works, however, as expected with pipeline.ecs_compatibility: v1. So, it seems like it was not made compatible with the new v8 ECS mode.