[Error] Logstash 8.11.3 pipeline.conf (logstash.conf) issue
F9Alejandro opened this issue · comments
When trying to run the logstash portion of the compose it throws an error about the formatting of the .conf file. I am using the file that is provided in this repo and haven't made any changes to it.
Logstash error:
elastic-stack-logstash01-1 | [2023-12-23T05:23:21,079][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
elastic-stack-es01-1 | {"@timestamp":"2023-12-23T05:22:36.930Z", "log.level": "INFO", "message":"loaded module [x-pack-aggregate-metric]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"docker-eck"}
elastic-stack-logstash01-1 | [2023-12-23T05:23:21,097][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 1, column 1 (byte 1)", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:931:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}
elastic-stack-logstash01-1 | [2023-12-23T05:23:21,104][INFO ][logstash.runner ] Logstash shut down.
elastic-stack-logstash01-1 | [2023-12-23T05:23:21,107][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
elastic-stack-logstash01-1 | org.jruby.exceptions.SystemExit: (SystemExit) exit
elastic-stack-logstash01-1 | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:808) ~[jruby.jar:?]
elastic-stack-logstash01-1 | at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:767) ~[jruby.jar:?]
elastic-stack-logstash01-1 | at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?]
Any recommendations for trying to fix this issue?
I've figured it out, logstash needs to know where is the pipeline .conf file, relevant part from docker-compose.yml:
logstash01:
depends_on:
es01:
condition: service_healthy
kibana:
condition: service_healthy
image: docker.elastic.co/logstash/logstash:${STACK_VERSION}
labels:
co.elastic.logs/module: logstash
user: root
volumes:
- certs:/usr/share/logstash/certs
- logstashdata01:/usr/share/logstash/data
- "./logstash_ingest_data/:/usr/share/logstash/ingest_data/"
- "./logstash.conf:/usr/share/logstash/pipeline/logstash.conf:ro"
environment:
- xpack.monitoring.enabled=false
- ELASTIC_USER=elastic
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- ELASTIC_HOSTS=https://es01:9200
command:
- -f
- /usr/share/logstash/pipeline/
Add the command part to the code.
thanks for the heads up. i'll test the one out.
i am unable to recreate this error and do not need to specify the -f command as this is taken care of by passing in the logstash.conf to the default pipeline directory.