filebeat error
baole1998 opened this issue · comments
I'm trying to run your project in docker but it keep raising an error that can not connect to logstash from filebeat. Any ideal for this ?
filebeat has dependency with logstash. So, you should check logstash container work well and 5044 port mapping to open outside.
Logstash container was fine as I tested it with postman. I was also exec into filebeat container to ping to logstash:5044 and with was connected. Then I remove everything, repo, docker image, container, clone your repo back again and only run it with docker-compose at first and still, the error was the same.
Okay, then you should wait for 1~2 minutes.
Because logstash takes longer to ready than filebeat.
Here is my logs
- filebeat
2022-11-30T11:12:08.638Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:12:08.638Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 1 reconnect attempt(s)
2022-11-30T11:12:11.398Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:12:11.398Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 2 reconnect attempt(s)
2022-11-30T11:12:18.327Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:12:18.327Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 3 reconnect attempt(s)
2022-11-30T11:12:26.945Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:12:26.945Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 4 reconnect attempt(s)
2022-11-30T11:12:35.698Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":31}},"total":{"ticks":360,"time":{"ms":367},"value":360},"user":{"ticks":330,"time":{"ms":336}}},"handles":{"limit":{"hard":65536,"soft":32768},"open":11},"info":{"ephemeral_id":"64dafced-9d58-4ee4-ace0-6077f180dd2b","uptime":{"ms":30134}},"memstats":{"gc_next":8393888,"memory_alloc":5790736,"memory_total":11621672,"rss":48709632},"runtime":{"goroutines":40}},"filebeat":{"events":{"active":6,"added":9,"done":3},"harvester":{"open_files":3,"running":3,"started":3}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":6,"filtered":3,"published":6,"retry":24,"total":9}}},"registrar":{"states":{"current":3,"update":3},"writes":{"success":4,"total":4}},"system":{"cpu":{"cores":2},"load":{"1":3.94,"15":0.45,"5":1.27,"norm":{"1":1.97,"15":0.225,"5":0.635}}}}}}
2022-11-30T11:12:57.783Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:12:57.783Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 5 reconnect attempt(s)
2022-11-30T11:13:05.698Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":1}},"total":{"ticks":370,"time":{"ms":6},"value":370},"user":{"ticks":340,"time":{"ms":5}}},"handles":{"limit":{"hard":65536,"soft":32768},"open":11},"info":{"ephemeral_id":"64dafced-9d58-4ee4-ace0-6077f180dd2b","uptime":{"ms":60133}},"memstats":{"gc_next":8393888,"memory_alloc":6066816,"memory_total":11897752},"runtime":{"goroutines":40}},"filebeat":{"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":6,"retry":6}}},"registrar":{"states":{"current":3}},"system":{"load":{"1":4.61,"15":0.62,"5":1.71,"norm":{"1":2.305,"15":0.31,"5":0.855}}}}}}
2022-11-30T11:13:35.701Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30},"total":{"ticks":370,"time":{"ms":5},"value":370},"user":{"ticks":340,"time":{"ms":5}}},"handles":{"limit":{"hard":65536,"soft":32768},"open":11},"info":{"ephemeral_id":"64dafced-9d58-4ee4-ace0-6077f180dd2b","uptime":{"ms":90135}},"memstats":{"gc_next":8299072,"memory_alloc":6432928,"memory_total":12432488,"rss":360448},"runtime":{"goroutines":40}},"filebeat":{"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":6}}},"registrar":{"states":{"current":3}},"system":{"load":{"1":4.14,"15":0.72,"5":1.89,"norm":{"1":2.07,"15":0.36,"5":0.945}}}}}}
2022-11-30T11:13:43.834Z ERROR pipeline/output.go:100 Failed to connect to backoff(async(tcp://logstash:5044)): dial tcp 172.17.0.3:5044: connect: connection refused
2022-11-30T11:13:43.834Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(async(tcp://logstash:5044)) with 6 reconnect attempt(s)
2022-11-30T11:13:43.835Z INFO pipeline/output.go:105 Connection to backoff(async(tcp://logstash:5044)) established
2022-11-30T11:14:05.698Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":5}},"total":{"ticks":380,"time":{"ms":14},"value":380},"user":{"ticks":350,"time":{"ms":9}}},"handles":{"limit":{"hard":65536,"soft":32768},"open":12},"info":{"ephemeral_id":"64dafced-9d58-4ee4-ace0-6077f180dd2b","uptime":{"ms":120134}},"memstats":{"gc_next":8299072,"memory_alloc":5790128,"memory_total":14074200,"rss":372736},"runtime":{"goroutines":41}},"filebeat":{"events":{"active":-6,"done":6},"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":6,"batches":1,"total":6},"read":{"bytes":6},"write":{"bytes":675}},"pipeline":{"clients":1,"events":{"active":0,"retry":12},"queue":{"acked":6}}},"registrar":{"states":{"current":3,"update":6},"writes":{"success":1,"total":1}},"system":{"load":{"1":2.56,"15":0.7,"5":1.72,"norm":{"1":1.28,"15":0.35,"5":0.86}}}}}}
2022-11-30T11:13:43.835Z INFO pipeline/output.go:105 Connection to backoff(async(tcp://logstash:5044)) established
Okay let me check it, thanks