Kafka not starting after all services restart

[Aleix Sala Bach]

Hi all,

After doing some modifications to logstash config I proceed with a sudo rock restart, to restart all services and load the new configuration for logstash.

The ansible playbook seems to fail restarting kafka

TASK [Register service status] ****************************************************************************************************************************** ok: [simplerockbuild.simplerock.lan] => (item=stenographer) ok: [simplerockbuild.simplerock.lan] => (item=logstash) ok: [simplerockbuild.simplerock.lan] => (item=elasticsearch) ok: [simplerockbuild.simplerock.lan] => (item=zookeeper) ok: [simplerockbuild.simplerock.lan] => (item=lighttpd) ok: [simplerockbuild.simplerock.lan] => (item=docket) ok: [simplerockbuild.simplerock.lan] => (item=kibana) ok: [simplerockbuild.simplerock.lan] => (item=fsf) failed: [simplerockbuild.simplerock.lan] (item=kafka) => {"ansible_loop_var": "item", "changed": false, "cmd": "systemctl status -n 3 kafka*", "delta": "0:00 :00.035322", "end": "2020-03-25 11:34:30.751250", "item": "kafka", "msg": "non-zero return code", "rc": 3, "start": "2020-03-25 11:34:30.715928", "stderr": " ", "stderr_lines": [], "stdout": "● kafka.service - Kafka\n Loaded: loaded (/usr/lib/systemd/system/kafka.service; enabled; vendor preset: disabled)\n Dro p-In: /etc/systemd/system/kafka.service.d\n └─override.conf\n Active: failed (Result: exit-code) since Wed 2020-03-25 11:33:18 UTC; 1min 11s ago\ n Process: 7364 ExecStart=/usr/share/kafka/bin/kafka-server-start.sh $KAFKA_SERVER_CONFIG (code=exited, status=1/FAILURE)\n Main PID: 7364 (code=exited, sta tus=1/FAILURE)\n\nMar 25 11:33:18 simplerockbuild.simplerock.lan systemd[1]: kafka.service: main process exited, code=exited, status=1/FAILURE\nMar 25 11:33: 18 simplerockbuild.simplerock.lan systemd[1]: Unit kafka.service entered failed state.\nMar 25 11:33:19 simplerockbuild.simplerock.lan systemd[1]: kafka.serv ice failed.", "stdout_lines": ["● kafka.service - Kafka", " Loaded: loaded (/usr/lib/systemd/system/kafka.service; enabled; vendor preset: disabled)", " D rop-In: /etc/systemd/system/kafka.service.d", " └─override.conf", " Active: failed (Result: exit-code) since Wed 2020-03-25 11:33:18 UTC; 1min 11 s ago", " Process: 7364 ExecStart=/usr/share/kafka/bin/kafka-server-start.sh $KAFKA_SERVER_CONFIG (code=exited, status=1/FAILURE)", " Main PID: 7364 (code=e xited, status=1/FAILURE)", "", "Mar 25 11:33:18 simplerockbuild.simplerock.lan systemd[1]: kafka.service: main process exited, code=exited, status=1/FAILURE" , "Mar 25 11:33:18 simplerockbuild.simplerock.lan systemd[1]: Unit kafka.service entered failed state.", "Mar 25 11:33:19 simplerockbuild.simplerock.lan syst emd[1]: kafka.service failed."]} ok: [simplerockbuild.simplerock.lan] => (item=filebeat) ok: [simplerockbuild.simplerock.lan] => (item=suricata) ok: [simplerockbuild.simplerock.lan] => (item=zeek)
Then of course kafka service is down and logstash complains it cannot connect to it. I'm using the lastest iso rocknsm-2.5.0-2002.iso

Can you help me with this?

[Aleix Sala Bach]

If I restart the whole server then it works properly, it's just when I try to restart services using the rock command.