Kafka Fails. Socket error occurred: zoo1/172.28.0.2:2181

Question

Kafka Fails. Socket error occurred: zoo1/172.28.0.2:2181

thearyanahmed opened this issue 3 years ago · comments

I've tried all of the docker-compose files given in this repository.
After spinning up the containers I get

Socket error occurred: zoo1/172.28.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.28.0.2:2181. Will not attempt to authenticate using SASL (unknown error)

Docker compose

version: '2.1'

services:
  zoo1:
    image: zookeeper:3.4.9
    hostname: zoo1
    ports:
      - "2181:2181"
    environment:
      ZOO_MY_ID: 1
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zoo1:2888:3888
    volumes:
      - ./zk-single-kafka-single/zoo1/data:/data
      - ./zk-single-kafka-single/zoo1/datalog:/datalog

  kafka1:
    image: confluentinc/cp-kafka:5.5.1
    hostname: kafka1
    ports:
      - "9092:9092"
      - "9999:9999"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka1:19092,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_JMX_PORT: 9999
      KAFKA_JMX_HOSTNAME: ${DOCKER_HOST_IP:-127.0.0.1}
    volumes:
      - ./zk-single-kafka-single/kafka1/data:/var/lib/kafka/data
    depends_on:
      - zoo1

Docker log

docker compose up --build
[+] Running 3/2
 ⠿ Network logflow_default     Created                                                                                                                                                                                                   3.8s
 ⠿ Container logflow_zoo1_1    Created                                                                                                                                                                                                   0.0s
 ⠿ Container logflow_kafka1_1  Created                                                                                                                                                                                                   0.0s
Attaching to kafka1_1, zoo1_1
kzoo1_1    | ZooKeeper JMX enabled by default
zoo1_1    | Using config: /conf/zoo.cfg
^R
kafka1_1  | ===> User
kafka1_1  | uid=0(root) gid=0(root) groups=0(root)
kafka1_1  | ===> Configuring ...
kafka1_1  | ===> Running preflight checks ... 
kafka1_1  | ===> Check if /var/lib/kafka/data is writable ...
kafka1_1  | ===> Check if Zookeeper is healthy ...
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=kafka1
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_212
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Azul Systems, Inc.
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/zulu-8-amd64/jre
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/etc/confluent/docker/docker-utils.jar
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=5.10.25-linuxkit
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=root
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/root
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=28MB
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=443MB
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=31MB
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zoo1:2181 sessionTimeout=40000 watcher=io.confluent.admin.utils.ZookeeperConnectionWatcher@cc34f4d
kafka1_1  | [main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
kafka1_1  | [main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
kafka1_1  | [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket error occurred: zoo1/172.19.0.2:2181: Connection refused
kafka1_1  | [main] ERROR io.confluent.admin.utils.ClusterStatus - Timed out waiting for connection to Zookeeper server [zoo1:2181].
kafka1_1  | [main-SendThread(zoo1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zoo1/172.19.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
kafka1_1  | [main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x0 closed
kafka1_1  | [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x0

using Docker Engine 20.10.6 Docker Client 20.10.6 on Apple Macbook Air M1.

KangSungHyun · Answer 1 · Mon Jun 28 2021 23:08:17 GMT+0800 (China Standard Time)

I'm having the same problem.

Alex_liuliuliu · Answer 2 · Tue Jul 27 2021 11:21:55 GMT+0800 (China Standard Time)

I have the same problem，and my pc is mbp m1 ,the docker version is v20.10.7.

nicolumibao24 · Answer 3 · Wed Sep 01 2021 13:02:21 GMT+0800 (China Standard Time)

Same problem using MacBook M1

Stephane Maarek · Answer 4 · Wed Oct 13 2021 22:37:30 GMT+0800 (China Standard Time)

this seems linked to confluentinc/common-docker#117

Stephane Maarek · Answer 5 · Wed Oct 13 2021 23:00:02 GMT+0800 (China Standard Time)

Can you all please test the docker-compose file on master and let me know if you're still having issues?

@nicolumibao24 @superliuliuliu @ksh901016 @thearyanahmed

Nikhil Pareek · Answer 6 · Fri Jul 22 2022 03:56:50 GMT+0800 (China Standard Time)

still facing the issue