EventHub Connection Error during authentication
RafaMunoz opened this issue · comments
Description
When start the Kafka Connect and it try to conected Event Hub I have the next error:
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (eh-rafa.servicebus.windows.net/13.69.111.138:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.
I have check that the port 9093 and 443 is open and the credentials are okey because from python script I connect and send data without problems.
How to reproduce
Follow the steps from this guide https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-connect-tutorial on Ubuntu Server 20.04.
I have also tried with docker but I have the same error.
Has it worked previously?
Yes, but it worked a few months ago on other Event Hub
Checklist
IMPORTANT: We will close issues where the checklist has not been completed or where adequate information has not been provided.
Please provide the relevant information for the following items:
- SDK (include version info):
openjdk 11.0.11 2021-04-20 LTS - Sample you're having trouble with:
Connection from Kafka Connect to Event Hub - [] If using Apache Kafka Java clients or a framework that uses Apache Kafka Java clients, version:
<REPLACE with e.g., 1.1.0> - Kafka client configuration:
The same of this web https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-connect-tutorial replacing bootstrap.servers and the password whit de connection string - Namespace and EventHub/topic name
eh-rafa.servicebus.windows.net:9093 and the EntityPath=my-eh - Consumer or producer failure
<REPLACE with e.g., Consumer failure> - Timestamps in UTC
always - [ x] group.id or client.id
connect-cluster-group - Logs provided (with debug-level logging enabled if possible, e.g. log4j.rootLogger=DEBUG) or exception call stack
- Standalone repro
https://github.com/RWaltersMA/azure-event-hubs-mongo or this with docker https://github.com/RWaltersMA/azure-event-hubs-mongo - Operating system:
Ubuntu Server 20.04 - Critical issue
If this is a question on basic functionality, please verify the following:
- Port 9093 should not be blocked by firewall ("broker cannot be found" errors)
- Pinging FQDN should return cluster DNS resolution (e.g.
$ ping namespace.servicebus.windows.netreturns ~ns-eh2-prod-am3-516.cloudapp.net [13.69.64.0]) - Namespace should be either Standard or Dedicated tier, not Basic (TopicAuthorization errors)
Make sure you are using the right connection string. See if the connection string works with a Kafka producer or consumer also.
The connection strings works fine, this I checked with python script and the software ServiceBusExplorer.
Can you share your Kafka properties after removing secrets?
The configuration of file connect-distributed.properties is:
bootstrap.servers=XXXXXXXXXXX.servicebus.windows.net:9093
group.id=connect-cluster-group
config.storage.topic=connect-cluster-configs
offset.storage.topic=connect-cluster-offsets
status.storage.topic=connect-cluster-status
config.storage.replication.factor=1
offset.storage.replication.factor=1
status.storage.replication.factor=1
rest.advertised.host.name=connect
offset.flush.interval.ms=10000
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://XXXXXXXXXXX.servicebus.windows.net/;SharedAccessKeyName=XXXXXXXXXXX;SharedAccessKey=XXXXXXXXXXXXXXXX";
producer.security.protocol=SASL_SSL
producer.sasl.mechanism=PLAIN
producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://XXXXXXXXXXX.servicebus.windows.net/;SharedAccessKeyName=XXXXXXXXXXX;SharedAccessKey=XXXXXXXXXXXXXXXX";
consumer.security.protocol=SASL_SSL
consumer.sasl.mechanism=PLAIN
consumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://XXXXXXXXXXX.servicebus.windows.net/;SharedAccessKeyName=XXXXXXXXXXX;SharedAccessKey=XXXXXXXXXXXXXXXX";
plugin.path=/opt/kafka/libs
Properties look OK to me. I will try to create the same configuration as you did and see if I can reproduce the failure.
Finally I found the problem. I change the level of the Event Hub from Basic to Standard and it work.
Thanks for your help!!
I am glad you figured it out. I am closing the issue for now.