IBM / ignition

Framework for VIM and Lifecycle driver applications

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Expose Kafka Producer and Consumer Configuration

sglover opened this issue · comments

There are many configuration settings for Kafka Producers and Consumers (see https://kafka-python.readthedocs.io/en/master/apidoc/KafkaProducer.html). Currently, Ignition does not allow any of these to be configured on a per-deployment basis. Ignition should allow these to be set through its normal configuration mechanism, ideally as a set of properties in the configuration that can be used verbatim to configure the Producers and Consumers.

Kafka Producer, Consumer and AdminClient configuration can now be configured as follows in the configuration file:

messaging:
    # Kafka connection url
    connection_address: foundation-kafka:9092
    config:
        # timeout waiting for initial version check on Kafka producer/consumer initialisation
        # 5000ms is usually sufficient, increase if problems with NoBrokersAvailable occur
        api_version_auto_timeout_ms: 5000

Any properties under messaging.config will be used to configure KafkaProducer, KafkaConsumer and KafkaAdminClient instances.