Deprecation notice

This project is now deprecated as since Kafka 3.5.0, Kafka implemented a built-in EnvVar Configuration Provider with KIP-887.

To migrate from Strimzi provider to Kafka's, you need to change the class to org.apache.kafka.common.config.provider.EnvVarConfigProvider.

For example:

config:
  # ...
  config.providers: env
  config.providers.env.class: io.strimzi.kafka.EnvVarConfigProvider
  # ...

becomes

config:
  # ...
  config.providers: env
  config.providers.env.class: org.apache.kafka.common.config.provider.EnvVarConfigProvider
  # ...

If you use Strimzi's EnvVar Configuration Provider in client applications, they also need to be updated to use Kafka's implementation. Kafka's EnvVar Configuration Provider, org.apache.kafka.common.config.provider.EnvVarConfigProvider, is included in the kafka-clients JAR from 3.5.0.

EnvVar Configuration Provider for Apache Kafka®

Apache Kafka® supports pluggable configuration providers which can load configuration data from external sources. The configuration provider in this repo can be used to load data from environment variables. It can be used in all Kafka components and does not depend on the other Strimzi components. So you could, for example, use it with your producer or consumer applications even if you don't use the Strimzi operators to provide your Kafka cluster. One of the example use-cases is to load certificates or JAAS configuration from environment variables mapped from Kubernetes Secrets.

Using it with Strimzi

From Strimzi Kafka Operators release 0.25.0, the EnvVar Configuration Provider is included in all the Kafka deployments. You can use it for example with Kafka Connect and Kafka Connect connectors. The following example shows how to use it to get a database password from an environment variable in the connector configuration:

Deploy Kafka Connect, enable the EnvVar Configuration Provider and map database password to the DB_PASSWORD environment variable:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  # ...
  config:
    # ...
    config.providers: env
    config.providers.env.class: io.strimzi.kafka.EnvVarConfigProvider
  # ...
  externalConfiguration:
    env:
      - name: DB_PASSWORD
        valueFrom:
          secretKeyRef:
            name: db-creds
            key: dbPassword
  # ...

Create the connector:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: my-connector
  labels:
    strimzi.io/cluster: my-connect
spec:
  # ...
  config:
    option: ${env:DB_PASSWORD}
    # ...

Adding the EnvVar Configuration Provider to Apache Kafka clients

You can add EnvVar Configuration Provider as any other Java dependency using Maven or any other build tool. For example:

<dependency>
    <groupId>io.strimzi</groupId>
    <artifactId>kafka-env-var-config-provider</artifactId>
    <version>0.1.0</version>
</dependency>

Adding the EnvVar Configuration Provider to Apache Kafka server components

You can also use the EnvVar Configuration Provider with your own Apache Kafka deployments not managed by Strimzi. To add EnvVar Configuration Provider to your own Apache Kafka server distribution, you can download the ZIP or TAR.GZ files frm the GitHub release page and unpack it into Kafka's libs directory.

Using the configuration provider

First, you need to initialize the configuration provider.

config.providers=env
config.providers.env.class=io.strimzi.kafka.EnvVarConfigProvider

Once you initialize it, you can use it to load data from environment variables. For example:

option=${env:MY_ENV_VAR}

Other Strimzi Configuration Providers

If you run Apache Kafka on Kubernetes, you might also be interested in our Kubernetes Configuration Provider.

strimzi / kafka-env-var-config-provider