CP testcontainers

Testcontainers modules for Confluent Platform components.

Example

The following snippet will start an Apache Kafka and a Schema Registry container on the same network

final var factory = new CPTestContainerFactory();

final KafkaContainer kafka = factory.createKafka();
final SchemaRegistryContainer schemaRegistry = factory.createSchemaRegistry(kafka);
schemaRegistry.start(); //will implicitly start kafka container

The following snippet can then be used to configure a producer to use these:

properties.put("bootstrap.servers",   kafka.getBootstrapServers());
//...
properties.put("schema.registry.url", schemaRegistry.getBaseUrl());

The following snippet starts a single node Kafka Connect cluster with two connectors from Confluent Hub installed:

final var connect = factory.createCustomConnector(
        Set.of(
            "confluentinc/kafka-connect-s3:latest", 
            "confluentinc/kafka-connect-datagen:0.4.0"
        ), kafka);
connect.start();

The following snippet configures a CP-server container and a schema registry container with RBAC enabled:

final var factory = new CPTestContainerFactory(network);
final var ldap = factory.createLdap();

final var confluentServer = factory.createConfluentServer().enableRbac();

final var sr = factory.createSchemaRegistry(confluentServer).enableRbac();
sr.start()

See the intTest source set for examples on how to set up containers.

Supported components

So far the following components are supported:

Confluent Server
Confluent Schema Registry
ksqlDB
Kafka Connect
Confluent Replicator running on Connect
Confluent REST Proxy.

Moreover, we can create KafkaContainer instances which have RBAC and MDS enabled. So far, the following components can be configured to use RBAC:

Confluent Server
Confluent Schema Registry
Kafka Connect (without secret registry)

Packages

Project package are hosted on jitpack for the time being. Add the following to your gradle file:

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

dependencies {
    implementation 'com.github.christophschubert:cp-testcontainers:Tag'
}

See the jitpack page of the project for information on how to use the package with mvn or sbt.

Noteworthy demos

LocalStackIntTest shows how to setup S3 sink connector with S3 installation based on localstack.

Troubleshooting

No non-overlapping IPv4 addresses

Status 404: {"message":"could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"}

Solution: use docker network prune to delete

TristinDavis / cp-testcontainers