Aiven-Open / tiered-storage-for-apache-kafka

RemoteStorageManager for Apache Kafka® Tiered Storage

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Build Docker image before E2E tests

ivanyu opened this issue · comments

Currently, we build the image as a part of E2E test stage. Since E2E run in a type matrix, we build the image multiple times.

There are some mismatches that make this challenging: we build the library against different JDK versions, but package it in the same docker base image -- so effectively it runs with the same JDK version.
Then the JDK version matching between build and e2e doesn't matter much as the e2e is in a standalone classpath -- so there's no dependency between JDK versions to care here.

Another mismatch is that we need to build the image once, and pass it to a 2nd set of parallel jobs. To pass docker image between jobs we are constraint to upload/download: https://docs.docker.com/build/ci/github-actions/share-image-jobs/
which means ~1min operation with a ~1.2GB image size (mostly Confluent base image)

Leaving this for context if someone would like to tackle it.