Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions

[Camila Macedo]

Problem Description

We've encountered an issue where our environment cannot pin a specific version of controller-runtime, leading to compatibility issues with earlier versions of kubebuilder when setting up envtest. This incompatibility prevents our projects from maintaining a consistent development environment, particularly affecting our CI/CD pipelines.

Error Output

While attempting to set up envtest (go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest), we're met with the following error message due to the version incompatibility besides the LATEST v0.17.2 does not support go 1.22 and old projects not be using it yet as well.

go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20240327193027-21368602d84b
go: downloading sigs.k8s.io/controller-runtime v0.17.2
go: sigs.k8s.io/controller-runtime/tools/setup-envtest@latest: sigs.k8s.io/controller-runtime/tools/setup-envtest@v0.0.0-20240327193027-21368602d84b requires go >= 1.22.0 (running go 1.21.8; GOTOOLCHAIN=local)

This error highlights the requirement for Go version >= 1.22.0, while we are currently using Go 1.21.8.

Ideal Solution (controller-runtime provide a envtest setup solution that allows us pin the tag releases)

Ideally, we need a way to setup envtest that allows version pinning per tag release. This would enable our community to manage dependencies more effectively and ensure compatibility across different versions of kubebuilder and controller-runtime. Note that in many scenarios we might need to do this setup in shell scripts.

Alternative Solution (how we could fix it and which seems the best available approach so far)

Use the release branch, example : go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-17

[Camila Macedo]

@joelanford @varshaprasad96 @vincepri @sbueringer

I'm reaching out to gather insights or possible solutions, as we're currently facing issues with the existing setup. Our immediate goal is to enhance the experience for Kubebuilder users and resolve CI failures by utilizing the release branch of controller-runtime.

Here's the snippet from our current approach:

ENVTEST ?= $(LOCALBIN)/setup-envtest-$(ENVTEST_VERSION)
ENVTEST_VERSION ?= release-0.17
...

.PHONY: envtest
envtest: $(ENVTEST) ## Download setup-envtest locally if necessary.
$(ENVTEST): $(LOCALBIN)
	$(call go-install-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest,$(ENVTEST_VERSION))

While this method serves as a temporary fix, a more ideal solution would involve pinning specific tag versions to ensure stability and predictability. The reliance on the release branch introduces a risk, especially if a release branch is not created as part of the controller-runtime's release process. This potential inconsistency could adversely affect Kubebuilder and the broader community.

[Lukas Koszegy]

proper versioning of setup-env is solving here #2646
but maintainers don't react.

[Stefan Büringer]

Hey folks,
we are currently experimenting a bit on setup-envtest because the envtest binaries are today hosted on the Google-owned kubebuilder repository.

Going forward we will start hosting the envtest binaries on controller-tools releases. We currently also plan to move setup-envtest to controller-tools as it fits better there. As part of that we will build & publish the setup-envtest binary on controller-tools releases in the future. This should also give proper versioning.
#2646 (comment)

Let me know if this covers the issue.