saschagrunert / zeitgeist

Zeitgeist: the language-agnostic dependency checker

Zeitgeist

(/ˈzaɪtɡaɪst/) is a language-agnostic dependency checker that keeps track of external dependencies across your project and ensure they're up-to-date.

• Rationale
• What is Zeitgeist
• Installation
• Supported upstreams
• Supported version schemes
• When is Zeitgeist not suggested
• Naming
• Releasing
• Credit
• To do
• Community, discussion, contribution, and support
  • Code of conduct

Rationale

More and more projects nowadays have external dependencies, and the best way to ensure stability and reproducibility is to pin these dependencies to a specific version.

However, this leads to a new problem: the world changes around us, and new versions of these dependencies are released all the time.

For a simple project with a couple of dependencies, a team can usually keep up to speed by following mailing lists or Slack channels, but for larger projects this becomes a daunting task.

This problem is pretty much solved by package managers in specific programming languages (see When is Zeitgeist not suggested below), but it remains a big issue when:

• Your project relies on packages outside your programming language of choice
• You declare infrastructure-as-code, where the "build step" is usually bespoke and dependencies are managed manually
• Dependencies do not belong in a classical "package manager" (e.g. AMI images)

What is Zeitgeist

Zeitgeist is a tool that takes a configuration file with a list of dependencies, and ensures that:

• These dependencies versions are consistent within your project
• These dependencies are up-to-date

A Zeitgeist configuration file (usually dependencies.yaml) is a list of dependencies, referenced in files, which may or may not have an upstream:

dependencies:
- name: terraform
  version: 0.12.3
  upstream:
    flavour: github
    url: hashicorp/terraform
  refPaths:
  - path: helper-image/Dockerfile
    match: TERRAFORM_VERSION
  - path: .github/actions/run.yaml
    match: terraform
- name: aws-eks-ami
  version: ami-09bbefc07310f7914
  scheme: random
  upstream:
    flavour: ami
    owner: amazon
    name: "amazon-eks-node-1.21-*"
  refPaths:
  - path: clusters.yaml
    match: workers_ami

Use zeitgeist validate to verify that the dependency version is correct in all files referenced in refPaths, and whether any newer version is available upstream:

You can also use zeitgeist upgrade to go ahead and upgrade your dependencies to the latest versions detected by Zeitgeist.

Installation

You will need to build Zeitgeist from source (for now at least!).

Clone the repository and run go build will give you the zeitgeist binary:

git clone https://github.com/kubernetes-sigs/zeitgeist.git
cd zeitgeist/
go build

Supported upstreams

Github

The Github upstream looks at releases from a Github repository.

Example:

dependencies:
- name: terraform
  version: 0.15.3
  upstream:
    flavour: github
    url: hashicorp/terraform
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: terraform_version

For API access, you will need to set the following env var:

export GITHUB_TOKEN=<YOUR_GITHUB_TOKEN>

Helm

The Helm upstream looks at chart versions from a Helm repository.

Example:

dependencies:
- name: linkerd
  version: 2.10.0
  upstream:
    flavour: helm
    repo: https://helm.linkerd.io/stable
    chart: linkerd2
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: linkerd-

Gitlab

The Gitlab upstream looks at releases from a Gitlab repository.

Example:

dependencies:
- name: gitlab-agent
  version: v14.0.1
  upstream:
    flavour: gitlab
    url: gitlab-org/cluster-integration/gitlab-agent
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: GL_VERSION

The Gitlab API requires authentication, so you will need to set an Access Token.

When using the public GitLab instance at https://gitlab.com/ :

export GITLAB_TOKEN=<YOUR_GITLAB_TOKEN>

When using a self-hosted GitLab instance, ie. https://my-gitlab.company.com/ :

export GITLAB_PRIVATE_TOKEN=<YOUR_GITLAB_PRIVATE_TOKEN>

You can use in the dependencies.yaml both public and private GitLab instances. The only limitation today is that you can only use one private GitLab at the moment.

AMI

The AMI upstream looks at Amazon Machine Images from AWS.

Example:

dependencies:
- name: aws-eks-ami
  version: ami-09bbefc07310f7914
  scheme: random
  upstream:
    flavour: ami
    owner: amazon
    name: "amazon-eks-node-1.21-*"
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: zeitgeist:aws-eks-ami

It uses the standard go AWS SDK authentication methods for authentication and authorization, so it can be used for both public & private AMIs.

Container

The container upstream talks to OCI container registries, such as Docker registries.

Example:

dependencies:
- name: docker-in-docker
  version: 19.03.15
  upstream:
    flavour: container
    registry: hub.docker.io/docker
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: docker-dind

If you're connecting to a private registry, you will need to set the following env vars:

export REGISTRY_USERNAME=<YOUR_REGISTRY_USERNAME>
export REGISTRY_USER_PASSWORD=<YOUR_REGISTRY_TOKEN_PASSWORD>

EKS

The EKS checks for updates to Elastic Kubernetes Service, Amazon's managed Kubernetes offering.

Example:

dependencies:
- name: eks
  version: 1.13.0
  upstream:
    flavour: eks
  refPaths:
  - path: testdata/zeitgeist-example/a-config-file.yaml
    match: eks

Supported version schemes

Zeitgeist supports several version schemes:

• semver: SemVer v2, default
• alpha: alphanumeric ordering. A newer version is considered an update if it's alphanumerically higher, e.g. "release-d" is higher "release-c" but "release-b-update-1" wouldn't be higher than "release-c".
• random: any newer version is considered an update. Useful for UUID or hash-based versioning.

See the full documentation to see configuration options.

When is Zeitgeist not suggested

While Zeitgeist aims to be a great cross-language solution for tracking external dependencies, it won't be as well integrated as native package managers.

If your project is mainly written in one single language with a well-known and supported package manager (e.g. npm, maven, rubygems, pip, cargo...), you definitely should use your package manager rather than Zeitgeist.

Naming

Zeitgeist, a German compound word, can be translated as "spirit of the times" and refers to a schema of fashions or fads which prescribes what is considered to be acceptable or tasteful for an era.

Releasing

Releases are generated with goreleaser.

git tag v0.0.0 # Use the correct version here
git push --tags
export GPG_TTY=$(tty)
goreleaser release --rm-dist

Credit

Zeitgeist is inspired by Kubernetes' script to manage external dependencies and extended to include checking with upstream sources to ensure dependencies are up-to-date.

To do

• Find a good name for the project
• Support helm upstream
• Support eks upstream
• Support ami upstream
• support docker upstream
• Cleanly separate various upstreams to make it easy to add new upstreams
• Implement non-semver support (e.g. for AMI, but also for classic releases)
• Write good docs :)
• Write good tests!
• Externalise the project into its own repo
• Generate releases
• Automate release generation from a tag

Community, discussion, contribution, and support

Learn how to engage with the Kubernetes community on the community page.

You can reach the maintainers of this project at:

• #release-management channel on Kubernetes Slack
• Mailing List

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.