Komodo

Komodo is a software distribution system.

The purpose of Komodo is to automatically, reproducibly, and testably create a software distribution. Automatic deploy of new releases, as well as nightly deploy and the option of automatically moving the testing stage, is supported.

Install

The tool is not hosted on PyPI but can be installed with pip directly from GitHub:

pip install git+https://github.com/equinor/komodo.git

Basic usage

We have a 'repository' of packages described in a YAML file. Each package contains a list of one or more versions. Each version contains:

Build information.
Maintainer.
Source, e.g. PyPI or a GitHub repository, if required.
Dependency list, if any.
Other metadata, depending on the type of package.

For example, we may have a repository.yml like this:

python:
  3-builtin:
    make: sh
    makefile: build__python-virtualenv.sh
    maintainer: foo@example.com
    makeopts: --virtualenv-interpreter python3

treelib:
  1.6.1:
    source: pypi
    make: pip
    maintainer: bar@example.com
    depends:
      - python

Note that build__python-virtualenv.sh is a script that comes with komodo (in komodo/data); it will use the system Python in the environment it builds.

Now a 'release', e.g. stable, is defined in another YAML file, e.g. stable.yml, containing some or all of the packages in the repository file:

python: 3-builtin
treelib: 1.6.1

A full software distribution can then be built and deployed to a specified path, e.g. ./builds/stable-0.0.1, with the following command:

kmd stable.yml repository.yml --prefix builds --release stable-0.0.1

To use this environment, type source builds/stable-0.0.1/enable.

Other komodo commands

As well as the kmd command, this package installs several other commands, each with its own options:

komodo-check-pypi — Checks if pypi packages are up to date
komodo-insert-proposals — Copy proposals into release and create PR
komodo-post-messages — Post messages to a release
komodo-check-symlinks — Verify symlinks for komodo versions are according to a given config
komodo-lint — Lint komodo setup
komodo-reverse-deps — Extracts dependencies from a given set of packages
komodo-clean-repository — Clean up unused versions in the repository file based on a set of releases
komodo-lint-maturity — Lint the maturity of packages
komodo-snyk-test — Test a release for security and license issues
komodo-create-symlinks — Create symlinks for komodo versions
komodo-lint-package-status — Lint the package status file
komodo-suggest-symlinks — Returns a pull request if the symlink configuration could be updated
komodo-extract-dep-graph — Extracts dependencies from a given set of packages
komodo-non-deployed — Outputs the name of undeployed matrices given an installation root and a release folder
komodo-transpiler — Build release files
komodo-show-version — Return the version of a specified package in the active release

Auto-formatting configuration files

You can auto-format repository and/or releases by running something like

komodo-clean-repository prettier --files repository.yml releases/*

If you are in e.g. CI and only want to check style compliance, add --check.

Finding reverse dependecies

You can show reverse dependencies of a package by running the tool komodo-reverse-deps:

komodo-reverse-deps releases/matrices/2022.09.02.yml repository.yml --pkg websockets

If --pkg is not specified, the program will prompt for it.

The --dot option outputs the reverse dependency graph in .dot format. Alternatively, if GraphViz and ImageMagick are available, the --display_dot option will try to render the graph directly.

Run tests

git clone https://github.com/equinor/komodo.git
cd komodo
pip install -r dev-requirements.txt
pytest tests

kwinkunks / komodo