Datadog CI

Execute commands with Datadog from within your Continuous Integration/Continuous Deployment scripts to perform end-to-end tests of your application before applying your changes or deploying. datadog-ci allows you to run Continuous Testing tests and wait for the results.

How to install the CLI

The package is under @datadog/datadog-ci and can be installed through NPM or Yarn:

# NPM
npm install --save-dev @datadog/datadog-ci

# Yarn
yarn add --dev @datadog/datadog-ci

If you need datadog-ci as a CLI tool instead of a package, you can run it with npx or install it globally:

# npx
npx @datadog/datadog-ci [command]

# NPM install globally
npm install -g @datadog/datadog-ci

# Yarn v1 add globally
yarn global add @datadog/datadog-ci

For more ways to install the CLI, see this section.

Usage

Usage: datadog-ci <command> <subcommand> [options]

The following values are available for each <command>. See the corresponding documentation for more details:

dsyms: iOS dSYM Files
flutter-symbols: Flutter Symbols
git-metadata: Git metadata
junit: JUnit XML
lambda: Lambda
metric: Metric
react-native: React Native sourcemaps
sourcemaps: Browser sourcemaps
stepfunctions: Step Functions
synthetics: Continuous Testing
tag: Tag
trace: Trace

Contributing

Pull requests for bug fixes are welcome, but before submitting new features or changes to current functionality, open an issue and discuss your ideas or propose the changes you wish to make. After a resolution is reached, a PR can be submitted for review.

Running command in development environment

When developing the tool, it is possible to run commands using yarn launch. It relies on ts-node, so does not require building the project for every new change.

yarn launch synthetics run-tests --config dev/global.config.json

Framework and libraries used

clipanion: CLI library to handle the different commands.
eslint: Linting (.eslintrc.js).
jest: Tests are written in Jest.
volta: NodeJS and yarn versioning.

Repository structure

Commands are stored in the src/commands folder.

The skeleton of a command is composed of a README, an index.ts and a folder for the tests.

src/
└── commands/
    └── fakeCommand/
         ├── __tests__/
         │   └── index.test.ts
         ├── README.md
         └── index.ts

Documentation of the command must be placed in the README.md file, the current README must be updated to link to the new command README.

The index.ts file must export classes extending the Command class of clipanion. The commands of all src/commands/*/index.ts files will then be imported and made available in the datadog-ci tool.

A sample index.ts file for a new command would be:

import {Command} from 'clipanion'

export class HelloWorldCommand extends Command {
  public async execute() {
    this.context.stdout.write('Hello world!')
  }
}

module.exports = [HelloWorldCommand]

Lastly, test files must be created in the __tests__/ folder. jest is used to run the tests and a CI has been set using GitHub Actions to ensure all tests are passing when merging a Pull Request.

The tests can then be launched through the yarn test command, it will find all files with a filename ending in .test.ts in the repo and execute them.

Continuous Integration tests

The CI performs tests to avoid regressions by building the project, running unit tests and running one end-to-end test.

The end-to-end test installs the package in a new project, configures it by using files in the .github/workflows/e2e folder, and runs a synthetics run-tests command in a Datadog org (such as Synthetics E2E Testing Org) to verify the command is able to perform a test.

The Synthetic tests that are run include a browser test (with a test ID of neg-qw9-eut) and an API test (with a test ID of v5u-56k-hgk). Both tests load a page which outputs the headers of the request and verifies the X-Fake-Header header is present. This header is configured as an override in the .github/workflows/e2e/test.synthetics.json file. The API and application keys used by the command are stored in GitHub Secrets named datadog_api_key and datadog_app_key.

The goal of this test is to verify the command is able to run tests and wait for their results as expected as well as handling configuration overrides.

Workflow

# Compile and watch
yarn watch

# Run the tests
yarn test

# Build code
yarn build

# Make bin executable
yarn prepack

Release Process

Instructions

To release a new version of datadog-ci:

Create a new branch for the version upgrade.
Update the package.json version, commit the change vX.X.X and tag it with git tag vX.X.X, based on the version you are upgrading to. You may refer to Semantic Versioning to determine what level to increment.
Push the branch along with the tag to the upstream (GitHub) with git push --tags origin name-of-the-branch, create a Pull Request with the changes introduced in the description details, and get at least one approval. For example, see this sample pull request.
Once you've received at least one approval, merge the Pull Request with the "Create a merge commit" strategy.
Create a GitHub Release from the Tags page with the description of changes introduced.
Once the release has been created, a GitHub Action publishes the package. Make sure the job succeeds.
When the package has been published, go to the Datadog GitLab pipelines, find the pipeline for your tag, and start the build stage to run the Docker image build jobs. Once the jobs pass, the release stage automatically triggers. Make sure all the jobs succeed.
If the release introduced any changes in the synthetics command, you have to upgrade datadog-ci in the following projects:

Pre-Release Process

Instructions

To create a pre-release or releasing in a different channel:

Create a new branch for the channel you want to release to (alpha, beta, and more).
Create a PR for your feature branch with the channel branch as a base.
Pick a version following this format: <version>-<channel>. For example, 0.10.9-alpha, 1-beta, and more.
Update the version field in package.json.
Once you've received at least one approval, merge the Pull Request with the "Create a merge commit" strategy.
Create a GitHub Release:
- Target the channel branch.
- Pick a tag based on your version <version>-<channel>.
- Check the This is a pre-release checkbox.
Publish the release and an action publishes it on npm.

More ways to install the CLI

Standalone binary (beta)

If installing NodeJS in the CI is an issue, standalone binaries are provided with releases. linux-x64, darwin-x64 (macOS), and win-x64 (Windows) are supported. These standalone binaries are in beta and their stability is not guaranteed.

To install:

Linux

curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

MacOS

curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci

Windows

Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64.exe" -OutFile "datadog-ci.exe"

Then, you can run datadog-ci commands normally:

datadog-ci version

Container image

To run datadog-ci from a container, you can use the datadog/ci image available in Dockerhub as well as the public Amazon ECR and Google GC registries.

docker pull datadog/ci

This example demonstrates how to run a command using the container and passing in the API and APP keys:

export DD_API_KEY=$(cat /secret/dd_api_key)
export DD_APP_KEY=$(cat /secret/dd_app_key)
docker run --rm -it -v $(pwd):/w -e DD_API_KEY -e DD_APP_KEY datadog/ci synthetics run-tests -p pub-lic-id1

Building your own container image

You can build an image using the provided Dockerfile:

cd container
docker build --tag datadog-ci .

Optionally, you can use the VERSION build argument to build an image for a specific version:

docker build --build-arg "VERSION=v1.14" --t datadog-ci .

License

Apache License, v2.0

adrianfalleiro / datadog-ci