0x00dec0de / jenny

Command line Jenkinsfile runner written in groovy. Does not need a Jenkins installation to run the Jenkinsfile.

Jenny

Command line Jenkinsfile runner written in groovy. Does not need a Jenkins installation to run the Jenkinsfile.

This does not require Jenkins to run, and will run everything locally. It’s great to debug and quickly iterate over Jenkinsfile creation, before pushing them for the real Jenkins.

In order to do that, a lot of effort is concentrated on execution paths. It allows skipping sections of Jenkinsfiles, and starting from different parts.

Requirements

1. *NIX

2. Java 8

3. curl or wget ( for some dependencies, that will be fetched only once.)

4. docker - if running docker.image or docker.build commands.

5. xunit-viewer - if you want junit commands to render a nice HTML report.

Usage

1. Just make the jenny script available in the path,

2. go in the folder where the Jenkinsfile resides and run:

jenny

This will try running all the stages, locally. Only a subset of Jenkins commands are supported. PRs welcome.

Configuration

The configuration items are made from a few sources:

1. global config: $HOME/.jenny/config,

2. project config: .jenny/config in the project folder,

3. command line arguments.

Options are generally merged (for example libraries), skip node ids, etc. only with the exception of resumeFrom and skipAfter.

Configurable options

libs: # list of libaries folders to load
  - /path/to/some/folder
  - /path/to/other/folder
libsInfoAllowed: # regex of allowed commands to execute even in --info
  - pluginCommand.*
# The environment can be overwritten, implicitly jenny sets these variables
# to match the variables set by Jenkins. If BUILD_ID will change, it will also
# change the currentBuild.
env:
  BUILD_ID: "1"
  BUILD_NUMBER: "1"
  BRANCH_NAME: "master"
noLogo: true
# What folder to use for temporary files, such as nodes, and workspace folders.
workFolder: /tmp
# Where the outputs of the archiveArtifacts should be stored. Implicitly it
# is in `${workFolder}/jenny/workspace/${name}/workspace/archive`
archiveFolder: target/out
# Where the junit files will be copied, when the `junit` command will be invoked.
# Implicitly it is in `${workFolder}/jenny/workspace/${name}/workspace/junit`
# This will try to use `xunit-viewer` if available to render a HTML out of the
# test reports.
junitFolder: target/junit
params:
  PARAMETER_NAME: "value"
execute:
  resumeFrom: "s1"
  skipAfter: "s7"
  only:
    - "s1.n1"
  skip:
    - "s2"
    - "s5"

To get the IDs of the elements to skip, just call jenny with the -i flag.

For this Jenkinsfile:

stage('Prepare') {
  node {
    sh """
        echo Prepare
    """
  }
}

stage('Test') {
  node {
    sh """
        echo Test part 1
    """
  }

  node {
    sh """
        echo Test part 2
    """
  }
}

stage('Deploy') {
  node {
    sh """
      echo Deploy
    """
  }
}

The outcome of jenny -i will be:

>    _
>   (_) ___ _ __  _ __  _   _
>   | |/ _ \ '_ \| '_ \| | | |
>   | |  __/ | | | | | | |_| |
>  _/ |\___|_| |_|_| |_|\__, |
> |__/                  |___/
> console jenkins runner
>

> workspace: /tmp/jenny/workspace/x/workspace
stage: Prepare [s1]
  node [s1.n1]
    sh: echo Prepare
stage: Test [s2]
  node [s2.n1]
    sh: echo Test part 1
  node [s2.n2]
    sh: echo Test part 2
stage: Deploy [s3]
  node [s3.n1]
    sh: echo Deploy

To see how the skips, etc. will affect the execution, jenny -i will also display that information as well. Running for example a run starting from the Test stage, and not running the deployment, we can:

jenny --resumeFrom s2 --skip s3 -i

>    _
>   (_) ___ _ __  _ __  _   _
>   | |/ _ \ '_ \| '_ \| | | |
>   | |  __/ | | | | | | |_| |
>  _/ |\___|_| |_|_| |_|\__, |
> |__/                  |___/
> console jenkins runner
>

> workspace: /tmp/jenny/workspace/x/workspace
> jenny: Skipped stage s1
stage: Test [s2]
  node [s2.n1]
    sh: echo Test part 1
  node [s2.n2]
    sh: echo Test part 2
> jenny: Skipped stage s3

Running it would also yield what we would expect:

jenny --resumeFrom s2 --skip s3

>    _
>   (_) ___ _ __  _ __  _   _
>   | |/ _ \ '_ \| '_ \| | | |
>   | |  __/ | | | | | | |_| |
>  _/ |\___|_| |_|_| |_|\__, |
> |__/                  |___/
> console jenkins runner
>
> jenny: Skipped stage s1
> ========================================================================
> = Stage: Test
> ========================================================================
> sh: ---------------------------------------

        echo Test part 1

> -------------------------------------------
Test part 1
> sh: ---------------------------------------

        echo Test part 2

> -------------------------------------------
Test part 2
> jenny: Skipped stage s3

Supported Commands

These are the commands where a specific implementation is made for them. If the method is not found, it will be mocked with a NOOP function. If the last parameter of the function is a callable, the callable will be invoked.

ansiColor

Only sets the TERM variable for the section. Does nothing else since the outputs should be redirected anyway to the console. If you’ll run this in your actual Jenkins installation, you will need to install the AnsiColor plugin.

archiveArtifacts

Extract the given artifacts in the folder specified by the archiveFolder in the config file.

booleanParam

Allow defining a boolean parameter in the parameters section.

build

Allow running a nested build triggered from the current build. The job must point to a project folder configured in the jenny config, or a sibling folder in case it’s not starting with ., and is not configured. If it’s starting with a . then either the full relative name is configured in the jenny config, and that one will be used, either the folder path will be resolved relative to the current project folder.

checkout

Checkout the source in the workspace. This will actually just copy the project folder into the current folder.

currentBuild

Current build information. The actual currentResult and result are not currently checked for the stage execution, just being displayed.

deleteDir

Delete the current folder recursively.

dir

Change the current folder for the commands in the execution block.

docker

Allow running certain steps in a docker container. Both docker.build and docker.image are supported.

docker.image has implemented: run, withRun and inside.

file

Specify a file for a withCredentials.

input

Ask for input from the user. If the user starts with the letter n it’s considered cancelled.

junit

Import and run xunit-viewer on the given xml files. This will generate a HTML with the output of the JUnit tests.

If xunit-viewer is not available, then only the xml files will be available in the junitFolder.

node

Specify a node. It will just call the code on the local instance.

parallel

Parallel sections will be run iteratively in a non parallel fashion.

properties

Allow defining properties for the current file.

parameters

Allow defining parameters for the current Jenkinsfile. The parameters can be overwritten at the execution using the --param flag.

pipelineTriggers

Allow registering what will trigger the build of this pipeline. Currently only upstream is supported.

pwd

Get the current folder, or a temporary folder.

stash

Allow stashing artifacts for the current build that can be retrieved later in the build with unstash.

string

Define a string param in a parameters section.

sh

Execute a shell script on the local node. All options such as returnStdout/encoding and returnStatus are supported.

Examples

def lsOutput = sh script: 'ls', returnStdout: true

or just the simple shell execution:

sh '''
    ls -la
    pwd
'''

stage

Define a stage. It will just printout its name, and execute the code inside.

unarchive

Unarchives (i.e. copies on the local node) one of the previously executed archiveArtifacts. For it to work you need to pass a mapping that is required by Jenkins even if it’s marked optional in the official documentation.

The naming in mapping must match what it was on archiveArtifacts

unarchive mapping: [
    "files/": ".",
    "file.txt": "renamed-file.txt"
]

unstash

Unstash one of the previously executed stashes.

upstream

Define an upstream dependency. It will just validate it, and print it out.

withCredentials

Will create the files given into, and delete them when the section is done. The files must exist in the project or home folder into .jenny/credentials/NAME_OF_FILE. They can also be symlinks.

writeFile

Writes the given content to a file in the current folder. This also works inside docker containers.

writeFile file: 'hello-world.txt',
          text: 'Hello world!'

Environment

Currently only the following environment variables are defined:

• BUILD_ID - the current build id (normally 1).

• TERM - when ansiColor is executed.

Mocked Classes

Some classes are mocked in order to allow the Jenkinsfile files that were written using those classes to function. Currently only the hudson.AbortException is mocked, in order to be available in the classpath.

Testing jenny locally

If you want to test jenny, you can just run bin/test_jenny.py on a Python 3.6+. (Might work also on 3.5, but it’s not tested)

Another option is to run jenny to test itself. Note, that you need to have docker for this to run. This build will start bin/test_jenny.py in a docker container. Since some tests will need docker (to spin of containers), we need to pass in the socket into the container itself. For this to work we need also to pass in the group id of the docker installation, as well as the user UID/GID that will run the command.

This command should do:

./jenny --param "JENNY_DOCKER_UID=$(id -u):$(id -g)" -param "JENNY_DOCKER_GID=$(cat /etc/group | grep ^docker: | cut -f3 -d:)"