Commander

Define language independent tests for your command line scripts and programs in simple yaml files.

It runs on windows, osx and linux
It is a self-contained binary - no need to install a heavy lib or language
It is easy and fast to write

For more information take a look at the manual, the examples or the integration tests.

Installation

Any system with Go installed

Probably the easiest way to install commander is by using go get to download and install it in one simple command:

go get github.com/SimonBaeumer/commander/cmd/commander

This works on any OS, as long as go is installed. If go is not installed on your system, follow one of the methods below.

Linux & osx

Visit the release page to get the binary for you system.

curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.0/commander-linux-amd64 -o commander
chmod +x commander

Windows

Download the current release
Add the path to your path environment variable
Test it: commander --version

Quick start

Commander will always search for a default commander.yaml in the current working directory and execute all defined tests in it.

# You can even let commander add tests for you!
$ ./commander test examples/commander.yaml
tests:
  echo hello:
    exit-code: 0
    stdout: hello

written to /tmp/commander.yaml

# ... and execute!
$ ./commander test
Starting test file commander.yaml...

✓ echo hello

Duration: 0.002s
Count: 1, Failed: 0

Complete YAML file

Here you can see an example with all features.

config: # Config for all executed tests
    dir: /tmp #Set working directory
    env: # Environment variables
        KEY: global
    timeout: 50s # Define a timeout for a command under test
    retries: 2 # Define retries for each test
    
tests:
    echo hello: # Define command as title
        stdout: hello # Default is to check if it contains the given characters
        exit-code: 0 # Assert exit-code
        
    it should fail:
        command: invalid
        stderr:
            contains: 
                - invalid # Assert only contain work
            not-contains:
                - not in there # Validate that a string does not occur in stdout
            exactly: "/bin/sh: 1: invalid: not found"
            line-count: 1 # Assert amount of lines
            lines: # Assert specific lines
                1: "/bin/sh: 1: invalid: not found"
        exit-code: 127
        
    it has configs:
        command: echo hello
        stdout:
            contains: 
                - hello #See test "it should fail"
            exactly: hello
            line-count: 1
        config:
            dir: /home/user # Overwrite working dir
            env:
                KEY: local # Overwrite env variable
                ANOTHER: yeah # Add another env variable
            timeout: 1s # Overwrite timeout
            retries: 5
        exit-code: 0

Executing

# Execute file commander.yaml in current directory
$ ./commander test 

# Execute a specific suite
$ ./commander test /tmp/test.yaml

# Execute a single test
$ ./commander test /tmp/test.yaml "my test"

Adding tests

You can use the add argument if you want to commander to create your tests.

# Add a test to the default commander.yaml
$ ./commander add echo hello
written to /tmp/commander.yaml

# Write to a given file
$ ./commander add --file=test.yaml echo hello
written to test.yaml

# Write to stdout and file
$ ./commander add --stdout echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello

written to /tmp/commander.yaml

# Only to stdout
$ ./commander add --stdout --no-file echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello

Usage

NAME:
   Commander - CLI app testing

USAGE:
   commander [global options] command [command options] [arguments...]

COMMANDS:
     test     Execute the test suite
     add      Automatically add a test to your test suite
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

Development

# Initialise dev environment
$ make init

# Build the project binary
$ make build

# Unit tests
$ make test

# Coverage
$ make test-coverage

# Integration tests
$ make integration

# Add depdencies to vendor
$ make deps

Misc

Heavily inspired by goss.

Similar projects:

svenfinke / commander