stephan-nordnes-eriksen / artillery-plugin-expect

✅ Run functional/integration tests with BDD-style expectations & assertions for HTTP scenarios

artillery-plugin-expect

Functional API testing with Artillery

• Add expectations and assertions to your HTTP scenarios
• Use the same artillery command to run functional / acceptance tests on your APIs
• See details of any failed assertions (request headers & body, response etc)
• Use your Artillery scenarios for both functional testing and load testing

🐞 Please report issues over at https://github.com/shoreditch-ops/artillery/issues

Usage

Install the plugin

npm install -g artillery-plugin-expect

Important: this plugin requires Artillery v1.6.0-26 or higher.

Enable the plugin in the config section

config:
  target: "http://example.com"
  plugins:
    expect: {}

Add expectations to your requests

scenarios:
  - name: Get pets
    flow:
      - get:
          url: "/pets"
          capture:
            - json: "$.name"
              as: name
          expect:
            - statusCode: 200
            - contentType: json
            - hasProperty: results
            - equals:
              - "Tiki"
              - "{{ name }}"
            - hasHeader: "<Any header key>"
            - headerEquals:
              - "<Any header key>"
              - "<Expected header value>"

Run your test & see results

Run your script that uses expectations with:

artillery run --quiet my-script.yaml

The --quiet option is to stop Artillery from printing its default reports to the console.

Failed expectations provide request and response details:

Re-using scenarios as load tests or functional tests

This plugin allows for the same scenario to be re-used for either load testing or functional testing of an API. (The only real difference between the two, of course, is how many virtual users you run -- only one for functional tests, and $BIG_NUMBER for a load test.)

In practical terms, what you probably want to do is use the environments functionality in Artillery to create a separate "environment" with configuration for functional testing. Something like:

config:
  target: "https://my.api.internal"
  environments:
    #
    # This is our load testing profile, where we create a lot of virtual users.
    # Note that we don't load the plugin here, so that we don't see the output
    # from the plugin.
    #
    load:
      phases:
        - duration: 600
          arrivalRate: 10
    #
    # This is our functional testing profile, with a single virtual user, and
    # the plugin enabled.
    #
    functional:
      phases:
        - duration: 1
          arrivalCount: 1
      plugins:
        expect: {}
scenarios:
  # Your scenario definitions go here.

Expectations

statusCode

Check that the response status code equals the code given.

expect:
  - statusCode: 201

contentType

Check the value of Content-Type header.

hasProperty and notHasProperty

When the response is JSON, check that the response object has a property (or does not in case of notHasProperty) Same behavior as lodash#has.

expect:
  - hasProperty: 'data[0].id'

equals

Check that two or more values are the same. NOTE only primitive values (e.g. booleans, strings and numbers) are currently supported.

- get:
    url: "/pets/f037ed9a"
    capture:
      - json: "$.species"
        as: species
    expect:
      - equals:
          - "{{ species }}"
          - "dog"

hasHeader

Check that the response contains a header.

- get:
    url: "/pets/f037ed9a"
    expect:
      - hasHeader: "object-version"

headerEquals

Check that the response contains a header and its value matches some string.

- get:
    url: "/pets/f037ed9a"
    expect:
      - headerEquals:
        - "object-version"
        - "v3"

matchesRegexp

Check that response body matches a regular expression. The regular expression provided must be a string which is a valid argument to the RegExp constructor.

- get:
    url: "/pets/f037ed9a"
    expect:
      - matchesRegexp: .+ # body is not empty

Feedback

This plugin is still in the early stages and we would love to hear your comments, ideas and suggestions for improvements. Please file an issue over at https://github.com/shoreditch-ops/artillery/issues.

License

MPL 2.0