remote-echidna

Run echidna on AWS

Motivation

Echidna is a program designed for fuzzing/property-based testing of Ethereum smart contracts. Because of its high demand on computational resources, it is not very practical to run long runs on your local development environment. The purpose of remote-echidna is to allow developers to test their contracts on AWS while they can continue working on something else.

Description

This project uses Terraform to create a virtual machine, together with other required AWS resources, for each echidna job. The instance starts up, install all required components, runs the fuzz tests, and automatically shuts itself down.

Setup

Manual steps:

Create a S3 bucket with private access to store and load echidna's output between runs
Create a private key used to connect to the remote instance if needed

Automated steps:

This project creates the following infrastructure on AWS:

Security Group with SSH access from anywhere, to allow for an easier debugging
IAM Policy with access to the S3 bucket
IAM User with the created IAM Policy
EC2 Instance that runs echidna on the desired git project and uses the IAM User credentials to upload results to S3

Usage with GitHub Actions

Create a GitHub Action on your CI that reuses remote-echidna's workflow. See a sample project here

name: Pull Request

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  remote-echidna:
    name: Run remote-echidna
    uses: pods-finance/remote-echidna/.github/workflows/remote-echidna.yml@v2
    with:
      project: "https://github.com/${{github.repository}}.git"
      project_git_url: "https://github.com/${{github.repository}}.git"
      project_git_checkout: ${{ github.head_ref || github.ref_name }}
      run_tests_cmd: "yarn && echidna-test test/invariants/STETHVaultInvariants.sol --contract STETHVaultInvariants --config test/invariants/config-assertion.yaml --test-limit 100000"
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: ${{ secrets.AWS_REGION }}
      REMOTE_ECHIDNA_S3_BUCKET: ${{ secrets.REMOTE_ECHIDNA_S3_BUCKET }}
      REMOTE_ECHIDNA_EC2_INSTANCE_KEY_NAME: ${{ secrets.REMOTE_ECHIDNA_EC2_INSTANCE_KEY_NAME }}

You may also choose to include the following action to have the GitHub bot update your Pull Request with a comment monitoring the job status:

name: Update PR comments

on:
  workflow_dispatch:
  schedule:
    - cron: '* * * * *'

jobs:
  update-pr-comments:
    name: Update PR comments
    permissions:
      pull-requests: write
    uses: pods-finance/remote-echidna/.github/workflows/update-pr-comments.yml@v2
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: ${{ secrets.AWS_REGION }}
      REMOTE_ECHIDNA_S3_BUCKET: ${{ secrets.REMOTE_ECHIDNA_S3_BUCKET }}

Configure the required input parameters below

Inputs

Parameter	Description	Example	Required
`project`	Your project name	`yield-contracts`	Yes
`project_git_url`	Project Git URL	`https://github.com/${{github.repository}}.git`	Yes
`project_git_checkout`	Project Git checkout (branch or commit hash)	`${{ github.head_ref \|\| github.ref_name }}`	Yes
`run_tests_cmd`	Command to run echidna tests	`yarn && echidna-test test/invariants/STETHVaultInvariants.sol --contract STETHVaultInvariants --config test/invariants/config-assertion.yaml --test-limit 100000`	Yes
`REMOTE_ECHIDNA_S3_BUCKET`	S3 Bucket name to store and load echidna's output between runs	`remote-echidna`	Yes
`REMOTE_ECHIDNA_EC2_INSTANCE_KEY_NAME`	EC2 instance key name. Needs to be manually created first on the AWS console	`key`	Yes

Output

The job status is stored on a S3 bucket, alongside echidna's output:

Provisioning
Provisioned
Started
Running
Finished
Deprovisioned

Some of these states are managed from within the instance (2 -> 3 -> 4), while others are managed through Github actions (0 -> 1 -> 2, 4 -> 5).

Echidna's output will be automatically included on your pull request with as a GitHub bot comment.

pods-finance / remote-echidna