wangkirin/oct

OCI Test

The ocitest aims to test if a runtime container is compliant with opencontainers/specs,
It is a light weight testing framework, using ocitools and 3rd-party tools, managing configurable high coverage bundles as cases, supporting testing different runtimes.

Summary for the impatient

Key note
Be sure to download specs source code and install runc first

$ go get github.com/huawei-openlab/oct                 #get source code       
$ cd $GOPATH/src/github.com/huawei-openlab/oct         #change dir to workspace 
$ make                                                 #build      
$ ./ocitest                                            #run

OCI Test Quickstart

Usage

$ ./ocitest --help
NAME:
   oci-testing - Utilities for OCI Testing,

    It is a light weight testing framework,
    using ocitools and 3rd-party tools, 
    managing configurable high coverage bundles as cases, 
    supporting testing different runtimes.

USAGE:
   ./ocitest [global options] command [command options] [arguments...]
   
VERSION:
   0.0.1
   
COMMANDS:
   help, h	Shows a list of commands or help for one command
   
GLOBAL OPTIONS:
   --runtime, -r "runc"    runtime to be tested, -r=runc or -r=rkt or -r=docker     
   --output, -o "all"   format and content to be ouputed, -o=all: ouput sucessful details and statics, -o=err-only: ouput failure details and statics           
   --debug, -d     switch of debug mode, defaults to false, with '--debug' to enable debug mode     
   --help, -h     show help     
   --generate-bash-completion           
   --version, -v    print the version

Supportted runtime

|Name|Status|Testing Flow| |------|----|------| ----- | | runc | Supported| Test bundles & runtime Validate | | rkt | Supported | Test bundles converted by oci2aci & runtime Validate | | docker | Not currently being worked|Test bundles converted by oci2docker & runtime Validate |

Using Tools

Tools used by ocitest as plugins, Key Notes

ocitools are foked from github.com/mrunalp, adding some adaptor changes for oct.

See plugins/Makefile

About Test Cases

Cases are listed in cases.conf, as the fomate of bunldes, It is going to be rich, in the fomate of below:

process= --args=./runtimetest --args=vp --rootfs=rootfs --terminal=false;--args=./runtimetest --args=vp --rootfs=rootfs --terminal=false     
# result to generate two cases in [bundle](./bundle), should be bundle/process0 and bundle/process1,        
# and '--args=./runtimetest --args=vp --rootfs=rootfs --terminal=false' is params for ocitools generate

What is good for runtimeValidator

Light weight testing freamwork
High coverage test cases, configurable, easy to add cases
Tools is used as plugins ,feel free to use any 3rd-paty tools
Uses goroutine, each go routine runs a case bundle to validate
Note
The ocitools are developed in github.com/mrunalp.

Next to Do

Rich cases:

Encrease the functionality of ocitools in cmd/runtimetest
Rich cases in cases.conf
Support other containers

Reference

OCI specs on https://github.com/opencontainers/specs

OCI runc on https://github.com/opencontainers/runc

wangkirin / oct