Instant Grader CLI

A tools for grading your code and test subject which is on the single file.

No need to create a new file for test subject just put in the same file where you code. The instant grader automatically split your code file to an individual test case. Compile the file and run every test case. Then show the result to you.

For example. You write the add program in C language. You want to test it with some test case.
Here what its code look like.

#include <stdio.h>
int main() {
  int a,b; scanf("%d%d",&a,&b);
  printf("%d\n", a + b);
}

/*
===
1 2
---
3
===
4 5
---
9
===
*/

Usage

Clone this project.
Copy the grader file in root directory.
Paste to the same directory of you code. (Anywhere you want)
Run instant grader. (command below)
Done.

$ ./grader YOUR_FILE

Advance Usage

From the name of project. It's called CLI. you can customize mostly thing you want in this tools.

Command

$ ./grader [OPTIONS] YOUR_CODE
$ ./grader YOUR_CODE [OPTIONS]

# show help documentation
$ ./grader -h

Options

You can add some parameters to grader.
Something like ./grader -v for expose all accepted subject and rejected subject.

Option	Description	Default
-v	Show content in both accepted and rejected case	-
-o	Specific execute file path	-
-b	Specific bin path	./bin
-in STR	Input delimiter in script file	"==="
-out STR	Output delimiter in script file	"---"
--compile CMD	Custom compile command.	"gcc -o %prog %file"
--run CMD	Custom run command.	"%prog < %in > %out"

Note: -in and -out delimiter matching only the head of content line.

Options Pattern

Some options are freely used for "Non-specific file"

When you give the options --compile in command.
You have to enter the CMD maybe look like "gcc -o %prog %file"

You don't know the "Exactly" program name. (Because the instant grader will automatic generate program name)
Can use %prog or %file instead.

Option pattern	Description
%bin	Bin directory
%code	Your code file
%prog	Compiled program (for interpreted language this file not created)
%progx	Compiled program name (only a name not include directory path)
%in	Input test case file
%out	Output file after run case

Note: %in and %out option is individual file for each test case.
(Not all test case. The program will split test cases to individual file and put each test case to your program)

Option Pattern Scope

Some pattern not allow in some option.
When you customize --compile CMD. You can't use %in %out pattern.

	%bin	%code	%prog	%progx	%in	%out
-o	Y			Y
--compile	Y	Y	Y
--run	Y	Y	Y	Y	Y	Y
Y Can use

Example:

--compile "gcc -o %prog %file" Correct.
--compile "gcc -o %in %out" Incorrect.
--run "python %code < %in > %out" Correct.
-o "%code" Incorrect.
-o "%bin/%progx" Correct.

Example

Currently tools are support 6 languages. (C, C++, Java, Python, Ruby, Haskell)
If you need more language you can customize via --compile and --run option.
Even customize delimiter for test case in your file can edit via -in and -out

There are some sample file in samepleFile directory provide some basic stuff. And some sample test case in each file. Also include a file with stack overflow behavior you can test it with instant grader to see what its result look like.

Here a list of useful command

Multi-language support (depend on extension of file). No need to configure anything.

$ ./grader code.c
$ ./grader code.cpp
$ ./grader code.hs
$ ./grader code.java
$ ./grader code.py
$ ./grader code.rb

Verbose mode (Completely show all content in each test case).

$ ./grader code.cpp -v

Customize gcc compile with optimization level 3.

$ ./grader code.c --compile "gcc -O3 -Wall -o %prog %code"

Export compiled program.

$ ./grader code.c -o code
$ ./grader code.c -o code --compile "gcc -O3 -Wall -o %prog %code"

Customize bin path

$ ./grader code.cpp -b tmp/

Customize delimieter

#include <cstdio>
int main() {
  int N;
  scanf("%d", &N);
  printf("%d", N * 2);
}

/*
###
5
+++
10
###abc
7
+++eiei
14
###
*/

$ ./grader code.cpp -in "###" -out "+++"
$ ./grader code.cpp -in "##" -out "++"
$ ./grader code.cpp -in "#" -out "+"

Grade with Python file
(Program currently support Python. No need to specific compile and run option. This is just example)

$ ./grader code.py --compile "" --run "python %code < %in > %out"

Grade with Java file

$ ./grader code.java --compile "javac -d %bin %code" --run "java -classpath %bin %progx < %in > %out"

Multi combination

$ ./grader code.cpp -v -b tmp/ -o mycode --compile "g++ -Wall -O3 -o %prog %code" --run "%prog < %in > %out"

Build

For own customization. Here how to build the instant grader;

$ make build

While developing.

$ make

For testing

$ make test

Limitation

Platform support

Instant grader is supported in Linux platform. Maybe Windows in future.
But currently, Windows is not supported.

Language support & Prerequire

You NEED to install compiler of each language before using instant grader. But NO setting for compile and run message.

C (require gcc)
C++ (require g++)
Java (require javac, java)
Python (require python)
Ruby (require ruby)
Haskell (require ghc)

If you using another language you can specific by --compile and --run option.

Author

Kosate Limpongsa (Project owner)

License

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Ima8 / instant-grader-cli