dlloyd09 / autograder

An autograder script for CS50 staff

README

Getting Ready

1. Download the latest version of the ZIP of this script and its dependencies by visiting https://github.com/dlloyd09/autograder/archive/master.zip.

2. Unzip the ZIP so that its contents are in a directory that contains only folders containing students' Problem Set 6 submissions.

   1. Indeed, due to the way we organized the Problem Set 6 distribution code, negative-words.txt and positive-words.txt are required to be in the same directory as the script. Example:

~/workspace/students $  README.md  check50.py  grade6.py  negative-words.txt  positive-words.txt  setup.sh  student1/ student2/ student3/

3. Execute ./setup.sh

4. Ensure that each student’s submission is properly formatted before running this script. This might require renaming files in student submissions (e.g. from caesar to caesar.py or from tweets.py to tweets) after checking out the pset6 branch. (Note any changes you have to make, as they are correctness issues!) In particular, a student’s submission must be organized as follows:

~/workspace/students $ cd student1
~/workspace/students/student1 $ tree
.
├── caesar.py
├── crack.py
├── credit.py
├── greedy.py
├── mario.py
├── sentiments
│   ├── analyzer.py
│   ├── application.py
│   ├── helpers.py
│   ├── negative-words.txt
│   ├── positive-words.txt
│   ├── requirements.txt
│   ├── smile
│   ├── templates
│   │   ├── index.html
│   │   ├── layout.html
│   │   └── search.html
│   └── tweets
└── vigenere.py

5. Edit grade6.py by putting your own API Key and API Secret in place of the TODOs.

Running

1. Execute python grade6.py. It takes just about 1 minute per student to execute all 9 of the checks (on a sample of 37 submissions, the script took 49 minutes to run), so once all the pre-work is done, set the script to run and go grab a bite to eat!

   1. This script will run mario_less and mario_more on all students, probabilistically determining which the student most likely completed based on how many tests their code passes in each.

   2. There is no check for application.py; you will have to assess that one manually.

2. When finished, view student results in the newly created results/ directory, and input scores into Gradebook.

Notes

1. During the tweets check, this program will create and delete files called __staff.txt and __student.txt. If for whatever reason you have files called that, you might want to move them; they will be clobbered.

2. In the output text files, if a student’s submission passes, you will notice that the script reports one more check than is printed out; to make things cleanly align with check50, I have deleted the "file exists" check (which counts as a passed test) from the text files.

3. If a student has not named his or her files correctly, or hierarchically organized their submission the way we expect, the script will not work as intended. You may need to rename files or rearrange their submission to get it to work correctly.