A fully automated test suite for 42 Philosopher's project. After working on this project for a number of days, I grew tired of rewriting all my test by hand hundreds of times after each small change in code. The tester I found did not do all the test I needed or were not fully automated so I decided to make my own. I hope it can save you time.
nesvoboda/socrates And newlinuxbot/Philosphers-42Project-Tester whose project inspired my tester.
The author of this tester shall not be held responsible for any damage that may result from the usage of this script. This script is provided as is and the user assumes all responsibility and risk associated with its usage. Moreover this tester should not be used during defence.
By default this tester performs a number of death cheks. During thoses checks the philosophers must die.
Death Occurence The script checks if a died message was printed. Proper Simulation Ending The script checks that the simulation ends properly after the death log (i.e. No message of any kind are logged after the death message). Death Timing The script checks the proper timing of death log. Valgrind The script reruns the program to sanitize it with valgrind. If valgrind detects any error it will be reported in the script as a leak. But it might be an invalid read size, always rerun the failed parameters to see for yourself. Helgrind The script reruns the program to sanitize it with helgrind. If helgrind detects any error it will be reported in the script as a race condition. But it might be a deadlock, always rerun the failed parameters to see for yourself.
By default this tester performs a number of meal checks. During thoses checks the philosophers must not die and eat all their meals thus ending the simulation.
Meal Occurence The script checks if each philo has eaten enough meals, and if they have eaten more than they should (No more than nb_of_meals + 2). Valgrind The script reruns the program to sanitize it with valgrind. If valgrind detects any error it will be reported in the script as a leak. But it might be an invalid read size, always rerun the failed parameters to see for yourself. Helgrind The script reruns the program to sanitize it with helgrind. If helgrind detects any error it will be reported in the script as a race condition. But it might be a deadlock, always rerun the failed parameters to see for yourself.
By default this tester performs a number of CPU usage test. The program tested must not use more than 50 % of the CPU during the test. The script keeps track of the CPU usage throughout the test.
The tester will launch for 40 seconds the tested program to check how long your program can run.
By default the tester will performs a number of checks with invalid program arguments. It will look for any segfault. If it finds one it will error out. If it doesn't find any, it will check the number of lines in the output. If there is one line, it will check wether invalid|error|wrong are present in the output indicating error handling. If it finds one it will print a success message. If it doesn't find one, it will print out the last two lines and leaves you to decide if the input was correctly handled. Finally if the script doesn't detect any output, it will print a success message.
The tester will check wether the threads are correctly joined in case of insufficient memory on thread creation. In the case that N-th thread fails because of insufficient memory, and they are not joined, leaks will be detected and reported by the tester. Due to the delicate nature of testing this behavior, here's the test I perform in case you would like to test it for yourself:
(ulimit -v 180000; valgrind --leak-check=full --errors-for-leak-kinds=all "$program_path/$program_name" 85 60 60 60)
For the bonus, the checks performed are:
- Death Checks
- Meal Checks
- Program Arguments Checks
- No Death Checks And
The tester will check how many forks were created.
Contributing to this tester is fairly easy. You can use the function already made to add new test.
Death Check: test_philosopher_death "$target" "$1" "1" "800" "200" "200" "1"
Do not change the first and second parameters. The third parameters is the nb_of_philo, fourth is time_to_die, fifth is time_to_eat, sixth is time_to_sleep and seventh is the test_number.
Meal Check: test_philosopher_meals "$target" "$1" "5" "800" "200" "200" "7" "10"
We only added the nb_of_meals as seventh parameter.
Check CPU Usage: check_cpu_usage "$target" "$1" "2" "800" "200" "200" "70" "15"
Same structure as Meal Check.
Check No Death: check_philosophers_nodeath "$target" "$1" "5" "800" "200" "200" "17"
Same structure as death check. For now the function automatically kills at 40 seconds.
Check Program Argument : check_program_arguments "$target" "$1" "-5" "600" "200" "200" "5" "18"
Same structure as Meal Check.
As you can see adding checks is pretty straightforward.
I will add sanity check on the existence of the commands used. I will add progress bar.