Automated Testing for Spade Builds
recursiveforte opened this issue · comments
As we start to put more time into improving Spade (whether that's feature adds or small fixes to make development easier for HCers), it would be useful to be able to automatically verify Spade builds.
I propose a two-part (really, three-part) solution to testing an individual Spade build before merge:
- Part zero includes a script to compile Spade using Docker, so any behavior isn't reliant on the environment of whoever originally built the UF2 / wrote the PR.
- Part one includes a demo program to be manually run to test hardware functions of the Sprig console (buttons, screen, lights, RNG, and speakers). Would include a demo tune, tests for each button, and a gradient shown on screen.
- Part two includes automated testing of ~10 games from the gallery. Button presses would be sent through a microcontroller wired to each button input, and a program would check game state using logs after each run. Deterministic behavior would be ensured by adding a function to manually set a seed for Spade's RNG.
@grymmy thoughts?
I have investigated sprig testing in somewhat depth due to wanting to implement some soon. Unity Test Framework is an interesting choice.
This seems like a good idea. I'm not sure I have the details to comment too much on specifics. What spade modifications are we planning on pushing that will necessitate testing (obviously it's always good to have)?
Any modifications we make to spade can/should be tested. I think there are plenty of opportunities to improve the fundamentals of spade, including but not limited to the available api set - it is decidedly not perfect, and anything we change we want to ensure we don't break the world.