Adding Tests + CI
bhuvy2 opened this issue · comments
The CI can be mostly copied over from broadway-api, but it'd be nice to have some assurance checks on each commit.
I think this issue needs to be shifted to Chain Link. Broadway grader just communicates with the API and passes on the job to chainlink to run. @nmagerko wrote some tests for chain link which tests what we want to test.
What would we want to test on the broadway-grader side of things?
There is also a discussion of whether we want to merge chain link. Assuming we want to keep the different, we should just sanity test heartbeating and whether chainlink calls the right funcs
Yep, I think we should keep them separate. Chainlink is meant to be a public python library for running a chain of containers. This project is more Broadway specific.
I don't want to get into the whole debate of what warrants a package
(I strongly side on functions or few functions don't warrant packages re: the left-pad dabacle, and chain-link in spirit is a different functionality, but is not a collection of functions or otherwise.
Either way, mocking should be added.
For testing, I think this project might need some refactoring first (this project can be tested in a way that is separate from how chainlink tests). We should at least be able to test that a job is submitted to some black box (e.g. chainlink mocked) to run, and that the results we get back from the broadway-grader routine that does that makes sense.
The actual execution of running containers in sequence should be tested more throughly, but that is not the responsibility of this project. I'd say that needs to be a chainlink issue.
As of now,
- We have added integration tests which run the api and grader together.
- Nick has refactored this project already.
- Chainlink has some tests.