Continuous integration is over-permissive
GuillaumeGen opened this issue · comments
Currently, the Makefiles in the libraries include -include .depend. But this minus sign implies that if dkdepoutputs a parse error, then the Makefile exits immediately with the error code 0.
Hence, we check that if our libraries remain syntax-correct, then they also remain type-checkable, but we do not check that the evolution of the syntax does not break the libraries.
A related question is: "What should we do with the libraries when the syntax evolves?"
It depends:
- Either the new syntax extends the old one, in that case, for the CI nothing needs to be done
- Otherwise, there might be an incompatibility. In that case, libraries should be regenerated.
To have a proper CI, this requires to have a CI to check the generation of libraries that we do not have yet. In the meantime, I think the best thing to help the test would be to create a PR for each project which generates Dedukti code (Krajono,Coquine) and check that the code generated is valid with the Dedukti PR. Once the Dedukti PR is merged, then the other PR can be merged. Unfortunately, I am not sure that GitHub proposes something better.
I think checking the generation of the libraries is the CI of the generators, not the one of Dedukti (but this is debatable, maybe a CI of the whole eco-system is preferable).
However, even extending the syntax breaks some previously valid files: for instance adding a keyword injective broke the matita's arithmetic library, because one identifier was called injective and it is now lexed as a keyword.
Adding a keyword is not purely a syntactic extension then.
I was exactly saying that that the generation of the libraries should be in the CI of the generators which does not exist, and this is why testing the syntax needs to be done manually now. But here, I think we are also limited by GitHub so it might not be easy to find a nice solution.
The fact that libraries are part of the CI is an issue:
- Libraries should have there own CI
- We lack of unit tests
Please reopen this issue if it becomes an issue again.