Run CI using CTest scripts
friendlyanon opened this issue · comments
This is just an idea, but I like how check-type-align's CI looks. It could be worth considering using these in generated projects' CI.
I did not know about this sort of functionality. Cool! I also agree that the referenced CI looks good and makes it easier (even though it's already extremely easy) for people to run what CI runs, locally.
One thing I do like about the currently implemented CI (and something that is lost with the current CTest script implementation) is the separation of steps for both readability of the logs and easily determining which step (configure, build, test) has failed.
Fortunately, with GitHub Actions, there is a way to separate logs within a step using workflow commands, for example:
echo "::group::My title"
echo "Inside group"
echo "::endgroup::"
It requires echoing a command which is interpreted by the runner. However, this means that the actual CTest files will contain more lines of CMake to do this command echoing; and a few more lines if you want to check whether the CI environment variable is set to determine whether you actually echo the commands (in the event you want to support running the same script on a local developer's machine).
Just something to consider!
Seemed like a great idea, but unfortunately the ordering of the messages is somehow messed up friendlyanon/check-type-align@d3d0538 https://github.com/friendlyanon/check-type-align/actions/runs/1762132747?check_suite_focus=true
Well that is in fact a real usability/convenience issue. With longer logs I can see it being not as nice to navigate, with normal verbosity you need to upload artifacts to see the actual outputs that require a manual download, unzipping and staring at XML files.
I have added the repository to the examples, so if anyone is curious about that, then it's going to be there.
One additional note about CTest scripts: the ctest_build command allows users to gather the number of warnings from the build output, which can be used to drive some additional tooling that can be used to track this metric over time. This is relevant because of this part in Daniel Pfeifer's talk, which I agree with in that a blanket -Werror or equivalent isn't exactly productive.