Discourage rot in linting documentation
jugglinmike opened this issue · comments
A number of open issues describe misalignment between the behavior of the WPT linter and the state of its documentation [1] [2] [3]. While we should fix each of these, I'd also like to take the opportunity to address what seems like a systemic problem.
Automatically generating documentation from the source code seems ideal. However, we're somewhat limited by our existing tools (Jekyll via GitHub Pages).
We could work around this limitation by configuring TravisCI to synchronize the linter source code and the documentation. It could perform this synchronization after every push to master and automatically committed to the repository. Integrating a third-party service with elevated rights is fairly complex, though, and I don't know if it's worth it.
Alternatively, we could extend the linter to verify agreement between the documentation and its own source code. We'd still have to maintain some duplication between the two, but this would at least prevent rot.
@gsnedders you are the foremost authority on linting in WPT. What do you think about this?
[1] web-platform-tests/wpt#5299
[2] web-platform-tests/wpt#10501
[3] web-platform-tests/wpt#11479
In a sense this is related to our problems with documenting our tooling in general: while we could do quite a lot with inline comments, we can't get them on our current website in any sane way.
I would love more developer documentation of our tooling (which I realise is a separate concern to this), and maybe we should revisit how we want to go about that (rST (ewww) and Sphinx and autodoc and mostly inline docs and use readthedocs.org?).
If we were to setup a separate build step for our documentation (and we can do this, I'm just not super-keen), we could quite easily do something to integrate both autogenerated documentation from Python (and JS for testharness.js?) into our larger documentation site, but that needs some agreement as to what approach we're taking and how to ensure it stays up to date.
#2 touches on developer documentation of our tooling too.
Funny you should say that! I have a proposal to move to RST/Sphinx/Readthedocs, but I've been hesitant to share because it felt too tech-heavy. There's a lot that we can do which doesn't require any new tools, after all. If you think improved generation is best way to handle this particular issue, then it might make sense to simply patch the docs for now and kick the problem of rot down the road a little.