Testdrive PlanetMath workflow - invalidation and indexing
dginev opened this issue · comments
@holtzermann17 I would like to see that we have a working indexing and invalidation workflow in PlanetMath before I release to CPAN, if possible.
I would like to try to help with that, if you tell me how to.
Can you start by outlining what's required in the form:
Planetary> some event, Planetary does something
NNexus> in response, NNexus does something else
time passes> something else happens
etc.? I'm happy to work on whatever's needed with the Planetary side and/or cron integration on PM....
- An article is edited/uploaded in Planetary
- Article is converted by LaTeXML
- Planetary sends the resulting HTML to nnexus: /indexentry route
The response to that request is a JSON array of article URLs to be invalidated. Save that for step 5. - Auto-link and serve the newly saved article we just indexed over (/linkentry route). Save returned HTML in Planetary.
- For each of the URLs that were returned, request auto-linking (/linkentry route). Save returned HTML in Planetary.
Cool! Steps 1, 2 and 4 are already working, so we're 3/5ths of the way there :-). The other items should be easy to add and should be easy for me to code up (you can provide a code review).
Big picture: I wonder if we should address MathHubInfo/Legacy-planetary#288 in the course of solving this, since step 5 might end up being somewhat processing intensive, and we don't want to keep the user waiting. This is the only part of this ticket that will require some thought. I'm not sure if there's an easy PHP way to "background" the requests in step 5 -- if there is, we could just use that and skip the more complex features of the issue I linked above.
Nice for being 3/5 there :)
Right, we need to make sure 5 happens in the background. Actually, since an article must never link to itself, the best order for user experience is to interchange steps 3 and 4, so that we first autolink and return for the user to preview, while indexing and invalidation happen in the background in some detached process.
We should investigate the best way of doing such detached jobs in Drupal.
I have pushed to CPAN, now at minor release alpha2, which has most of the docs updated and some of the tests updated to work cross-platform.
Could we make some PlanetMath workflow tests as well, so that I am sure things work as expected? I have an actual test file that stresses the invalidation, but maybe it's nice to have this done as well just in case for the official release.
Moving to 3.0 milestone.