Reduce seemingly inactive / dead time

[Bernhard Schuster]

Is your feature request related to a use-case? Please describe.
Currently for larger repositories, the program blocks for quite a bit (~10s of seconds) and ultimately spits out all the output.
This sometimes poses the question if the program does anything at all and surely caused a few CTRL+C actions, when it actually was running just fine.

Describe the solution you'd like
Currently one giant documentation object is created and then displayed, but it would be much more flexible if we would work on a per document basis.

With that said, a worker pool could be utilized to do the actual checking and the output could be an async core just processing an mpsc channel.

Describe alternatives you've considered

• Use rayon with a split into documents, that would reduce the time of initial blockage and would be a low hanging fruit.
• Add a progress bar, but that would only add complexity to the codebase without any speedup, and adds an additional burden to impl it in the future.

So both alternatives are kind of 🍌

---

Relevant files:

• documentation.rs
• check.rs
• main.rs

[Bernhard Schuster]

The rayon impl is already done, currently nlprule takes quite a bit of time to parse the inbaked rules data. So that's something left todo.

[Bernhard Schuster]

bminixhofer/nlprule#66 remedies the nlprule upfront loadtime a bit as of v0.8.0-beta.3, but not quite enough to close this

[Bernhard Schuster]

Now processing is async so, output is not bound to all the processing being complete anymore. The whole thing is now somewhat latency optimized.

The remaining item is to cache the nlprule::Tokenizer so the initial loading time is kept to a minimum, which will require changes to nlprule and - if one wants to avoid regex recompilation - also to fancy_regex/regex.