Parallelise the indexDocument call in IndexService
boyter opened this issue · comments
There is no reason to run the conversion to documents not in a parallel stream. This should speed things up in theory because the buildDocument call can be slow for large documents. It will however use more RAM so probably need to feature toggle it.
Also need to verify that it is indeed faster.
Relates to #133
On a sample project with lots of complex files, the index time for the current version took about 200 seconds. The new one took about 140 seconds. Not a bad improvement for a few lines of code.
Modifying it further to see if if the index writer is indeed thread safe took that time down to 90 seconds. Might want to burn this one in for a while before considering a release though.
It does appear to peg the CPU I am testing on though which is a better result than the previous runs.