Improve language profile generation
kefniark opened this issue · comments
Description
To be more accurate, an algorithm like this one needs big datasets and processing abilities. Currently the script execution time is getting longer and longer but only on one core.
Multi threading and multi processing could allow us to ingest more data and get a better detection accuracy
with 1.1.0 most language data are cached and not generated every time.
This allow really fast profile generation and iteration