Unique words accuracy?

Question

ravidreams opened this issue 9 years ago · comments

I ran this today and got

1962477 final-tamil.txt

But the .txt file has lot of repeat words. Is this normal? I thought final-tamil.txt gives unique list of words.

Shrinivasan T · Answer 1 · Thu Jan 07 2016 09:22:42 GMT+0800 (China Standard Time)

Thanks for reporting.

Changed the process to find unique words.

Check now.

Ravishankar Ayyakkannu · Answer 2 · Thu Jan 07 2016 16:30:06 GMT+0800 (China Standard Time)

I ran again and got 1360126 unique words. It is different from 1197913 you show in readme. Did you get this count today?

Also, you have left the sort function now. Need it back.

Shrinivasan T · Answer 3 · Fri Jan 08 2016 10:28:12 GMT+0800 (China Standard Time)

My results are for the dump on Jan 6.
Added sorting.