daac-tools / vibrato

🎤 vibrato: Viterbi-based accelerated tokenizer

https://docs.rs/vibrato

daac-tools/vibrato Issues

Add N best results
Updated 24 days ago5
Loading a large dictionary such as UniDic cwj 2023-02 is slow in comparison to mecab
Updated 25 days ago2
Using a memmap for the dictionary
Closed 3 months ago2
The difference between Vibrato and Vaporetto regarding the performance for a general tokenizing
Closed a year ago6
Embed versions into models
Closed a year ago
65535を超える文字で分かち書きできない。
Closed a year ago3
Dictionary::read() cannot read compressed dictionary
Closed a year ago9
ipadic-mecab-2_7_0-small is broken
Closed a year ago3
Remove lifetime parameter from `Worker`
Updated a year ago
Distribute compiled dictionaries from neologd-ipadic
Closed 2 years ago1
Add compile option for user dictionary
Updated 2 years ago
Distribute compiled dictionaries from JumanDic
Closed 2 years ago3
Update the benchmark results
Updated 2 years ago
Distribute precompiled dictionaries instead scripts download external resources
Closed 2 years ago
Keep member functions of a struct in the same file
Closed 2 years ago
Length 65535 causes errors
Closed 2 years ago
Support text longer than 65535 characters
Closed 2 years ago2
Keep an easy description in the top-level readme
Closed 2 years ago4
Take `BufRead` as arguments
Closed 2 years ago7