[BUG] AttributeError: 'MosesTokenizer' object has no attribute 'split'

[Amgad Hasan]

Hello again.

I managed to run the server. However, I got this error when I was translating foreign audio into English:

  File "/home/eng_amghasan/whisper_streaming/whisper_online_server.py", line 215, in <module>
    proc.process()
  File "/home/eng_amghasan/whisper_streaming/whisper_online_server.py", line 186, in process
    o = online.process_iter()
  File "/home/eng_amghasan/whisper_streaming/whisper_online.py", line 276, in process_iter
    self.chunk_completed_sentence()
  File "/home/eng_amghasan/whisper_streaming/whisper_online.py", line 322, in chunk_completed_sentence
    sents = self.words_to_sentences(self.commited)
  File "/home/eng_amghasan/whisper_streaming/whisper_online.py", line 376, in words_to_sentences
    s = self.tokenizer.split(t)
AttributeError: 'MosesTokenizer' object has no attribute 'split'
whisper-server-INFO: killing process 138718

[Dominik Macháček]

hello,

have you installed the intended version? pip install opus-fast-mosestokenizer

[Amgad Hasan]

hello,

have you installed the intended version? pip install opus-fast-mosestokenizer

Yeah it works! Thanks
Maybe it's a good idea to create a requirements.txt file that contains librosa + the right version of moses tokenizer?
I'll open a pr.

[MagicSource]

has anyone able to install opus-fast-mosestokenizer on macOS?