microsoft / BlingFire

A lightning fast Finite State machine and REgular expression manipulation library.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to generate bin model file based on specified vocab file for wordpiece tokenizer ?

Distancehe opened this issue · comments

For wordpiece tokenizer, do we support generate bin file based on specified vocab.txt ? If so, is there a wiki for detailed step ? Thanks !