Actual C Sharp Byte Pair Encoder that works. Use bin folder or add your own data to be able to train your own model, this model is then used to encode into train.bin and val.bin binary files to use to train an LLM or similar.
Home Page:https://animaai.co.uk
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
