Giters

mdabir1203 / Rust_Tokenizer_BPE

Byte-Pair Algorithm implementation (Karpathy version of Rust)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bpekarpathyllmtokenizer-nlp

A Rust implementation of the Karpathy implemented BPE Tokenizer Algorithm [[ WIP ]]

About

Byte-Pair Algorithm implementation (Karpathy version of Rust)

bpekarpathyllmtokenizer-nlp

Languages

Language:Makefile 50.9%Language:Rust 49.1%