Giters
Abel2076
/
json2binidx_tool
Geek Repo:
Geek Repo
Github PK Tool:
Github PK Tool
Stargazers:
75
Watchers:
2
Issues:
2
Forks:
24
Abel2076/json2binidx_tool Issues
transform parquet file into binidx
Updated
8 months ago
RWKV tokenizer should generate uint16 indices, instead of int32
Updated
a year ago