Giters

shjwudp / c4-dataset-script

Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese data processing and cleaning methods in MassiveText.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

shjwudp/c4-dataset-script Stargazers