huggingface / datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Examples don't reflect current sate of codebase

hynky1999 opened this issue · comments

Some of the examples use old config for setting the hash config etc..
Most of them thus currently don't work out of box.

Steps to fix:

  1. Change the linter setting to also check the examples folder
  2. Update the examples to reflect current sate of the code