komodojp / tinyld

Simple and Performant Language detection library for NodeJS

Home Page:https://komodojp.github.io/tinyld/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

missing 'data' directory in package

leonardlin opened this issue · comments

I'm trying to install the tinyld to work on it a bit. (see the two issues I opened)

npm run build runs into a problem with missing './data/bench/tinyld.json''

I saw in the code that it's created during bench
npm run bench
npm run train

both run into missing data-directory files.

Can these files be added to the repo or made available for download?

so the process, sorry I should have documented that better in here

  • yarn train is to generate profile files. It require full tatoeba dataset which add up for more than 500MB and are updated weekly, so no it's not gonna be added to git directly. But the profiles files are in the repository and are enough for other steps like building. (src/profiles/light.json, src/profiles/normal.json)
  • yarn bench is to generate benchmark data. And this is optional, you shouldn't need to run it

But it's a totally valid point, we shouldn't need to run yarn bench before yarn build.
So I added that data/bench folder to the repo and make it an optional step.

👍