hplt-project / OpusCleaner

OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.

Home Page:https://pypi.org/project/opuscleaner/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Move filters to their own packages

jelmervdl opened this issue · comments

commented

Right now, opuscleaner is tricky to install because it will pull in all the dependencies for most filters.

I'm tempted to remove most of the external filters (opusfilter, bicleaner, etc) from the main opuscleaner package, and move them (their requirements + their json files) into their own python packages, e.g. opuscleaner-bicleaner.

The alternative is using optional dependencies but I don't see a way to hide the filters from the interface when the dependencies aren't installed. And I think that would be a wanted feature.