dirty_cat
dirty_cat is a Python module for machine-learning on dirty categorical variables.
Website: https://dirty-cat.github.io/
dirty_cat's SuperVectorizer automatically turns pandas data frames into numerical arrays suitable for learning.
For a detailed description of the problem of encoding dirty categorical data, see Similarity encoding for learning with dirty categorical variables [1] and Encoding high-cardinality string categorical variables [2].
Installation
Dependencies
dirty_cat requires:
- Python (>= 3.6)
- NumPy (>= 1.16)
- SciPy (>= 1.2)
- scikit-learn (>= 0.21.0)
- pandas (>= 1.1.5)
Optional dependency:
- python-Levenshtein for faster edit distances (not used for the n-gram distance)
User installation
If you already have a working installation of NumPy and SciPy,
the easiest way to install dirty_cat is using pip
pip install -U --user dirty_cat
Other implementations
References
| [1] | Patricio Cerda, Gaël Varoquaux, Balázs Kégl. Similarity encoding for learning with dirty categorical variables. 2018. Machine Learning journal, Springer. |
| [2] | Patricio Cerda, Gaël Varoquaux. Encoding high-cardinality string categorical variables. 2020. IEEE Transactions on Knowledge & Data Engineering. |