datomi79 / KaggleLib

A library of most useful basic functions for Kaggle competitions

KaggleLib

Library contains most useful functions to participate in Kaggle and other machine learning competitions. Module is divided into parts usually included in machine learning application pipeline: Preprocessing -> Features engineering -> Features selection -> Model tuning -> Model training -> Model predicting -> Building ensemble of models

Requirements

All dependencies listed in requirements.txt

Installation and setup

0. clone repository
1. cd KaggleLib
2. pip install -r requirements.txt
3. pip install .
4. check installation by running example: python examples/example.py

API

The library contains following parts:

0. Model - generic class for machine learning models

   • type : model type (XGBoost, LightGBM, Keras or Scikit-Learn)
   • params : dictionary of model parameters
   • model : instance of model object
   • cv_score : cross-validation score

1. Preprocessing

   • hash_data : hashing of categorical columns (one-hot)
   • normalize_data : numerical data normalization

2. Feature engineering

   • make_numerical_interactions : feature interactions of 2 and 3 order, operations: sum, division, multipliciation, division
   • make_categorical_interactions : categorical feature interactions of 2 and 3 order
   • categorical_target_encoding:
   • logarithm : log feature transformation
   • exponent : exponent feature transformation
   • sigmoid : sigmoid feature transformation
   • trgonometry : sin, cos, tan feature transformation

3. Feature selection

   • genetic_feature_selection : select subset of features with best cross-validation metric by genetic algorithm (evolutional change of features subsets)

4. Model tuning

   • cross_validation : calculate cross-validation score of a model
   • tune_lgbm : find best LightGBM parameters by HyperOpt
   • tune_xgb : find best XGBoost parameters by HyperOpt

5. Model training

   • train_keras : train Keras model
   • train_lgbm : train LightGBM model
   • train_xgb : train XGBoost model
   • train_sklearn : train Scikit-Learn mdoel

6. Model predicting

   • predict_keras : prediction by Keras model
   • predict_lgbm : prediction by LightGBM model
   • predict_xgb : prediction by XGBoost model
   • predict_sklearn : prediction by Scikit-Learn model

7. Model ensembles

   • stacking : creating stack of model using out-of-fold predictions technique

8. Utils

   • make_folds : split data into folds
   • generate_keras_model : generate Keras model by dictionary
   • HistoryCallback : callback to preserve Keras training information on every epoch