Winning solution to the Avito CTR competition
Some high level description of the solution can be found in the .pdf file.
How to run:
- change the folder at the top of _fast_10pct_run.R and _full_100pct_run.R to point to where the data files are stored
- (recommended) run _fast_10pct_run.R and verify that it produces expected results -- this will take a few hours
- run _full_100pct_run.R to produce a submission file.
Summary of other files: avito_data1.R -- Load datasets avito_phone.R -- feature extraction from phone dataset avito_search.R -- feature extraction from search dataset avito_visit.R -- feature extration from visit dataset avito_cat_cat.R -- more features + data merge avito_data_merge.R -- rest of the features + data merge avito_train_xgb.R -- fit xgboost models avito_utils.R -- some utility functions
Hardware requirement: To run the full data solution (_full_100pct_run.R) I recomment a machine with 256GB of ram + (at least) 200GB of swap space. This is due to the combination of my own inefficient code and inefficiency in R's memory handling.