subham-agrawall / Efficient-Pandas

This repository is a compilation of resources for reducing the data size.

Efficient Pandas

This repository is a compilation of resources for reducing the data size and optimizing python pandas.

Data compression and storage

1. Basics of datatype optimizations - https://www.youtube.com/watch?v=u4_c2LDi4b8
2. Basics of storage optimizations - https://www.youtube.com/watch?v=u4rsA5ZiTls
3. How to reduce data size? Detailed steps - https://www.kaggle.com/competitions/amex-default-prediction/discussion/328054

Speeding up Python codes

1. Speed up python dataframe loops - https://www.youtube.com/watch?v=SAFmrTnEHLg