Diaz-De-Vivar / ML_Resources

GitHub Repo with various ML/AI/DS resources that I find useful

ML Resources

GitHub Repo with various ML/AI/DS resources that I find useful. I'll populate it with links to articles, libraries, and other resources that I come across.

Tabular Data ML

General

• Tabular Data: Deep Learning is Not All You Need: https://arxiv.org/abs/2106.03253

NNs for Tabular Data

• pytorch-widedeep, deep learning for tabular data IV: Deep Learning vs LightGBM. A thorough comparison between DL algorithms and LightGBM for tabular data for classification and regression problems: https://jrzaurin.github.io/infinitoml/2021/05/28/pytorch-widedeep_iv.html

• SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training: https://github.com/somepago/saint (repo)

• SAINT paper: https://arxiv.org/abs/2106.01342

• Regularization is all you Need: Simple Neural Nets can Excel on Tabular Data https://arxiv.org/abs/2106.11189

• Revisiting Deep Learning Models for Tabular Data: https://arxiv.org/abs/2106.11959v1

Boosting

• XGboost documentation: https://xgboost.readthedocs.io/en/latest/

Neural Networks - General

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine: https://arxiv.org/abs/2012.00152

NNs vs Kernel Methods

The quest for adaptivity: exploring https://francisbach.com/quest-for-adaptivity/

Stacking

vecstack is a handy little library that implements the stacking transformations with your train and test data. It has both the functional interface and the sklearn fit transform interface: https://github.com/vecxoz/vecstack

Autoencoders

RecoTour III: Variational Autoencoders for Collaborative Filtering with Mxnet and Pytorch: https://jrzaurin.github.io/infinitoml/2020/05/15/mult-vae.html

Hyperparameter Tuning

Auto ML

• Automated Machine Learning: Methods, Systems, Challenges. Probably the single best monograph on AutoML. Published in 2019, so not quite the cutting edge, but still very useful. https://www.amazon.com/Automated-Machine-Learning-Challenges-Springer-ebook/dp/B07S3MLGFW/
• H2O Driverless AI documentation: https://docs.h2o.ai/driverless-ai/latest-stable/docs/userguide/index.html

Computer Vision

• Scaling Vision Transformers: ‘As a result, we successfully train a ViT model with two billion parameters, which attains a new state-of-the-art on ImageNet of 90.45% top-1 accuracy. The model also performs well on few-shot learning, for example, attaining 84.86% top-1 accuracy on ImageNet with only 10 examples per class.’ https://arxiv.org/abs/2106.04560

Social Media Analysis

• Birds of the Same Feather Tweet Together. Bayesian Ideal Point Estimation Using Twitter Data. Analysis of homophily of Politicians on Twitter. http://pablobarbera.com/static/barbera_twitter_ideal_points.pdf
• Leadership Communication and Power: Measuring Leadership in the U.S. House of Representatives from Social Media Data: https://preprints.apsanet.org/engage/apsa/article-details/60c239b28214c646e0a61589

Policy

• Final Report: National Security Commission on Artificial Intelligence https://www.nscai.gov/wp-content/uploads/2021/03/Full-Report-Digital-1.pdf

ML Competition Platforms

• Kaggle: https://www.kaggle.com
• Eval AI: https://eval.ai
• Zindi: https://zindi.africa
• Driven Data: https://www.drivendata.org
• Codalab: https://competitions.codalab.org/

Other Resources

Blockchain

A from-scratch tour of Bitcoin in Python http://karpathy.github.io/2021/06/21/blockchain/