Shun-Ryu / awesome-self-supervised-learning-for-tabular-data

A collection of research materials on SSL for tabular data

Awesome Self-supervised Learning for Tabular Data

This repository contains the frontier research on self-supervised learning for tabular data which is a popular topic recently.
This list is maintained by Wei-Wei Du. (Actively keep updating)

Papers

• VIME: Extending the Success of Self- and Semi-supervised Learning to Tabular Domain (NeurIPS'20)
  • Supplementary
  • Code
• CORE: Self- and Semi-supervised Tabular Learning with COnditional REgularizations (NeurIPS'21)
• SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning(NeurIPS'21)
  • Supplementary
  • Code
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption (ICLR'22 Spotlight)
• STab: Self-supervised Learning for Tabular Data (NeurIPS'22 Workshop on Table Representation Learning)
• SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (NurIPS‘22 Workshop on Table Representation Learning)
  • Code
• TabNet: Attentive Interpretable Tabular Learning (AAAI'21)
  • Code
• TransTab: Learning Transferable Tabular Transformers Across Tables (NeurIPS'22)
  • Code
  • Blog

Use correlation to capture relations between features

• Self-Supervision Enhanced Feature Selection with Correlated Gates (ICLR'22)
  • Code
• Local Contrastive Feature Learning for Tabular Data (CIKM'22)

Use pretrained language model to learn better representation

• TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data (ACL'20)
• LIFT: Language-Interfaced Fine-Tuning for Non-language Machine Learning Tasks (NeurIPS'22)
  • Code
• Pretrained Transformers As Universal Computation Engines
  • Code

Tutorials

• Self-Supervised Learning: Self-Prediction and Contrastive Learning (NeurIPS'21)