serjtroshin / sparse_continuous_distributions

This repository provides open-source code for sparse continuous distributions and corresponding Fenchel-Young losses.

spcdist: sparse continuous distributions in Python

---

This repository implements the beta-Gaussian 1-d and n-d distributions, as well as continuous attention mechanisms based on it.

This is the companion code of the paper

Sparse Continuous Distributions and Fenchel-Young Losses André F. T. Martins, Marcos Treviso, António Farinhas, Pedro M. Q. Aguiar, Mário A. T. Figueiredo, Mathieu Blondel, Vlad Niculae preprint link

which builds upon

Sparse and Continuous Attention Mechanisms André Martins, António Farinhas, Marcos Treviso, Vlad Niculae, Pedro Aguiar, Mario Figueiredo NeurIPS 2020 link

Requirements:

numpy, scipy, (optional: pytorch>=1.8.1)

(Note: pytorch is optional, the spcdist.{scipy, scipy_1d} modules work without it.)

Installation:

pip install .
# or
pip install .[torch]  # to also install the pytorch dependency 

Applications:

• Speech Classification
• Visual Question Answering