eugenium

flash-attention

Fast and memory-efficient exact attention

micrograd

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

XLM

PyTorch original implementation of Cross-lingual Language Model Pretraining.

bittensor

Internet-scale Neural Networks

continual_learning_papers

Relevant papers in Continual Learning

CLIP-Mesh

Official implementation of CLIP-Mesh: Generating textured meshes from text using pretrained image-text models

lookahead.pytorch

lookahead optimizer (Lookahead Optimizer: k steps forward, 1 step back) for pytorch

atari-representation-learning

Code for "Unsupervised State Representation Learning in Atari"

hidden-networks

sgg

Train Scene Graph Generation for Visual Genome and GQA in PyTorch >= 1.2 with improved zero and few-shot generalization.

Maximally_Interfered_Retrieval

Codebase for "Online Continual Learning with Maximally Interfered Retrieval"

InfoPro-Pytorch

Learning recognition/segmentation models without end-to-end training. 40%-60% less GPU memory footprint. Same training time. Better performance.

head2toe

LocalSGD-Code

DGL

adaptive-quantization-modules

Code for "Online Learned Continual Compression with Adaptive Quantization Modules"

sign-symmetry

online-alt-min

Source code for paper Choromanska et al. -- Beyond Backprop: Online Alternating Minimization with Auxiliary Variables -- http://proceedings.mlr.press/v97/choromanska19a.html

VideoNavQA

An alternative EQA paradigm and informative benchmark + models (BMVC 2019, ViGIL 2019 spotlight)

parametricScatteringNetworks

[CVPR'22] Parametric Scattering Networks

gqa_single_hop_baseline

A simple but well-performing "single-hop" visual attention model for the GQA dataset

AML

Codebase for " Reducing Representation Drift in Online Continual Learning"

few_shot_3dr

Implementation of "Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors" ECCV'20 paper.

interferometric-graph-transform

This repository contains the code corresponding to the ICML-2020 publication: https://arxiv.org/abs/2006.05722

congrad

distributed-dtp

Distributed implementation of our proposed DTP algorithm parallelizing feedback weight training across GPUs (ICML 2022)

femnist-dataset-PyTorch

A custom PyTorch dataset class for the federated learning femnist dataset

flash_fl

online-alt-min

Source code for paper Choromanska et al. -- Beyond Backprop: Online Alternating Minimization with Auxiliary Variables -- http://proceedings.mlr.press/v97/choromanska19a.html