efficient-transformers

There are 2 repositories under efficient-transformers topic.

swz30 / Restormer
[CVPR 2022--Oral] Restormer: Efficient Transformer for High-Resolution Image Restoration. SOTA for motion deblurring, image deraining, denoising (Gaussian/real data), and defocus deblurring.
cvpr2022 defocus-deblurring efficient-transformers high-resolution image-deblurring image-deraining image-restoration low-level-vision motion-deblurring pytorch transformer
Language:Python 1723
SysCV / transfiner
Mask Transfiner for High-Quality Instance Segmentation, CVPR 2022
cvpr cvpr2022 efficient-transformers high-quality instance-segmentation transformer
Language:Python 528
lucidrains / FLASH-pytorch
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"
artificial-intelligence attention-mechanism deep-learning efficient-transformers transformers
Language:Python 345
NVIDIA / transformer-ls
Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
efficient-transformers long-sequence transformer vision-transformer
Language:Python 221
feixue94 / imp-release
[CVPR 2023] IMP: iterative matching and pose estimation with transformer-based recurrent module
efficient-transformers pose-estimation superglue adaptive-pooling training-graph-matcher fast-superglue geometry-aware-matching
Language:Python 138
xmindflow / DAEFormer
[MICCAI 2023] DAE-Former: Dual Attention-guided Efficient Transformer for Medical Image Segmentation
efficient-transformers medical-image-analysis segmentation transformer medical-image-segmentation unet-image-segmentation deep-learning pytorch
Language:Python 103
VITA-Group / SViTE
[NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang
dynamic-sparsity efficient-transformers model-compression pruning sparse-training token-slimming vision-transformers
Language:Python 90
ziplab / EcoFormer
[NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"
classification efficient-transformers neurips-2022 pytorch vision-transformer
Language:Python 66
szq0214 / SReT
Official PyTorch implementation of our ECCV 2022 paper "Sliced Recursive Transformer"
vit transformer-architecture vision-transformer efficient-transformers efficient-neural-networks
Language:Python 62
VITA-Group / UVC
[ICLR 2022] "Unified Vision Transformer Compression" by Shixing Yu*, Tianlong Chen*, Jiayi Shen, Huan Yuan, Jianchao Tan, Sen Yang, Ji Liu, Zhangyang Wang
vision-transformer compression efficient-transformers admm
Language:Python 45
MarkusSagen / Master-Thesis-Multilingual-Longformer
Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).
longformer-models multilingual-models roberta xlm huggingface huggingface-transformers efficient-transformers longformer multilingual language-models
Language:Jupyter Notebook 32
MCG-NJU / EVAD
[ICCV 2023] Efficient Video Action Detection with Token Dropout and Context Refinement
action-detection action-recognition efficient-transformers pytorch transformer
Language:Python 24
Lemon-cmd / energy-transformer-graph
This repository contains the official code for Energy Transformer---an efficient Energy-based Transformer variant for graph classification
associative-memory deep-learning energy-based-model energy-transformer graph-neural-networks hopfield jax neural-network transformer efficient-transformers
Language:Python 18
Lemon-cmd / energy-transformer-torch
Official Implementation of Energy Transformer in PyTorch for Mask Image Reconstruction
assocative-memory efficient-transformers energy-transformer hopfield mask-image pytorch
Language:Python 18
keshavbhandari / Electra-With-Compositional-Embeddings
A custom Tensorflow implementation of Google's Electra NLP model with compositional embeddings using complementary partitions
electra efficient-transformers bert compositional-embeddings deep-neural-networks pretrained-models
Language:Jupyter Notebook 13
lim142857 / Sparsifiner
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
efficient-transformers efficient-vision-transformers fast-inference sparse-attention attention-mechanism low-rank sparse-neural-networks vision-transformer
Language:Python 11
MAGICS-LAB / NonparametricHopfield
Nonparametric Modern Hopfield Models
efficient-attention efficient-transformers modern-hopfield-model modern-hopfield-networks efficient-hopfield-models efficient-hopfield-networks
Language:Jupyter Notebook 7
shsarv / ChatBot
This is the source code of article how to create a chatbot in python . i.e A chatbot using the Reformer, also known as the efficient Transformer, to generate dialogues between two bots.
chat-bot efficient-transformers nlp nltk-python chatbots-framework chatbot-notebook source-code-study
Language:Jupyter Notebook 7
HySonLab / SAMSA
SAMSA: Efficient Transformer for Many Data Modalities
deep-learning differentiable-computing efficient-transformers multimodal
Language:Python 6
yubin1219 / MetaSeg
MetaFormer-Based Global Contexts-Aware Network for Efficient Semantic Segmentation (Accepted by WACV 2024)
efficient-transformers segmentation pytorch
4
brandnewchoppa / gau-tensorflow
Gated Attention Unit (TensorFlow implementation)
tensorflow2 arxiv-papers attention-mechanism efficient-transformers gated-attention deep-learning tensorflow transformer
Language:Python 1
ustcml / LISA
Pytorch implementation of LISA (Linear-Time Self Attention with Codeword Histogram for Efficient Recommendation. WWW 2021)
efficient-recommendation efficient-transformers product-quantization self-attention
Language:Python 0

efficient-transformers

swz30 / Restormer

SysCV / transfiner

lucidrains / FLASH-pytorch

NVIDIA / transformer-ls

feixue94 / imp-release

xmindflow / DAEFormer

VITA-Group / SViTE

ziplab / EcoFormer

szq0214 / SReT

VITA-Group / UVC

MarkusSagen / Master-Thesis-Multilingual-Longformer

MCG-NJU / EVAD

Lemon-cmd / energy-transformer-graph

Lemon-cmd / energy-transformer-torch

keshavbhandari / Electra-With-Compositional-Embeddings

lim142857 / Sparsifiner

MAGICS-LAB / NonparametricHopfield

shsarv / ChatBot

HySonLab / SAMSA

yubin1219 / MetaSeg

brandnewchoppa / gau-tensorflow

ustcml / LISA