rhinojosa / efficientvit

EfficientViT is a new family of vision models for efficient high-resolution vision.

EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction (paper, poster)

News

If you are interested in getting updates, please join our mailing list here.

• [2024/02/07] We released EfficientViT-SAM, the first accelerated SAM model that matches/outperforms SAM-ViT-H's zero-shot performance, delivering the SOTA performance-efficiency trade-off.
• [2023/11/20] EfficientViT is available in the NVIDIA Jetson Generative AI Lab.
• [2023/11/20] We will soon release the second version of EfficientViT SAM models.
• [2023/10/22] ImageNet training scripts for the EfficientViT L series have been released.
• [2023/09/18] EfficientViT for Segment Anything Model (SAM) is released. EfficientViT SAM runs at 1009 images/s on A100 GPU, compared to ViT-H (12 images/s), mobileSAM (297 images/s), and nanoSAM (744 image/s, but much lower mIoU)
• [2023/09/12] EfficientViT is highlighted by MIT home page and MIT News.
• [2023/07/18] EfficientViT is accepted by ICCV 2023.

About EfficientViT Models

EfficientViT is a new family of ViT models for efficient high-resolution dense prediction vision tasks. The core building block of EfficientViT is a lightweight, multi-scale linear attention module that achieves global receptive field and multi-scale learning with only hardware-efficient operations, making EfficientViT TensorRT-friendly and suitable for GPU deployment.

Third-Party Implementation/Integration

• NVIDIA Jetson Generative AI Lab
• timm: link
• X-AnyLabeling: link

Getting Started

conda create -n efficientvit python=3.10
conda activate efficientvit
conda install -c conda-forge mpi4py openmpi
pip install -r requirements.txt

EfficientViT Applications

Segment Anything

Image Classification

Semantic Segmentation

Contact

Han Cai: [e-mail]

TODO

• ImageNet Pretrained models
• Segmentation Pretrained models
• ImageNet training code
• EfficientViT L series, designed for cloud
• EfficientViT for segment anything
• EfficientViT for image generation
• EfficientViT for super-resolution
• Segmentation training code

Citation

If EfficientViT is useful or relevant to your research, please kindly recognize our contributions by citing our paper:

@article{cai2022efficientvit,
  title={Efficientvit: Enhanced linear attention for high-resolution low-computation visual recognition},
  author={Cai, Han and Gan, Chuang and Han, Song},
  journal={arXiv preprint arXiv:2205.14756},
  year={2022}
}