wxbbuaa2011 / Edge-MoE

Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts

Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts

Rishov Sarkar¹, Hanxue Liang², Zhiwen Fan², Zhangyang Wang², Cong Hao¹

¹School of Electrical and Computer Engineering, Georgia Institute of Technology
²School of Electrical and Computer Engineering, University of Texas at Austin

Overview

This is Edge-MoE, the first end-to-end FPGA accelerator for multi-task ViT with a rich collection of architectural innovations.

Currently, this repository contains a prebuilt bitstream for the AMD/Xilinx ZCU102 FPGA and a video demo. Our HLS code will be open-source upon acceptance.