antonyvigouret / Pay-Attention-to-MLPs

My implementation of the gMLP model from the paper "Pay Attention to MLPs".

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Pay-Attention-to-MLPs

Implementation of the gMLP model introduced in Pay Attention to MLPs.

The authors of the paper propose a simple attention-free network architecture, gMLP, based solely on MLPs with gating, and show that it can perform as well as Transformers in key language and vision applications.

About

My implementation of the gMLP model from the paper "Pay Attention to MLPs".

License:MIT License


Languages

Language:Python 100.0%