Giters

SHI-Labs / Neighborhood-Attention-Transformer

Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Any plans to release the Neighborhood Attention code using only Pytorch?

qwopqwop200 opened this issue · comments

commented

Hi. Thank you for your good work.
I implemented Neighborhood Attention using only pytorch. And as far as I know, Neighborhood Attention was implemented using only Pytorch at the beginning of this work. Personally, I would like to compare the implemented code with the original code.
Do you have any plans to release a Neighborhood Attention implementation using only Pytorch?

commented

Hello and thank you for your interest.
We do plan to release the version based on vanilla PyTorch modules in the near future, due to the increasing number of requests from the community. (We were planning on releasing a CPU version of the extension, but we'll release the PyTorch version first.)
Please stay tuned for more.
I'd also encourage moving the discussion to #10 to prevent duplicates.

commented

We released the pure PyTorch version (PR #23 ) along with a new version of the kernel.
I'd like to thank you for your implementation of our RPB, which was used and referenced instead of our own in the PyTorch version.