Giters

SHI-Labs / Neighborhood-Attention-Transformer

Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About the detail of NATTENQKRPBFunction and NATTENAVFunction

myt889 opened this issue · comments

commented

Hi, first congratulations on your great work! I am a newer to Transformer. I have successfully installed the NA version of cuda. But when I read your core code about NA, I can not understand the detailed codes of this two functions:NATTENQKRPBFunction and NATTENAVFunction. Could you explian ? Thank you so much.

commented

Hello and thank you for your interest.

Those are Autograd functions, which call the forward and backward pass CUDA kernels. If you like to dig deeper on those, you'd have to refer to the CUDA kernel.

commented

Hello and thank you for your interest.

Those are Autograd functions, which call the forward and backward pass CUDA kernels. If you like to dig deeper on those, you'd have to refer to the CUDA kernel.

I got it, thank you so much!