desire2020 / pytorch_memory_efficient_attention_with_learnable_bias

Implements Memory Efficient Attention in plain pytorch. Added support for (learnable!) biases.

Pytorch Memory Efficient Attention with Learnable Attention bias

Implements Memory Efficient Attention in plain pytorch. Added support for (learnable!) biases. Warning: The introduction of learnable biases could slightly compromise the space complexity improvements of the original memory efficient attention algorithm.