facebookresearch / xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

https://facebookresearch.github.io/xformers/

facebookresearch/xformers Issues

Request for tutorial on how to modify an attention processor into its xformers version
Updated 2 days ago
Builds for Pytorch 2.4 (CUDA 12.1)?
Closed 3 days ago2
Autocast does not work with xformers
Updated 5 days ago1
How to build from source if I have different architcure cards?
Closed 5 days ago1
suitable for L40
Updated 8 days ago
importing `xformers.ops` implicitly initializes CUDA context
Updated 11 days ago3
ERROR: Could not build wheels for xformers, which is required to install pyproject.toml-based projects
Updated 12 days ago1
memory_efficient_attention fw produce inconsistent results
Updated 16 days ago1
Built wheel for torch 2.3.0
Closed 18 days ago3
AttributeError: 'str' object has no attribute 'impl'
Updated 19 days ago3
attention's flop calculation when casual is set to True.
Updated 19 days ago2
Poor performance of sequence parallel fused kernels in real model training
Updated 21 days ago4
Masked MLP
Updated 21 days ago
Bucketing strategy in triton kernels of sequence parallel fused operators.
Updated 22 days ago
Xformers are slowing down on 3090/4090
Updated 24 days ago1
Compiling took more than 100GB memory in 4090
Updated a month ago
[ROCm/xformers] error generated when compiling for gfx1100
Updated a month ago8
`flshattF@v2.3.6` is not supported because: max(query.shape[-1] != value.shape[-1]) > 256
Closed a month ago2
BlockDiagonalAttention computes NaN gradients, when using bfloat16 and deterministic torch
Updated a month ago
output from memory_efficient_attention not exactly the same with pytorch equivalent implementation
Updated a month ago
[Bug] benchmark shows 0.00MB consumed
Updated a month ago2
efficient_attention_forward_cutlass op is incompatible with Torch JIT
Updated a month ago2
Implementation ideas for equivalent replacement from xformers to pytorch
Updated a month ago3
swiglu_op _Stack_or_None RunTimeError
Updated a month ago
Install xformers without changing pytorch version
Updated a month ago2
[feat] add python3.12 support
Updated a month ago
How to use Relative Position Bias with xformers
Updated a month ago
Built wheel for torch 2.2.2
Closed a month ago2
XFormer AssertionError: This attention does not support attention masks
Updated 2 months ago1
Xformers can not perform memory_efficient_attention
Updated 2 months ago
Old version of Xformers fit for lower torch
Updated 2 months ago
MicroGPT example uses encoders
Updated 2 months ago
fix link in examples README.md
Updated 2 months ago
RuntimeError: CUDA error in running xformers with attention_mask arg
Updated 2 months ago7
Cant install xformers
Updated 2 months ago2
100 errors detected in the compilation of "/tmp/tmpxft_0000e855_00000000-6_attention_backward_generic.cpp1.ii"
Updated 2 months ago1
Conditionalize sp24.py for windows support of cusparselt.py
Updated 2 months ago6
Library raises `NameError` if _has_cpp_library is False
Updated 2 months ago6
Tests fail: -c: error: the following arguments are required: example_exe
Updated 2 months ago
Cross-attention
Closed 2 months ago
Splitting an BlockCausalDiagonalMask on query dimension
Closed 2 months ago5
Built wheel for torch 2.2.1
Closed 2 months ago1
Support Taylor Series Linear Attention
Updated 2 months ago
[Bug] subprocess-exited-with-error problem installing xformers package on python3.12
Updated 2 months ago4
pip install from a specific branch
Updated 2 months ago
Customization of BlockDiagonalMask or BlockDiagonalCausalMask
Updated 2 months ago2
Does xformer require CUDA or can it also be run on TPU/Tranium instances?
Closed 2 months ago1
Error during loss backward
Updated 2 months ago
Is there a way of masking some noisy vectors from the sequence, different at each sequence?
Updated 3 months ago
2.0.1 version
Updated 3 months ago