positional-encoding

There are 4 repositories under positional-encoding topic.

lucidrains / rotary-embedding-torch
Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
artificial-intelligence deep-learning positional-encoding
Language:Python 774
therealoliver / Deepdive-llama3-from-scratch
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.
inference kv-cache llama llms attention attention-mechanism gpt language-model llm-configuration mask multi-head-attention positional-encoding residuals rms rms-norm rope swiglu tokenizer transformer rotary-position-encoding
Language:Jupyter Notebook 609
liruilong940607 / prope
Cameras as Relative Positional Encoding
multi-view positional-encoding transformer
Language:Python 607
yiqun-wang / PET-NeuS
PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces (CVPR 2023)
cvpr2023 implicit-surfaces multi-view-reconstruction nerf surface-reconstruction positional-encoding tri-planes pytorch
Language:Python 272
gnn-lspe
vijaydwivedi75 / gnn-lspe
Source code for GNN-LSPE (Graph Neural Networks with Learnable Structural and Positional Representations), ICLR 2022
graph-neural-networks transformers graph-transformer positional-encoding graph-deep-learning transformer-networks attention graphs molecules gnn-lspe lspe graph-representation-learning geometric-deep-learning representation-learning gnn message-passing
Language:Python 265
universome / inr-gan
[CVPR 2021] Adversarial Generation of Continuous Images
inr generative-model gan cvpr2021 siren positional-encoding
Language:Python 233
ziplab / FASeg
[CVPR 2023] This is the official PyTorch implementation for "Dynamic Focus-aware Positional Queries for Semantic Segmentation".
detr positional-encoding semantic-segmentation
Language:Python 60
willGuimont / learnable_fourier_positional_encoding
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
attention-is-all-you-need deep-learning deep-neural-networks deeplearning positional-encoding pytorch transformer
Language:Python 56
gcambara / cape
Continuous Augmented Positional Embeddings (CAPE) implementation for PyTorch
positional-encoding positional-encoder positional-embedding transformer cape pytorch audio speech text vit visual-transformer
Language:Python 42
osiriszjq / complex_encoding
Trading Positional Complexity vs Deepness in Coordinate Networks
deep-learning positional-encoding
Language:Jupyter Notebook 31
VITA-Group / Ms-PoE
"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.
large-language-models long-context lost-in-the-middle positional-encoding
Language:Python 30
AliAmini93 / ViViT-Medical-Video-Classification
Developed the ViViT model for medical video classification, enhancing 3D organ image analysis using transformer-based architectures.
video-classification vivit positional-encoding tubelet-transformer medmnist medical-imaging
Language:Jupyter Notebook 28
HySonLab / Multires-Graph-Transformer
Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Long-Range and Hierarchical Structures
density-functional-theory graph-neural-networks graph-transformer positional-encoding wavelet-transform
Language:Python 23
AlgonetLabs / Cable
Context-aware Biases for Length Extrapolation
auto-regressive-model gpt large-language-models length-extrapolation positional-encoding llm alibi cable context-aware rope
Language:Python 22
imics-lab / positional-encoding-benchmark
This repository offers a comprehensive overview and quantitative benchmarking of positional encoding methods in transformer-based time series models.
positional-encoding time-series transformer
Language:Jupyter Notebook 21
tanjeffreyz / attention-is-all-you-need
PyTorch implementation of "Attention Is All You Need" by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
attention-mechanism bleu-score nlp positional-encoding pytorch transformer machine-translation
Language:Python 18
konstantinosKokos / ape
🧮 Algebraic Positional Encodings.
positional-encoding pytorch transformer neurips-2024 rope neurosymbolic
Language:Python 16
liaoyanqing666 / transformer_pytorch
完整的原版transformer程序，complete origin transformer program
beginner multi-head-attention positional-encoding python pytorch transformer
Language:Python 15
deep-div / Custom-Transformer-Pytorch
A clean, ground-up implementation of the Transformer architecture in PyTorch, including positional encoding, multi-head attention, encoder-decoder layers, and masking. Great for learning or building upon the core model.
attention-mechanism deep-learning encoder-decoder machine-learning nlp positional-encoding pytorch self-attention sequence-to-sequence transformers transformers-architecture
Language:Jupyter Notebook 14
VITA-Group / TAPE
[ICML'25] "Rethinking Addressing in Language Models via Contextualized Equivariant Positional Encoding" by Jiajun Zhu, Peihao Wang, Ruisi Cai, Jason D. Lee, Pan Li, Zhangyang Wang
equivariant-neural-networks large-language-models positional-encoding
Language:Python 14
JHLew / Learnable-Fourier-Features
Unofficial pytorch implementation of the paper "Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding", NeurIPS 2021.
fourier-features neurips-2021 positional-encoding
Language:Python 13
AryaAftab / rotary-embedding-tensorflow
Implementation of Rotary Embeddings, from the Roformer paper, in Tensorflow
deeplearning positional-encoding tensorflow2
Language:Python 12
osiriszjq / RobustPPE
Robust Point Cloud Processing through Positional Embedding
positional-encoding point-clouds point-transformer 3dv2024
Language:HTML 11
bkhanal-11 / transformers
The implementation of transformer as presented in the paper "Attention is all you need" from scratch.
attention-is-all-you-need attention-mechanism multihead-attention positional-encoding self-attention transformers
Language:Python 9
Breeze648 / Transformer-from-Scratch
本仓库定位为 AI论文复现 / 从零实现 Transformer。代码遵循原论文的模块划分，包含位置编码、多头注意力、前馈网络、编码器‑解码器等全部组件，并附带详细的中文拆解文档与英文注释，方便学习与二次开发。
ai attention-mechanism deep-learning encoder-decoder large-language-model natural-language-processing positional-encoding transformer
Language:Python 7
ETH-DISCO / Benchmarking-PEs
Benchmarking Positional Encodings for GNNs and Graph Transformers
expressiveness gnn graph-neural-networks graph-transformer lrgb positional-encoding gnn-benchmark
Language:Python 7
harveybc / feature-extractor
Application for training an autoencoder for generating an encoder that can be used as feature extractor for dimensionality and noise reduction, while the decoder can be used for synthetic data generation. Supports dynamic plugin integration, allowing users to extend its capabilities by adding custom encoder and decoder models.
ann attention-mechanism autoencoder cnn cnn-keras dimensionality-reduction keras lstm machine-learning plugin-system positional-encoding pretrained-models pretrained-weights pretraining reusable-components synthetic-data tensorflow transformers vae
Language:Python 7
PKU-ML / LaplacianCanonization
Official code for NeurIPS 2023 paper "Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding".
gnn laplacian-eigenmaps positional-encoding spectral-embedding
Language:Python 7
Aalto-QuML / PIPE
Positional Encoding meets Persistent Homology on Graphs
gnns graph persistent-homology-graphs positional-encoding representation-learning
Language:Python 5
Ave-Sergeev / Tictonix
Crate for `Embedings` and `Positional Encoding` (Rust) (Q2:2025)
embeddings positional-encoding rust math deep-learning machine-learning
Language:Rust 4
xmarva / transformer-architectures
Teaching transformer-based architectures
attention-mechanism bpe-tokenizer nlp positional-encoding transformer weights-and-biases
Language:Jupyter Notebook 4
linhthi / alpha-SPE
Codebase of paper "Balancing structure and position information in Graph Transformer network with a learnable node embedding"
graph-neural-networks graph-transformer positional-encoding
Language:Python 3
tigerchen52 / locality_symmetry
Code for "The Locality and Symmetry of Positional Encodings" EMNLP Findings
positional-encoding probing-task transformer-architecture
Language:Python 3
areeba0 / English-to-French-Translation-using-NLTK-and-Hugging-Face-Transformers-MarianMTModel
This repository provides a complete workflow for text processing using Hugging Face Transformers and NLTK. It includes modules for sentence normalization, spelling correction, word embedding generation, positional encoding computation, and English-to-French translation
huggingface-transformers jupyter-notebook nlp nltk positional-encoding python text-normalization word-embeddings english-to-french-translation
Language:Jupyter Notebook 2
GeorgeMLP / laplacian-canonization
Official code for NeurIPS 2023 paper "Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding".
gnn laplacian-eigenmaps positional-encoding spectral-embedding
Language:Python 2
rhoadesScholar / RoSE
PyTorch implementation of Rotary Spatial Embeddings
attention positional-encoding rotary-position-embedding rotary-position-encoding pytorch transformer
Language:Python 2

positional-encoding

lucidrains / rotary-embedding-torch

therealoliver / Deepdive-llama3-from-scratch

liruilong940607 / prope

yiqun-wang / PET-NeuS

vijaydwivedi75 / gnn-lspe

universome / inr-gan

ziplab / FASeg

willGuimont / learnable_fourier_positional_encoding

gcambara / cape

osiriszjq / complex_encoding

VITA-Group / Ms-PoE

AliAmini93 / ViViT-Medical-Video-Classification

HySonLab / Multires-Graph-Transformer

AlgonetLabs / Cable

imics-lab / positional-encoding-benchmark

tanjeffreyz / attention-is-all-you-need

konstantinosKokos / ape

liaoyanqing666 / transformer_pytorch

deep-div / Custom-Transformer-Pytorch

VITA-Group / TAPE

JHLew / Learnable-Fourier-Features

AryaAftab / rotary-embedding-tensorflow

osiriszjq / RobustPPE

bkhanal-11 / transformers

Breeze648 / Transformer-from-Scratch

ETH-DISCO / Benchmarking-PEs

harveybc / feature-extractor

PKU-ML / LaplacianCanonization

Aalto-QuML / PIPE

Ave-Sergeev / Tictonix

xmarva / transformer-architectures

linhthi / alpha-SPE

tigerchen52 / locality_symmetry

areeba0 / English-to-French-Translation-using-NLTK-and-Hugging-Face-Transformers-MarianMTModel

GeorgeMLP / laplacian-canonization

rhoadesScholar / RoSE