xrdaukar

xrdaukar

User data from Github https://github.com/xrdaukar

GitHub:@xrdaukar

xrdaukar's repositories

ALCF_Hands_on_HPC_Workshop

The ALCF hosts a regular simulation, data, and learning workshop to help users scale their applications. This repository contains the examples used in the workshop.

Language:HTMLStargazers:1Issues:0Issues:0

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Language:PythonLicense:NOASSERTIONStargazers:1Issues:0Issues:0

mlx-examples

Examples in the MLX framework

Language:PythonLicense:MITStargazers:1Issues:0Issues:0
Language:PythonLicense:Apache-2.0Stargazers:1Issues:0Issues:0

ring-flash-attention

Ring attention implementation with flash attention

Language:PythonLicense:MITStargazers:1Issues:0Issues:0

RustOrBust

My Rust deep dive down the rabbit-hole !

Language:RustStargazers:1Issues:0Issues:0

tree_attention

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters

Language:PythonStargazers:1Issues:0Issues:0

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

Language:PythonLicense:Apache-2.0Stargazers:1Issues:0Issues:0

aminrezae-memory-efficient-attention

Memory Efficient Attention (O(sqrt(n)) for Jax and PyTorch

Language:PythonLicense:MITStargazers:0Issues:0Issues:0

attention-gym

Helpful tools and examples for working with flex-attention

Language:PythonLicense:BSD-3-ClauseStargazers:0Issues:0Issues:0

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

Language:PythonLicense:Apache-2.0Stargazers:0Issues:0Issues:0

context-parallelism

Context Parallelism, support Blockwise Attention, Ring Attention and Tree Attention.

Language:Jupyter NotebookStargazers:0Issues:0Issues:0

EasyContext

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Language:PythonLicense:Apache-2.0Stargazers:0Issues:0Issues:0

LightSeq

Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers

Language:PythonStargazers:0Issues:0Issues:0

oyEasyContext

modified on origin repo

Language:PythonLicense:Apache-2.0Stargazers:0Issues:0Issues:0

LLaVA-MORE

LLaVA-MORE: Enhancing Visual Instruction Tuning with LLaMA 3.1

License:Apache-2.0Stargazers:0Issues:0Issues:0

pytorch-memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

License:MITStargazers:0Issues:0Issues:0

ringattention

Transformers with Arbitrarily Large Context

License:Apache-2.0Stargazers:0Issues:0Issues:0

smol-vision

Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜

License:Apache-2.0Stargazers:0Issues:0Issues:0