anton-jeran

ears_dataset

Expressive Anechoic Recordings of Speech (EARS)

see2sound

Official code for SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial Sound

MESH2IR

This is the official implementation of our mesh-based neural network (MESH2IR) to generate acoustic impulse responses (IRs) for indoor 3D scenes represented using a mesh.

FAST-RIR

This is the official implementation of our neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating room impulse responses (RIRs) for a given acoustic environment.

EyefulTower

Official release of the Eyeful Tower dataset, a high-fidelity multi-view capture of 11 real-world scenes, from the paper “VR-NeRF High-Fidelity Virtualized Walkable Spaces” (Xu et al., SIGGRAPH Asia 2023).

L2S

This is the official implementation of our end-to-end binaural audio rendering approach (Listen2Scene) for virtual reality (VR) and augmented reality (AR) applications.

diffroomacoustics

A Differentiable Room Acoustics Simulator

FastDiff

PyTorch Implementation of FastDiff (IJCAI'22)

lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

real-acoustic-fields

Real Acoustic Fields An Audio-Visual Room Acoustics Dataset and Benchmark

AVRIR

whisperX

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

languagecodec_tmp

Temporary anonymous version

jepa

PyTorch code and models for V-JEPA self-supervised learning from video.

LRV-Instruction

[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

fcn.berkeleyvision.org

Fully Convolutional Networks for Semantic Segmentation by Jonathan Long*, Evan Shelhamer*, and Trevor Darrell. CVPR 2015 and PAMI 2016.

Video-LLaMA

[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

stable-audio-tools

Generative models for conditional audio generation

Catch-A-Waveform

Official pytorch implementation of the paper: "Catch-A-Waveform: Learning to Generate Audio from a Single Short Example" (NeurIPS 2021)

awesome-vision-language-pretraining-papers

Recent Advances in Vision and Language PreTrained Models (VL-PTMs)

audioseal

Localized watermarking for AI-generated speech audios, with SOTA on robustness and very fast detector

Fast3DScattering-release

Repo for our research paper "Learning Acoustic Scattering Fields for Dynamic Interactive Sound Propagation"

audio2photoreal

Code and dataset for photorealistic Codec Avatars driven from audio

ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

M2IR

S2IR

Listen2Scene_Code

anton-jeran

M2SYN

MAD