audio-visual

There are 5 repositories under audio-visual topic.

krantiparida / awesome-audio-visual
A curated list of different papers and datasets in various areas of audio-visual processing
awesome audio-visual cross-modal mutli-modal localization source-separation awesome-list
621
TaoRuijie / TalkNet-ASD
ACM MM 2021: 'Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection'
active-speaker-detection audio-visual multimedia awesome-asd
Language:Python 265
ekazakos / temporal-binding-network
Implementation of "EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition, ICCV, 2019" in PyTorch
action-recognition audio-visual convolutional-networks egocentric fusion
Language:Python 104
guyyariv / TempoTokens
This repo contains the official PyTorch implementation of: Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation
ai-art audio-to-video audio-visual deep-learning diffusion-models generative-ai video-synthesis modelscope pytorch
Language:Python 89
libvisual
Libvisual / libvisual
Libvisual Audio Visualization
alsa audio-library audio-visualizer c cxx library music-visualization music-visualizer pulseaudio sdl libvisual audio-visual audio-visualization spectrum-analyser spectrum-analysis spectrum-analyzer opengl portaudio visualization
Language:C 81
ankurbhatia24 / MULTIMODAL-EMOTION-RECOGNITION
Human Emotion Understanding using multimodal dataset.
deeplearning deep-learning machine-learning keras audio-visual audio-visualization audio-visual-speech-recognition python tensorflow multimodal-emotion-recognition librosa opensmile
Language:Jupyter Notebook 75
waveform-path
jerosoler / waveform-path
🎙 Generator waveform paths for SVG 🎶
audio audio-player audio-visual audio-visualizer music path svg waveform webaudio
Language:JavaScript 75
samhirtarif / react-audio-visualize
An audio visualizer for React. Provides separate components to visualize both live audio and audio blobs.
audio audio-visual audio-visualization audio-visualizer audio-waveform audio-waveforms-visualization
Language:TypeScript 54
v-iashin / SparseSync
Source code for "Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors." (Spotlight at the BMVC 2022)
audio-visual bmvc lrs pytorch synchronization transformer vggsound multi-modal sparse
Language:Python 45
satelllte / remotion-audio-visualizer
Programmatic minimalistic audio visualizations.
audio audio-visual audio-visualizer dsp remotion video
Language:TypeScript 33
dialogtekgeek / AudioVisualSceneAwareDialog
audio-visual dialog dstc7 scene-aware-dialog
Language:Python 27
dialogtekgeek / AVSD-DSTC10_Official
Audio Visual Scene-Aware Dialog (AVSD) Challenge at the 10th Dialog System Technology Challenge (DSTC)
audio-visual dialog qa scene-aware video-description
27
MCG-NJU / JoMoLD
[ECCV 2022] Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing
audio-visual
Language:Python 26
georgesterpu / Taris
Transformer-based online speech recognition system with TensorFlow 2
online speech-recognition speech-recognizer audio-visual-speech-recognition audio-visual multimodal multimodal-deep-learning transformer tensorflow tensorflow2 python mahcine-learning deep-learning taris live-caption
Language:Python 25
joannahong / AV-RelScore
Audio-Visual Corruption Modeling of our paper "Watch or Listen: Robust Audio-Visual Speech Recognition with Visual Corruption Modeling and Reliability Scoring" in CVPR23
audio-visual avsr multimodal
Language:Python 24
MengyuanChen21 / CVPR2023-CMPAE
[CVPR 2023] Collecting Cross-Modal Presence-Absence Evidence for Weakly-Supervised Audio-Visual Event Perception
audio-visual audio-visual-learning audio-visual-video-parsing cvpr2023 video-understanding
Language:Python 24
Yu-Wu / Modaily-Aware-Audio-Visual-Video-Parsing
Code for CVPR 2021 paper Exploring Heterogeneous Clues for Weakly-Supervised Audio-Visual Video Parsing
cvpr2021 cvpr audio-visual
Language:Python 22
hmartelb / avlit
Official source code of the INTERSPEECH 2023 paper: "Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model" (AVLIT)
audio-visual iterative multi-modal pytorch pytorch-lightning speech-enhancement speech-separation lightweight
Language:Python 17
Overcautious / ADENet
Accepted by TMM 2022
active-speaker-detection audio-visual multimodel speech-enhancement
Language:Python 16
v-iashin / Synchformer
Efficient synchronization from sparse cues
audio-visual contrastive-learning multi-modal synchronization video-understanding
Language:Python 14
cogmhear / Intelligibility-Oriented-Audio-Visual-Speech-Enhancement
Towards Intelligibility-Oriented Audio-Visual Speech Enhancement
audio-visual-speech-separation audio-visual speech-enhancement speech-intelligibility
Language:Python 13
FannyChao / AVS360_audiovisual_saliency_360
Towards Audio-Visual Saliency Prediction for Omnidirectional Video with Spatial Audio
audio-visual saliency-prediction 360-video virtual-reality vr spatial-audio ambisonics
Language:Python 13
jinxiang-liu / anno-free-AVS
Official code for WACV 2024 paper, "Annotation-free Audio-Visual Segmentation"
audio-visual segmentation audio-visual-segmentation semantic-segmentation
Language:Python 13
SAGNIKMJR / move2hear-active-AV-separation
Code and datasets for 'Move2Hear: Active Audio-Visual Source Separation' (ICCV 2021)
reinforcement-learning active-learning audio-visual audio-separation multimodal-deep-learning
Language:Python 12
tutaru99 / Internet-Radio-Player-Vue
Internet Radio Player with an Audio Visualizer made using VueJS, Vuetify & Howler.JS frameworks. The Player has a bunch of radio stations. Check out the demo below.
howlerjs howler howlerjs-library howler-audio-library vuejs vue vue2 vuejs2 vuetify radio internet-radio internet-radio-player internet-radio-stations internet-radio-website vuetify-frontend vuetifyjs audio-visualizer audio-visualization audio-visual visualizer
Language:Vue 11
dkurzend / ClipClap-GZSL
Audio-Visual Generalized Zero-Shot Learning using Large Pre-Trained Models
clap clip generalized-zero-shot-learning gzsl zero-shot-learning zsl audio-visual-generalized-zero-shot-learning audio-visual audio-visual-learning learning
Language:Python 9
usc-sail / mica-multimodal-ads
Segment-level autoencoders for multimodal representation
multimodal-representation autoencoders advertisements audio-visual segment-level-autoencoders
Language:Python 9
OpenGVLab / perception_test_iccv2023
Champion Solutions repository for Perception Test challenges in ICCV2023 workshop.
audio-visual deep-learning iccv2023
Language:Python 8
tridivb / attention_based_tbn
Attention-based Temporal Binding Network
temporal-binding-network egocentric-action-recognition action-recognition epic-kitchens audio-visual fusion attention attention-model
Language:Python 8
magdalenafuentes / urbansas
Urban Sound & Sight dataset and baseline
audio-visual dataset sound-localization two-channel-audio deep-learning
Language:Jupyter Notebook 7
dedobbin / img_stripper
Library to convert image files to audio files and vice versa
audio wav jpg audio-visual steganography nonsense opencv
Language:C++ 6
SMIL-SPCRAS / DAVIS
Official repo for "Audio-Visual Speech Recognition In-the-Wild: Multi-Angle Vehicle Cabin Corpus and Attention-based Method" in ICASSP 2024
attention-mechanism audio-visual avsr corpus icassp icassp2024 in-the-wild multi-modal signal-processing spatio-temporal-features speech-recognition
Language:JavaScript 6
TIB-Digital-Preservation / FilmConservationMetadata
a standardized way to record and store the finding of an inspection of an analogue film in order to document the state at the moment of digitization
audio-visual preservation
6
markus-wa / av-clj
Audio Visual stuff in Clojure with Shadertone / GLSL
audio-visual shadertone overtone glsl clojure
Language:GLSL 5
bfidatadigipres / BFI_scripts
Respository for BFI National Archive open source preservation workflow scripts
archive audio-visual digital preservation
Language:Python 4
tutorial_audio_analysers
polygonjs / tutorial_audio_analysers
🎵 Tutorial showing how to use audio analysers to update a WebGL scene 🔊
audio audio-processing audio-visualizer audio-visual audio-analysis fft tonejs threejs webgl glsl particles procedural creative-coding animation 3d node-based
Language:JavaScript 4

audio-visual

krantiparida / awesome-audio-visual

TaoRuijie / TalkNet-ASD

ekazakos / temporal-binding-network

guyyariv / TempoTokens

Libvisual / libvisual

ankurbhatia24 / MULTIMODAL-EMOTION-RECOGNITION

jerosoler / waveform-path

samhirtarif / react-audio-visualize

v-iashin / SparseSync

satelllte / remotion-audio-visualizer

dialogtekgeek / AudioVisualSceneAwareDialog

dialogtekgeek / AVSD-DSTC10_Official

MCG-NJU / JoMoLD

georgesterpu / Taris

joannahong / AV-RelScore

MengyuanChen21 / CVPR2023-CMPAE

Yu-Wu / Modaily-Aware-Audio-Visual-Video-Parsing

hmartelb / avlit

Overcautious / ADENet

v-iashin / Synchformer

cogmhear / Intelligibility-Oriented-Audio-Visual-Speech-Enhancement

FannyChao / AVS360_audiovisual_saliency_360

jinxiang-liu / anno-free-AVS

SAGNIKMJR / move2hear-active-AV-separation

tutaru99 / Internet-Radio-Player-Vue

dkurzend / ClipClap-GZSL

usc-sail / mica-multimodal-ads

OpenGVLab / perception_test_iccv2023

tridivb / attention_based_tbn

magdalenafuentes / urbansas

dedobbin / img_stripper

SMIL-SPCRAS / DAVIS

TIB-Digital-Preservation / FilmConservationMetadata

markus-wa / av-clj

bfidatadigipres / BFI_scripts

polygonjs / tutorial_audio_analysers