audio-visual-speech-recognition

There are 3 repositories under audio-visual-speech-recognition topic.

• modelscope / FunASR

  A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity Detection, Text Post-processing etc.

  conformerpytorchspeech-recognitionparaformerpunctuationspeaker-diarizationrnntaudio-visual-speech-recognitionpretrained-modelvoice-activity-detectionwhisperdfsmnvadspeechgptspeechllm
  Language:Python 12594

• smeetrs / deep_avsr

  A PyTorch implementation of the Deep Audio-Visual Speech Recognition paper.

  audio-visual-speech-recognitionautomatic-speech-recognitionlip-readingspeech-recognitionspeech-to-textvisual-speech-recognition
  Language:Python 227

• ankurbhatia24 / MULTIMODAL-EMOTION-RECOGNITION

  Human Emotion Understanding using multimodal dataset.

  audio-visualaudio-visual-speech-recognitionaudio-visualizationdeep-learningdeeplearningkeraslibrosamachine-learningmultimodal-emotion-recognitionopensmilepythontensorflow
  Language:Jupyter Notebook 96

• georgesterpu / Taris

  Transformer-based online speech recognition system with TensorFlow 2

  onlinespeech-recognitionspeech-recognizeraudio-visual-speech-recognitionaudio-visualmultimodalmultimodal-deep-learningtransformertensorflowtensorflow2pythonmahcine-learningdeep-learningtarislive-caption
  Language:Python 26

• Sreyan88 / LipGER

  Code for InterSpeech 2024 Paper: LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition

  audio-visualaudio-visual-speech-recognitiongenerative-aillmpromptingspeech-recognition
  Language:Python 16

• umbertocappellazzo / Llama-AVSR

  [ICASSP 2025] Official Pytorch implementation of "Large Language Models are Strong Audio-Visual Speech Recognition Learners".

  audio-visual-speech-recognitionlarge-language-modelsvisual-speech-recognition
  Language:Python 16

• aidayang / FunASR-OneClick

  FunASR实时语音识别版，识别麦克风和电脑内播放的声音，电脑语音打字软件

  audio-visual-speech-recognitionconformerdfsmnfunasrparaformerpretrained-modelspunctuationpytorchrnntspeaker-diarizationspeech-recognitionspeechgptspeechllmvadvoice-activity-detectionwhisper
  12

• david-gimeno / tailored-avsr

  Official source code for the paper "Tailored Design of Audio-Visual Speech Recognition Models using Branchformers"

  audio-visual-speech-recognitioninterpretabilitylipreadingparameter-efficientrobust-asrvisual-speech-recognition
  Language:Python 11

• lzuwei / end-to-end-multiview-lipreading

  End to End Multiview Lip Reading

  audio-visual-speech-recognitiondeep-learningend-to-end-learning
  Language:Python 10

• sungnyun / avsr-temporal-dynamics

  (SLT 2024) Learning Video Temporal Dynamics with Cross-Modal Attention for Robust Audio-Visual Speech Recognition

  audio-visual-speech-recognition
  Language:Python 9

• hmeutzner / kaldi-avsr

  Kaldi-based audio-visual speech recognition

  speech-recognitionkaldideep-neural-networksavsrasraudio-visual-speech-recognition
  Language:Shell 6

• karlsimsBBC / cassette-bot

  🤖 📼 Command-line tool for remixing videos with time-coded transcriptions.

  text-to-videoaudio-visual-speech-recognitionvideo
  Language:Python 5

• sungnyun / cav2vec

  (ICLR 2025) Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation

  audio-visual-speech-recognitionself-supervised-learningnoise-robustness
  Language:Python 4

• zulfiqar-ali01 / audio-visual-Transcription

  Real-Time Audio-visual Speech Recongition

  audio-processingaudio-visual-speech-recognitionrealtime-analytics
  Language:Python 4

• luomingshuang / lipreading_with_icefall

  In this repository, I try to use k2, icefall and Lhotse for lip reading. I will modify it for the lip reading task. Many different lip-reading datasets should be added. -_-

  lip-readingk2icefallvisual-speech-recognitionaudio-visual-speech-recognition
  Language:Python 2

• Remi-Gau / McGurk_prior_code

  Code related to the fMRI experiment on the contextual modulation of the McGurk Effect

  fmrifmri-data-analysismultisensory-integrationaudio-visual-speech-recognition
  Language:MATLAB 1

• MaazKhan98 / Multimodal-Emotion-Recognition-speech-facial-and-body-gestures

  Human Emotion Understanding using multimodal dataset

  audio-visual-speech-recognitiondeep-learningmachine-learningtensorflow
  Language:Jupyter Notebook 0