Giters

GuangxueWang / SLPapers

Papers about AI+ Sign language

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

LATEST Awesome AI Sign Language Papers & Popularization of Professional Knowledge

Keywords: Sign Language, Sign Language Translation (SLT), Sign Language Recognition (SLR), Sign Language Linguistics

AI Sign language Papers [UPDATING]

This repository is for those interested in the field of AI sign language (SL). The collected papers have been categorized according to different criteria (btime, type, institution, etc.) for ease of searching.

If useful, please Star this repo, we are keeping it updated.

NOTE: There is overlap between the different categories. Please feel free to submit your Pull Requests for any updates.

Popularization of Professional Knowledge

- Why AI Sign Language Reseaarch?


Figure 1. Communication gap between the hearing and the deaf.

Sign languages are the primary language of the deaf community. However, most hearing people find it difficult to understand sign languages. With the development of AI, researchers are trying to help people understand sign languages using AI techniques that are designed to convert sign languages into spoken languages in textual form.

- Do you know the diffence between SLT and SLR?


Figure 2. SLR (Sign Language Recognition) vs. SLT (Sign Language Translation).

At the very beggining, I wanna explain the difference between SLT and SLR, as shown in Fig. 2. I'm sure this is very important for most of you!

In early efforts, researchers explored this problem (sign languages -> texts) as a recognition problem (i.e., SLR), which converts sign languages to glosses word by word according to the sign languages sequentially. Although glosses are in textual form, they do not provide meaningful interpretations of what a signer is saying because sign languages and glosses have their own specific linguistic rules, which are quite different from spoken languages.

As a result, researchers find it terrible to ignore the linguistic properties of sign language. Contrary to SLR, sign language translaton (SLT) systems aim to translate sign language videos into spoken sentences directly.

As far as AI technology is concerned, SLR belongs to the field of Computer Vision + Text Recognition, while SLT belongs to the field of Computer Vision + Text Translation.

Table of Content - Papers

Timeline of AI Sign Language

2023

[Back to TOP]

  • CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. CVPR 2023; Highlight Paper [paper] [code] ==ZJU&XMU&THU; SLR; Latest==

2022

[Back to TOP]

  • 【CVPR 2022】A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. [paper] Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin
  • 【CVPR 2022】MLSLT: Towards Multilingual Sign Language Translation. [paper] Aoxiong Yin, Zhou Zhao, Weike Jin, Meng Zhang, Xingshan Zeng, Xiaofei He
  • 【CVPR 2022】C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. [paper] Ronglai Zuo, Brian Mak
  • 【NeurIPS 2022】Two-Stream Network for Sign Language Recognition and Translation. [paper] Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, Brian Mak
  • 【AAAI 2023】Self-Emphasizing Network for Continuous Sign Language Recognition. [paper] Lianyu Hu, Liqing Gao, Zekang liu, Wei Feng
  • 【EACL 2023】Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation. [paper] Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu
  • 【EMNLP 2022】Open-Domain Sign Language Translation Learned from Online Video. [paper] Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu
  • 【NAACL 2022 findings】Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation. [paper] Yong Cao, Wei Li, Xianzhi Li, Min Chen, Guangyong Chen, Long Hu, Zhengdao Li, Hwang Kai
  • 【ICASSP 2023】A Token-level Contrastive Framework for Sign Language Translation. [paper] Biao Fu, Peigen Ye, Liang Zhang, Pei Yu, Cong Hu, Yidong Chen, Xiaodong Shi
  • 【ACL 2022】WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language. [paper] Federico Tavella, Viktor Schlegel, Marta Romeo, Aphrodite Galata, Angelo Cangelosi
  • 【ACL 2022】Searching for fingerspelled content in American Sign Language. [paper] Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu
  • 【ECCV 2022】Automatic dense annotation of large-vocabulary sign language videos. [paper] Liliane Momeni, Hannah Bull, K R Prajwal, Samuel Albanie, Gül Varol, Andrew Zisserman
  • 【ECCV 2022】Temporal Lift Pooling for Continuous Sign Language Recognition. [paper] Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng
  • 【ACM MM 2022】MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation. [paper] Tao Jin, Zhou Zhao, Meng Zhang, Xingshan Zeng
  • 【WACV 2022】Sign Language Translation With Hierarchical Spatio-Temporal Graph Neural Network. [paper] Jichao Kan, Kun Hu, Markus Hagenbuchner, Ah Chung Tsoi, Mohammed Bennamoun, Zhiyong Wang

2021

[Back to TOP]

  • 【2021 TMM】Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation.[paper]
    Hao Zhou, Wengang Zhou, Yun Zhou, Houqiang Li
  • 【ICCV 2021】YouRefIt: Embodied Reference Understanding With Language and Gesture.[paper]
    Yixin Chen; Qing Li; Deqian Kong; Yik Lun Kei; Song-Chun Zhu; Tao Gao; Yixin Zhu; Siyuan Huang
  • 【ICCV 2021】Speech Drives Templates: Co-Speech Gesture Synthesis With Learned Templates.
    Shenhan Qian; Zhi Tu; Yihao Zhi; Wen Liu; Shenghua Gao
  • 【ICCV 2021】Audio2Gestures: Generating Diverse Gestures From Speech Audio With Conditional Variational Autoencoders.
    Jing Li; Di Kang; Wenjie Pei; Xuefei Zhe; Ying Zhang; Zhenyu He; Linchao Bao
  • 【ICCV 2021】Aligning Subtitles in Sign Language Videos.[paper]
    Hannah Bull; Triantafyllos Afouras; Gül Varol; Samuel Albanie; Liliane Momeni; Andrew Zisserman
  • 【ICCV 2021】Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives.[paper]
    Ben Saunders; Necati Cihan Camgoz; Richard Bowden
  • 【ICCV 2021】SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition. [paper] Hezhen Hu; Weichao Zhao; Wengang Zhou; Yuechen Wang; Houqiang Li
  • 【ICCV 2021】Visual Alignment Constraint for Continuous Sign Language Recognition.[paper] [code] Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen
  • 【ICCV 2021】Self-Mutual Distillation Learning for Continuous Sign Language Recognition. [paper]
    Aiming Hao, Yuecong Min, and Xilin Chen
  • 【CVPR 2021】Improving Sign Language Translation With Monolingual Data by Sign Back-Translation.[paper]
    Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, Houqiang Li
  • 【CVPR 2021】How2Sign: A Large-Scale Multimodal Dataset for Continuous American Sign Language. [paper]
    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, Xavier Giro-i-Nieto
  • 【CVPR 2021】Fingerspelling Detection in American Sign Language. [paper]
    Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu
  • 【CVPR 2021】 Read and Attend: Temporal Localisation in Sign Language Videos.[paper]
    Gül Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, Andrew Zisserman
  • 【CVPR 2021】iMiGUE: An Identity-Free Video Dataset for Micro-Gesture Understanding and Emotion Analysis.[paper]
    Xin Liu, Henglin Shi, Haoyu Chen, Zitong Yu, Xiaobai Li, Guoying Zhao
  • 【CVPR 2021】Body2Hands: Learning To Infer 3D Hands From Conversational Gesture Body Dynamics. [paper]
    Evonne Ng, Shiry Ginosar, Trevor Darrell, Hanbyul Joo
  • 【CVPR 2021】Model-Aware Gesture-to-Gesture Translation.[paper]
    Hezhen Hu, Weilun Wang, Wengang Zhou, Weichao Zhao, Houqiang Li
  • 【AAAI 2021】Hand-Model-Aware Sign Language Recognition.[paper]
    Hezhen Hu, Wengang Zhou, Houqiang Li
  • 【AAAI 2021】 Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition.[paper]
    Benjia Zhou, Yunan Li, Jun Wan
  • 【WACV 2021】 Hand Pose Guided 3D Pooling for Word-level Sign Language Recognition.[paper]
    Al Amin Hosain; Panneer Selvam Santhalingam; Parth Pathak; Huzefa Rangwala; Jana Kosecka
  • 【WACV 2021】Whose hand is this? Person Identification from Egocentric Hand Gestures.[paper] Satoshi Tsutsui; Yanwei Fu; David Crandall

2020

[Back to TOP]

  • 【2020 IJCV】Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks.[paper]
    Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden
  • 【ACM MM 2020】INCLUDE: A Large Scale Dataset for Indian Sign Language Recognition.[paper]
    Sridhar, Advaith, Rohith Gandhi Ganesan, Pratyush Kumar, and Mitesh Khapra
  • 【ACM MM 2020】Boosting Continuous Sign Language Recognition via Cross Modality Augmentation.[paper]
    Pu, Junfu, Wengang Zhou, Hezhen Hu, and Houqiang Li
  • 【ACM MM 2020】Recognizing Camera Wearer from Hand Gestures in Egocentric Videos.[paper]
    Thapar, Daksh, Aditya Nigam, and Chetan Arora
  • 【NIPS 2020】TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation. [paper]
    Li, Dongxu, Chenchen Xu, Xin Yu, Kaihao Zhang, Benjamin Swift, Hanna Suominen, and Hongdong Li
  • 【FG 2020】Feature Selection for Zero-Shot Gesture Recognition. [paper]
    Naveen Madapana, Juan Wachs
  • 【FG 2020】Image-Based Pose Representation for Action Recognition and Hand Gesture Recognition. [paper]
    Zeyi Lin, Wei Zhang, Xiaoming Deng, Cuixia Ma, Hongan Wang
  • 【FG 2020】Neural Sign Language Translation by Learning Tokenization. [paper]
    Orbay, Alptekin, and Lale Akarun
  • 【FG 2020】Sign Language Recognition in Virtual Reality. [paper]
    Jacob Schioppo, Zachary Meyer, Diego Fabiano, Shaun Canavan
  • 【FG 2020】SILFA: Sign Language Facial Action Database for the Development of Assistive Technologies for the Deaf.[paper]
    Emely Pujólli da Silva, Paula Dornhofer Paro Costa, Kate Mamhy Oliveira Kumada, José Mario De Martino
  • 【FG 2020】FineHand: Learning Hand Shapes for American Sign Language Recognition. [paper]
    Al Amin Hosain, Panneer Selvam Santhalingam, Parth Pathak, Huzefa Rangwala, Jana Košecká
  • 【FG 2020】Introduction and Analysis of an Event-Based Sign Language Dataset [paper]
    Ajay Vasudevan, Pablo Negri, Bernabe Linares-Barranco, Teresa Serrano-Gotarredona
  • 【FG 2020】Towards a Visual Sign Language Dataset for Home Care Services. [paper]
    D. Kosmopoulos, I. Oikonomidis, C. Constantinopoulos, N. Arvanitis, K. Antzakas, A. Bifis, G. Lydakis, A. Roussos, A. Argyros
  • 【ECCV 2020】 SLRTP 2020 Sign language recognition, translation & production. [Accepted papers]
  • 【ECCV 2020】BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues.[paper]
    Samuel Albanie, Gül Varol, Liliane Momeni, Triantafyllos Afouras, Joon Son Chung, Neil Fox, Andrew Zisserman
  • 【ECCV 2020】Progressive Transformers for End-to-End Sign Language Production.[paper] [code]
    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden
  • 【ECCV 2020】Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition.[paper]
    Niu Zhe, Brian Mak
  • 【ECCV 2020】Fully Convolutional Networks for Continuous Sign Language Recognition.[paper]
    Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, and Yu-Wing Tai
  • 【ECCV 2020】 Collaborative Learning of Gesture Recognition and 3D Hand Pose Estimation with Multi-Order Feature Analysis.[paper]
    Yang, Siyuan, Jun Liu, Shijian Lu, Meng Hwa Er, and Alex C. Kot.
  • 【ECCV 2020】Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach. [paper]
    Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano, and Louis-Philippe Morency
  • 【ECCV 2020】Towards Efficient Coarse-to-Fine Networks for Action and Gesture Recognition.[paper]
    Quader, Niamul, Juwei Lu, Peng Dai, and Wei Li
  • 【CVPR 2020】Decoupled Representation Learning for Skeleton-Based Gesture Recognition.[paper]
    Jianbo Liu, Yongcheng Liu, Ying Wang, Véronique Prinet, Shiming Xiang, Chunhong Pan
  • 【CVPR 2020】An Efficient PointLSTM for Point Clouds Based Gesture Recognition.[paper]
    Yuecong Min, Yanxiao Zhang, Xiujuan Chai, Xilin Chen
  • 【CVPR 2020】Music Gesture for Visual Sound Separation. [paper]
    Chuang Gan, Deng Huang, Hang Zhao, Joshua B. Tenenbaum, Antonio Torralba
  • 【CVPR 2020】Transferring Cross-Domain Knowledge for Video Sign Language Recognition.[paper]
    Dongxu Li, Xin Yu, Chenchen Xu, Lars Petersson, Hongdong Li
  • 【CVPR 2020] Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation.[paper]
    Necati Cihan Camgöz, Oscar Koller, Simon Hadfield, Richard Bowden
  • 【AAAI 2020】Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. [paper]
    Hao Zhou, Wengang Zhou, Yun Zhou, Houqiang Li
  • 【WACV 2020】Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. [paper]
    Dongxu Li, Cristian Rodriguez, Xin Yu, Hongdong Li
  • 【WACV 2020】Neural Sign Language Synthesis: Words Are Our Glosses. [paper]
    Jan Zelinka, Jakub Kanis

2019

[Back to TOP]

2018

[Back to TOP]

Earlier

[Back to TOP]

AI Sign Language in Types

SLT

[Back to TOP]

  1. Multi-channel Transformers for Multi-articulatory Sign Language Translation arxiv2020 paper code
  2. Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation CVPR2020 paper code ==德国==
  3. Sign Language Translation with Transformers ArXiv2020 paper code
  4. Neural Sign Language Translation by Learning Tokenization Arxiv2020 paper code
  5. Neural Sign Language Translation based on Human Keypoint Estimation Arxiv2018 paper code
  6. Neural Sign Language Translation CVPR2018 paper code

SLR

[Back to TOP]

CTC-Iteration for Alignment

  • mainly CNN+RNN*
  1. Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition ECCV2020 paper code
  2. Fully Convolutional Networks for Continuous Sign Language Recognition ECCV2020 paper code
  3. Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition AAAI2020 paper code ==中科大&对齐==
  4. Dense Temporal Convolution Network for Sign Language Translation IJCAI2019 paper code ==合工大==
  5. A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training IEEE TRANSACTIONS ON MULTIMEDIA 2019 paper code ==中科大&对齐==
  6. Iterative Alignment Network for Continuous Sign Language CVPR2019 paper code ==中科大&对齐==
  7. Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos TRAMI2019 paper code 非官方caffe ==Koller&开源&对齐==
  8. SF-Net: Structured Feature Network for Continuous Sign Language Recognition ArXiv2019 paper code
  9. Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition IJCAI2018 Paper code ==开源&中科大==
  10. Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs CVPR2017 paper code 非官方caffe ==Koller&开源&对齐==
  11. Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization CVPR2017 paper code ==对齐==

GCN

GCN for SLR

  1. Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition ICANN2019 paper code

*GCN-related Work

GCN for Action Recognition:

  1. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition paper code

GCN & Zero-shot: 2. Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs paper code ==GCN&zero-shot==

Others

Gesture Recognition

  1. Fingerspelling recognition in the wild with iterative visual attention ICCV2019 paper code
  2. Attention in Convolutional LSTM for Gesture Recognition NIPS2018 paper code ==开源==
  3. SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition ICCV2017 paper code ==开源&Camgoz&CNN+RNN==
  4. Learning Spatiotemporal Features Using 3DCNN and Convolutional LSTM for Gesture Recognition ICCV2017 paper code ==开源&CNN+RNN==
  5. Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition ICPR2016 paper code
  6. Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks CVPR2016 paper code
  7. Hand Gesture Recognition with 3D Convolutional Neural Networks CVPRW2015 paper code
  8. Sign Language Recognition Analysis using Multimodal Data DSAA2019 paper code

Untitle

  1. BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues ECCV2020 paper code

  2. Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison WACV2020 paper code

  3. Dynamic Sign Language Recognition Based on Video Sequence With BLSTM-3D Residual Networks ACCESS2019 paper code

  4. Temporal Accumulative Features for Sign Language Recognition ICCV2019 paper code

  5. Thai Sign Language Recognition Using 3D Convolutional Neural Networks ICCCM2019 paper code

  6. Human-like sign-language learning method using deep learning ETRI2018 paper code

  7. Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled CVPR2016 paper code

  8. Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition BMVC2016 paper code

  9. SIGN LANGUAGE RECOGNITION WITH LONG SHORT-TERM MEMORY ICIP2016 paper code

  10. Iterative Reference Driven Metric Learning for Signer Independent Isolated Sign Language Recognition. ECCV2016 paper code

  11. Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition LREC2016 paper code

  12. Sign Language Recognition using 3D convolutional neural networks ICME2015 paper code

  13. Curve Matching from the View of Manifold for Sign Language Recognition ACCV2014 paper code

  14. Sign Language Recognition and Translation with Kinect AFGR2013 paper code

  15. Large-scale Learning of Sign Language by Watching TV (Using Cooccurrences). BMVC2013 paper code

  16. Sign Language Recognition using Sequential Pattern Trees CVPR2012 paper code

  17. Sign language recognition using sub-units JMLR2012 paper code

  18. American Sign Language word recognition with a sensory glove using artificial neural networks Eng.Appl.Artif.Intell.2011 paper code

  19. Learning sign language by watching TV (using weakly aligned subtitles) CVPR2009 paper code

G2T

[Back to TOP]

  1. Translation of sign language glosses to text using sequence-to-sequence attention models. SITIS2019 paper code

Dataset Paper

  1. English-ASL Gloss Parallel Corpus 2012: ASLG-PC12, The Second Release ICTA2013 paper code

HMM

  1. Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition TOMM2017 paper code
  2. Chinese sign language recognition with adaptive HMM ICME2016 paper code
  3. Sign language recognition based on adaptive HMMS with data augmentation ICIP2016 paper code
  4. Continuous sign language recognition using level building based on fast hidden Markov model Pattern Recognit.Lett.2016 paper code
  5. Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications TACCESS2016 paper code
  6. A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition ICIMCS2014 paper code
  7. Improving Continuous Sign Language Recognition: Speech Recognition Techniques and System Design SLPAT2013 paper code
  8. Using Viseme Recognition to Improve a Sign Language Translation System IWSLT2013 paper code
  9. Advances in phonetics-based sub-unit modeling for transcription alignment and sign language recognition CVPRW2011 paper code
  10. Speech Recognition Techniques for a Sign Language Recognition System INTERSPEECH2007 paper code
  11. Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models TSMC2007 paper code
  12. Real-time American sign language recognition using desk and wearable computer based video TPAMI1998 paper code

Text2Sign

[Back to TOP]

  1. Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks IJCV2020 paper code
  2. Progressive Transformers for End-to-End Sign Language Production ECCV2020 paper code

Text-Gloss Translation

[Back to TOP]

XMU-SL

[Back to TOP]

  • CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. CVPR 2023; Highlight Paper [paper] ==ZJU&XMU&THU; SLR; Latest==

  • A Token-level Contrastive Framework for Sign Language Translation. ICASSP 2023 [paper]

  • Efficient Sign Language Translation with a Curriculum-based NAR Decoder. IJCAI 2023 [paper]

  • Enhancing neural sign language translation by highlighting the facial expression information. Neurocomputing 2021 [paper]

  • Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation. Arxiv 2021 [paper]

  • Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. CIN 2020 [paper]

  • Technical approaches to Chinese sign language processing: A review. IEEE Access 2019 [paper] ==Review Paper==

USTC-SL

[Back to TOP]

Conference papers:
  1. Boosting Continuous Sign Language Recognition via Cross Modality Augmentation ACMMM2020
  2. Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition AAAI2020 paper code
  3. Deep Grammatical Multi-classifier for Continuous Sign Language Recognition BigMM2019 paper code
  4. Continuous Sign Language Recognition via Reinforcement Learning ICIP2019 paper code ==#TODO==
  5. Iterative Alignment Network for Continuous Sign Language CVPR2019 paper code
  6. Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition ICME 2019. paper code
  7. Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition IJCAI2018 Paper code
  8. Video-based sign language recognition without temporal segmentation AAAI2018 paper code ==CNN+RNN==
  9. Hierarchical LSTM for Sign Language Translation AAAI2018 paper code ==CNN+RNN==
  10. Connectionist Temporal Fusion for Sign Language Translation ACMMM018 paper code ==CNN+RNN==
  11. Chinese Sign Language Recognition with Adaptive HMM ICME2016 paper code
  12. Sign Language Recognition with Multi-modal Features PCM2016
  13. Sign Language Recognition with Long Short Term Memory ICIP2016
  14. Sign Language Recognition based on Adaptive HMMs with Data Augmentation ICIP2016
  15. Sign Language Recognition Based on Trajectory Modeling with HMMs MMM2016
  16. Sign Language Recognition using Real-Sense ChinaSIP2015
  17. A New System forChinese Sign Language Recognition ChinaSIP2015
  18. Sign language recognition using 3D convolutional neural networks ICME2015
  19. A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition ICIMCS2014
Journal papers:
  1. Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition TCSVT2020
  2. Attention based 3D-CNNs for Large-Vocabulary Sign Language Recognition TCSVT2018 paper code

ZJU-SL

[Back to TOP]

  • CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. CVPR 2023; Highlight Paper [paper] [code] ==ZJU&XMU&THU; SLR; Latest==

RWTH

*Oscar Koller、Camgoz et. al

  1. Multi-channel Transformers for Multi-articulatory Sign Language Translation arxiv2020 paper code
  2. Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation CVPR2020 paper code
  3. Sign Language Translation with Transformers ArXiv2020 paper code
  4. Neural Sign Language Translation CVPR2018 paper code
  5. Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos TRAMI2019 paper code 非官方caffe
  6. Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs CVPR2017 paper code 非官方caffe

Neural SLT

  1. Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation CVPR2020 paper code
  2. Neural Sign Language Translation CVPR2018 paper code

SLR ---TODO

  1. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective ASSETS2019 paper code

  2. Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs IJCV2018 paper code

  3. Deep Learning of Mouth Shapes for Sign Language ICCVW2015 paper code

SL SURVEY REVIEW PAPERS

[Back to TOP]

  • Technical approaches to Chinese sign language processing: A review. IEEE Access 2019 [paper] ==Review Paper==

SL Linguistics

[Back to TOP]

Datasets

[Back to TOP]

Dataset Language Classes Samples Data Type Language Level
ASLG-PC12paper American - 87,709 GLOSS&Sentences isolated
CSL Dataset I Chinese 500 125,000 Videos&Depth from Kinect isolated
CSL Dataset II Chinese 100 25,000 Videos&Depth from Kinect continuous
RWTH-PHOENIX-Weather 2014 German 1,081 6,841 Videos continuous
RWTH-PHOENIX-Weather 2014 T German 1,066 8,257 Videos continuous
ASLLVD American 3,300 9,800 Videos(multiple angles) isolated
ASLLVD-Skeleton American 3,300 9,800 Skeleton isolated
SIGNUM German 450 33,210 Videos continuous
DGS Kinect 40 German 40 3,000 Videos(multiple angles) isolated
DEVISIGN-G Chinese 36 432 Videos isolated
DEVISIGN-D Chinese 500 6,000 Videos isolated
DEVISIGN-L Chinese 2000 24,000 Videos isolated
LSA64 Argentinian 64 3,200 Videos isolated
GSL isol. Greek 310 40,785 Videos&Depth from RealSense isolated
GSL SD Greek 310 10,290 Videos&Depth from RealSense continuous
GSL SI Greek 310 10,290 Videos&Depth from RealSense continuous
IIITA -ROBITA Indian 23 605 Videos isolated
PSL Kinect Polish 30 300 Videos&Depth from Kinect isolated
PSL ToF Polish 84 1,680 Videos&Depth from ToF camera isolated
BUHMAP-DB Turkish 8 440 Videos isolated
LSE-Sign Spanish 2,400 2,400 Videos isolated
Purdue RVL-SLLL American 39 546 Videos isolated
RWTH-BOSTON-50 American 50 483 Videos(multiple angles) isolated
RWTH-BOSTON-104 American 104 201 Videos(multiple angles) continuous
RWTH-BOSTON-400 American 400 843 Videos continuous
WLASL American 2,000 21,083 Videos isolated

Related Fields

[Back to TOP]

Action Recognition

  1. DistInit: Learning Video Representations Without a Single Labeled Video ICCV2019 paper code
  2. SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition ICCV2019 paper code
  3. Reasoning About Human-Object Interactions Through Dual Attention Networks ICCV2019 paper code
  4. SlowFast Networks for Video Recognition ICCV2019 paper code
  5. Video Classification with Channel-Separated Convolutional Networks ICCV2019 paper code
  6. BMN: Boundary-Matching Network for Temporal Action Proposal Generation ICCV2019 paper code
  7. DynamoNet: Dynamic Action and Motion Network ICCV2019 paper code
  8. Graph Convolutional Networks for Temporal Action Localization ICCV2019 paper code
  9. Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet? CVPR2018 paper code
  10. A Closer Look at Spatiotemporal Convolutions for Action Recognition CVPR2018 paper code
  11. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition AAAI2018 paper code
  12. Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation IJCAI2018 paper code
  13. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset CVPR2017 paper code
  14. Action Recognition using Visual Attention ICLR2016 paper code
  15. Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors CVPR2015 paper code
  16. Two-Stream Convolutional Networks for Action Recognition in Videos NIPS2014 paper code

Speech Recognition

  1. State-of-the-art Speech Recognition With Sequence-to-Sequence Models ICASSP2018 paper code
  2. Lip Reading Sentences in the Wild CVPR2017 paper code
  3. Listen, Attend and Spell ICASSP2016 paper code
  4. Deep speech 2: End-to-end speech recognition in english and mandarin ICML2016 paper code
  5. Attention-Based Models for Speech Recognition NIPS2015 paper code
  6. Convolutional Neural Networks for Speech Recognition TASLP2014 paper code
  7. Hybrid speech recognition with Deep Bidirectional LSTM ASRU2013 paper code
  8. New types of deep neural network learning for speech recognition and related applications: an overview ICASSP2013 paper code
  9. Speech Recognition with Deep Recurrent Neural Networks ICASSP2013 paper code

Video Captioning

  1. Video Description A Survey of Methods, Datasets and Evaluation Metrics ACM Computing Surveys2019 paper code
  2. Spatio-Temporal Dynamics and Semantic Attribute Enriched Visual Encoding for Video Captioning CVPR2019 paper code
  3. Reconstruction Network for Video Captioning CVPR2018 paper code
  4. Multimodal Memory Modelling for Video Captioning CVPR2018 paper code
  5. Interpretable Video Captioning via Trajectory Structured Localization CVPR2018 paper code
  6. Video Captioning with Transferred Semantic Attributes CVPR2017 paper code
  7. Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks CVPR2016 paper code
  8. Jointly Modeling Embedding and Translation to Bridge Video and Language CVPR2016 paper code
  9. Describing Videos by Exploiting Temporal Structure ICCV2015 paper code

About

Papers about AI+ Sign language