jack57lee / Diversify-MHA

EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement

Geek Repo

Github PK Tool

neural-machine-transliteration deep-learning multi-head-attention

On the Diversity of Multi-Head Attention

Implementation for EMNLP 2018 paper: Multi-Head Attention with Disagreement Regularization and NAACL 2019 paper: Information Aggregation for Multi-Head Attention with Routing-by-Agreement, based on the THUMT toolkit.

More details including data and pre-trained models are coming later.

About

EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Routing-by-Agreement

neural-machine-transliteration deep-learning multi-head-attention

Languages

Language:Python 100.0%