pavanchhatpar / copynet-tf

CopyNet (Copy Mechanism in Seq2Seq) implementation with TensorFlow 2

CopyNet implementation with TensorFlow 2

• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• Uses TensorFlow 2.0 and above APIs with tf.keras too
• Adapted from AllenNLP's PyTorch implementation, their blog referenced below was very helpful to understand the math from an implementation perspective

Install package

Using pip

pip install copynet-tf

Compile from source

python -m pip install --upgrade pip
pip install setuptools wheel
python setup.py sdist bdist_wheel

Examples

Abstract usage

...
import MyEncoder
from copynet_tf import GRUDecoder

...

class MyModel(tf.keras.Model):

  ...

  def call(self, X, y, training):
    source_token_ids, source2target_ids = X
    target_token_ids, target2source_ids = y

    enc_output, enc_final_output, mask = self.encoder(X, y, training)
    output_dict = self.decoder(
      source_token_ids, source2target_ids, mask,
      target_token_ids, target2source_ids, training)
    return output_dict

  ...

Concrete examples

Find concrete examples inside examples folder of the repo

References

• Incorporating Copying Mechanism in Sequence-to-Sequence Learning: (paper)
• AllenNLP implementation: (blog) (code)
• BLEU score metric: (code)