lilianweng / transformer-tensorflow

Implementation of Transformer Model in Tensorflow

Home Page:https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

some typo

jszhujun2010 opened this issue · comments

learned a lot from this, some typo found either.
line 177, shape should be [batch, seq_len, target_vocab]

# Make the prediction out of the decoder output.<br/>
logits = tf.layers.dense(dec_out, target_vocab)  # [batch, target_vocab]

line 433, should be num_dec_layers

for i in range(self.num_enc_layers):
    out = self.decoder_layer(out, enc_out, input_mask, target_mask, f'dec_{i}')