google-deepmind / dnc

A TensorFlow implementation of the Differentiable Neural Computer.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

dynamic batch size not work

ricky1203 opened this issue · comments

in util.py, tf.pack/tf.unpack are used to process the batch data, will it be better to use tf.scan for dynamic batch?
for example:

def batch_invert_permutation(permutations):
  """Returns batched `tf.invert_permutation` for every row in `permutations`."""
  with tf.name_scope('batch_invert_permutation', values=[permutations]):
    return tf.scan(lambda a, x: tf.invert_permutation(x), permutations)

def batch_gather(values, indices):
  """Returns batched `tf.gather` for every row in the input."""
  with tf.name_scope('batch_gather', values=[values, indices]):
    return tf.scan(lambda a, x: tf.gather(x[0], x[1]), (values, indices))

This will be solved if enhancement #21 is addressed. Essentially, unstack is required if top_k is used, in order to apply indices to each batch, however the usage mechanism can be reformulated to avoid sorting at all.

Apologies for the very late response but unfortunately we are not planning on supporting dynamic batch sizes, and we don't want to change the addressing mechanism in this repo as it is being kept frozen for clarity and consistency with the paper.