hw_colors - Decoder foward method requiring lengths
maerory opened this issue · comments
Hi,
I was working on the hw_colors notebook to create the updated EncoderDecoder model following the instructions. I thought I got all my updates correct until I received an error in the last test.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-217-f204130de1b0> in <module>
----> 1 test_full_system(ColorizedInputDescriber)
<ipython-input-216-df38441535ee> in test_full_system(describer_class)
8 toy_mod = describer_class(toy_vocab)
9
---> 10 _ = toy_mod.fit(toy_color_seqs_train, toy_word_seqs_train)
11
12 acc = toy_mod.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
~/Downloads/Stanford-CS224U/codebase/torch_model_base.py in fit(self, *args)
359 y_batch = batch[-1]
360
--> 361 batch_preds = self.model(*X_batch)
362
363 err = self.loss(batch_preds, y_batch)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
<ipython-input-214-6cb1cf97c969> in forward(self, color_seqs, word_seqs, seq_lengths, hidden, targets)
18 output, hidden = self.decoder(
19 word_seqs,
---> 20 target_colors=color_seqs[:,-1,:])
21
22 # Your decoder will return `output, hidden` pairs; the
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
~/Downloads/Stanford-CS224U/codebase/torch_color_describer.py in forward(self, word_seqs, seq_lengths, hidden, target_colors)
232 batch_first=True,
233 lengths=seq_lengths,
--> 234 enforce_sorted=False)
235 # RNN forward:
236 output, hidden = self.rnn(embs, hidden)
/usr/local/lib/python3.7/site-packages/torch/nn/utils/rnn.py in pack_padded_sequence(input, lengths, batch_first, enforce_sorted)
232 'the trace incorrect for any other combination of lengths.',
233 stacklevel=2)
--> 234 lengths = torch.as_tensor(lengths, dtype=torch.int64)
235 if enforce_sorted:
236 sorted_indices = None
TypeError: an integer is required (got type NoneType)
I was having a hard time understanding this error because I thought we didn't need to input hidden and seq_lengths in the forward step.
I am defining my decoder call as:
output, hidden = self.decoder(
word_seqs,
target_colors=color_seqs[:,-1,:])
Is it a mistake in my code or do I have to include an additional length variable?
I know this is part of the homework and not necessarily related to the code base, but any help would be appreciated.
Thank you
@maerory My hunch is that it is just the fact that you're not passing through seq_lengths to self.decoder. Do try that. If it doesn't work, then perhaps email me your notebook and I'll check it out.
@cgpotts Thank you for your response!
In the torch_color_describer, it says the variable seq_lengths is a LongTensor of shape(m,) where m is the number of examples in the batch.
Does this mean that we are returning the lengths of each sequence in the word_seqs, in code which translates to:
torch.LongTensor([len(seqs)-1 for seqs in word_seqs])
However, this returned an index out of range error. So, I am not sure whether I understood it correctly.
@maerory You should just add seq_lengths=seq_lengths to your call to self.decoder!
My bad, I didn't know that seq_lenghts were automatically generated.
However, I am now getting an IndexError.
output, hidden = self.decoder(
word_seqs,
seq_lengths = seq_lengths,
hidden = hidden,
target_colors=color_seqs[:,-1,:])
This is my call to decode.forward now. For target colors, we are assuming that the last element of each color batches are our target right? Given my error, I think the only mistake would lie in my input of target_colors.
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-82-f204130de1b0> in <module>
----> 1 test_full_system(ColorizedInputDescriber)
<ipython-input-79-df38441535ee> in test_full_system(describer_class)
8 toy_mod = describer_class(toy_vocab)
9
---> 10 _ = toy_mod.fit(toy_color_seqs_train, toy_word_seqs_train)
11
12 acc = toy_mod.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
~/Downloads/Stanford-CS224U/codebase/torch_model_base.py in fit(self, *args)
359 y_batch = batch[-1]
360
--> 361 batch_preds = self.model(*X_batch)
362
363 err = self.loss(batch_preds, y_batch)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
<ipython-input-81-839c49bd1137> in forward(self, color_seqs, word_seqs, seq_lengths, hidden, targets)
20 seq_lengths = seq_lengths,
21 hidden = hidden,
---> 22 target_colors=color_seqs[:,-1,:])
23
24 # Your decoder will return `output, hidden` pairs; the
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
~/Downloads/Stanford-CS224U/codebase/torch_color_describer.py in forward(self, word_seqs, seq_lengths, hidden, target_colors)
234 enforce_sorted=False)
235 # RNN forward:
--> 236 output, hidden = self.rnn(embs, hidden)
237 # Unpack:
238 output, seq_lengths = torch.nn.utils.rnn.pad_packed_sequence(
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/rnn.py in forward(self, input, hx)
563 # Each batch of the hidden state should match the input sequence that
564 # the user believes he/she is passing in.
--> 565 hx = self.permute_hidden(hx, sorted_indices)
566
567 self.check_forward_args(input, hx, batch_sizes)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/rnn.py in permute_hidden(self, hx, permutation)
529 if permutation is None:
530 return hx
--> 531 return apply_permutation(hx[0], permutation), apply_permutation(hx[1], permutation)
532
533 @torch._jit_internal._overload_method # noqa: F811
/usr/local/lib/python3.7/site-packages/torch/nn/modules/rnn.py in apply_permutation(tensor, permutation, dim)
18 def apply_permutation(tensor, permutation, dim=1):
19 # type: (Tensor, Tensor, int) -> Tensor
---> 20 return tensor.index_select(dim, permutation)
21
22
IndexError: index out of range in self
@maerory It won't be possible for me to debug this without seeing all of your code. Please verify that your answers are passing all the tests that the notebook provides, and then share it with me if those tests don't reveal what the problem is. Please don't post your notebook here, though, as that will give information away about the requirements.
This problem can be fixed by using a GRU rather than an LSTM cell. LSTM cells require further modifications because they return tuples, which complicates the step of augmenting their representations.