error in building my bi-lstm with attention, help
denglizong opened this issue · comments
Dear author,
Thanks for your keras-self-attention.
Recently I am learning to develop a bi-lstm with attention model, and meet a mistake when use self-attention:
(for imdb dataset)
`model3 = Sequential()
model3.add( Embedding(max_features, 32) )
model3.add( layers.Bidirectional( layers.LSTM(32, return_sequences=True) ) )
model3.add(SeqSelfAttention(activation='sigmoid') )
model3.add(Dense(1, activation='sigmoid') )
model3.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
history = model3.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)`
when I run model.fit, the value error comes,
`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
----> 1 history = model3.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)
~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
950 sample_weight=sample_weight,
951 class_weight=class_weight,
--> 952 batch_size=batch_size)
953 # Prepare validation data.
954 do_validation = False
~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
787 feed_output_shapes,
788 check_batch_axis=False, # Don't enforce the batch size.
--> 789 exception_prefix='target')
790
791 # Generate sample-wise weight values given the sample_weight
and
~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
126 ': expected ' + names[i] + ' to have ' +
127 str(len(shape)) + ' dimensions, but got array '
--> 128 'with shape ' + str(data_shape))
129 if not check_batch_axis:
130 data_shape = data_shape[1:]
ValueError: Error when checking target: expected dense_6 to have 3 dimensions, but got array with shape (25000, 1)`
Am i using the keras-self-attention in wrong way? Need your help , thanks a lot..
Use SeqWeightedAttention
.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
@CyberZHG using SeqWeightedAttention
Does not solve the problem.