CyberZHG / keras-self-attention

Attention mechanism for processing sequential data that considers the context for each timestamp.

Home Page:https://pypi.org/project/keras-self-attention/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

error in building my bi-lstm with attention, help

denglizong opened this issue · comments

commented

Dear author,
Thanks for your keras-self-attention.

    Recently I am learning to develop a bi-lstm with attention model, and meet a mistake when use self-attention:

(for imdb dataset)
`model3 = Sequential()

model3.add( Embedding(max_features, 32) )
model3.add( layers.Bidirectional( layers.LSTM(32, return_sequences=True) ) )
model3.add(SeqSelfAttention(activation='sigmoid') )
model3.add(Dense(1, activation='sigmoid') )

model3.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
history = model3.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)`

 when I run model.fit, the value error comes,

`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
----> 1 history = model3.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)

~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
950 sample_weight=sample_weight,
951 class_weight=class_weight,
--> 952 batch_size=batch_size)
953 # Prepare validation data.
954 do_validation = False

~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
787 feed_output_shapes,
788 check_batch_axis=False, # Don't enforce the batch size.
--> 789 exception_prefix='target')
790
791 # Generate sample-wise weight values given the sample_weight and

~/denglz/venv4re/lib/python3.6/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
126 ': expected ' + names[i] + ' to have ' +
127 str(len(shape)) + ' dimensions, but got array '
--> 128 'with shape ' + str(data_shape))
129 if not check_batch_axis:
130 data_shape = data_shape[1:]

ValueError: Error when checking target: expected dense_6 to have 3 dimensions, but got array with shape (25000, 1)`

 Am i using the keras-self-attention in wrong way? Need your help , thanks a lot..

Use SeqWeightedAttention.

commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@CyberZHG using SeqWeightedAttention Does not solve the problem.