关于添加的Bilstm_atttention代码的问题？
garboa opened this issue · comments
你好，运行代码出现这样的问题？请问如何修改？
Tensor("dense_1/Tanh:0", shape=(?, 100), dtype=float32)
Layer (type) Output Shape Param #
input_1 (InputLayer) (None, 1000) 0
embedding_1 (Embedding) (None, 1000, 100) 8056700
bidirectional_1 (Bidirection (None, 1000, 200) 160800
attention_layer_1 (Attention (None, 1000, 200) 40200
dense_1 (Dense) (None, 1000, 100) 20100
dense_2 (Dense) (None, 1000, 2) 202
Total params: 8,278,002
Trainable params: 221,302
Non-trainable params: 8,056,700
IMDB_Classifier_BiLSTM_Attention.py:124: UserWarning: The nb_epoch argument in fit has been renamed epochs.
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)
Traceback (most recent call last):
File "IMDB_Classifier_BiLSTM_Attention.py", line 124, in
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1522, in fit
batch_size=batch_size)
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1382, in _standardize_user_data
exception_prefix='target')
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 132, in _standardize_input_data
str(array.shape))
ValueError: Error when checking target: expected dense_2 to have 3 dimensions, but got array with shape (20000, 2)
你这个应该是网络最后层的输出与你传进去的label数据不对应引起的。按理说attention层的输出就应该是2维的了，不知道你这里怎么还是3维的，你看一下你的网络结构有没有问题。
这份代码我什么都没有改跑的，这个包里的lstm的代码是跑通了的，这个文件相比起lstm的是加了attention层，是否单独写attention层的代码需要改一下？
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
l_gru = Bidirectional(LSTM(100, return_sequences=True))(embedded_sequences)
l_att = Attention_layer()(l_gru)
dense_1 = Dense(100,activation='tanh')(l_att)
print(dense_1)
dense_2 = Dense(2, activation='softmax')(dense_1)
dense_2 = tf.reshape(dense_2)
model = Model(sequence_input, dense_2)
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['acc'])
model.summary()
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)
我解决了这个问题keras 1.2.2版本 @garboa @mpk001 在输出层前加一个flatten，代码如下
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
l_gru = Bidirectional(LSTM(100, return_sequences=True))(embedded_sequences)
dense_1 = Dense(100,activation='tanh')(l_gru)
flatten = Flatten()(dense_1)
dense_2 = Dense(2, activation='softmax')(flatten)
model = Model(sequence_input, dense_2)
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['acc'])
model.summary()
请问大家在跑程序的时候，会不会出现头两个epoch时，val_acc>acc的情况？一般情况下测试集的准确率不会比训练集大吧？
keras 2.1.5 tensorflow 1.7环境 在Attention_layer.py中增加下面代码可以解决ValueError: Error when checking target: expected dense_2 to have 3 dimensions,问题
def compute_output_shape(self, input_shape):
# output_dim = K.int_shape(self.u)
return (input_shape[0], input_shape[-1])