AttributeError: 'L1' object has no attribute 'l2'

[Abbey]

I have the following code:

lr_multipliers = {'lstm_1': 0.5}
optimizer = AdamW(lr=1e-4, model=model_AdamW, lr_multipliers=lr_multipliers,
use_cosine_annealing=True, total_iterations=24)
model_AdamW.compile(optimizer, loss='categorical_crossentropy', metrics=['accuracy'])

and got error: AttributeError: 'L1' object has no attribute 'l2'

[John Muradeli]

Thanks for reporting. To be able to help, I'll need a reproducible example (minimal code to create the error), and versions of TensorFlow and/or Keras used.

[Abbey]

Thanks for reporting. To be able to help, I'll need a reproducible example (minimal code to create the error), and versions of TensorFlow and/or Keras used.

Here are the example and the version of TensorFlow and Keras:

print("You have TensorFlow version", tf.__version__)
print(tf.keras.__version__)
# You have TensorFlow version 2.3.1
# 2.4.0

#char_input
sequence_length = 5000
input_shape_char = (sequence_length,)
input_layer_char = Input(shape=input_shape_char, name='input_layer_char') 

# char_embed with LSTM
embedded = embedding_layer(input_layer_char)
x = LSTM(256, return_sequences=True)(embedded)
x = Dropout(0.5)(x)
x = LSTM(256, name='lstm_1', kernel_regularizer=l1(1e-4), recurrent_regularizer=l2(2e-4))(x)
x = Dropout(0.3)(x)
lstm_out = Dense(128, activation = 'relu')(x)
# word_input
input_shape_word = (sequence_length,)
input_layer_word = Input(shape=input_shape_word, name='input_layer_word') 
# Embedding layer Initialization
embedding_layer_word = Embedding(vocab_size+1, embedding_dim_word, input_length=sequence_length,
                            weights=[embedding_matrix_word])
# Embedding
embedded_word = embedding_layer_word(input_layer_word)
# 1-D CNN
y = Conv1D(32, 6, activation='relu')(embedded_word)
y = MaxPooling1D(2)(y)
y = Conv1D(64, 8, activation='relu')(y)
y = MaxPooling1D(2)(y)
y = Flatten()(y)
y = Dropout(0.3)(y)
cnn_out = Dense(128, activation='relu')(y)
concat_inp = concatenate([cnn_out, lstm_out])
z = Dense(256, activation='relu')(concat_inp)
z = Dropout(0.3)(z)
z = Dense(1024, activation='relu')(z)
z = Dropout(0.3)(z)
predicted = Dense(num_of_classes, activation='softmax', kernel_regularizer=l1_l2(1e-4, 2e-4))(z)
model_AdamW = Model(inputs=[input_layer_char,input_layer_word], outputs=[predicted])

lr_multipliers = {'lstm_1': 0.5}
optimizer = AdamW(lr=1e-4, model=model_AdamW, lr_multipliers=lr_multipliers,
                  use_cosine_annealing=True, total_iterations=24)
model_AdamW.compile(optimizer, loss='categorical_crossentropy', metrics=['accuracy'])

and got error: AttributeError: 'L1' object has no attribute 'l2'

[John Muradeli]

I'm afraid there are missing imports and variable definitions; I should be able to run the code by copy/pasting into an empty file.

[John Muradeli]

If I were to guess, overwriting keras_adamw\utils.py with this should fix it; I'll test with 2.3.1 later.

[Abbey]

If I were to guess, overwriting keras_adamw\utils.py with this should fix it; I'll test with 2.3.1 la

[Abbey]

I'm afraid there are missing imports and variable definitions; I should be able to run the code by copy/pasting into an empty file.

Same error when I try to run example released with the library on Tensorflow 2.3.1 and Keras 2.4.0.