Problem with RCFR using Keras 3
lanctot opened this issue · comments
Running RCFR on Ubuntu 24.04 using Python 3.12 and Keras 3.1.1:
(venv) lanctot@nitro-exp:~/open_spiel/open_spiel/python/algorithms$ python rcfr_test.py
2024-04-13 08:00:16.251457: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-13 08:00:16.281499: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-13 08:00:16.830370: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING:tensorflow:From /home/lanctot/venv/lib/python3.12/site-packages/tensorflow/python/compat/v2_compat.py:98: disable_resource_variables (from tensorflow.python.ops.resource_variables_toggle) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Running tests under Python 3.12.2: /home/lanctot/venv/bin/python
2024-04-13 08:00:17.510387: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-04-13 08:00:17.510634: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[ RUN ] RcfrTest.test_all_states
INFO:tensorflow:time(__main__.RcfrTest.test_all_states): 0.0s
I0413 08:00:17.512515 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_all_states): 0.0s
[ OK ] RcfrTest.test_all_states
[ RUN ] RcfrTest.test_cfr
INFO:tensorflow:time(__main__.RcfrTest.test_cfr): 0.04s
I0413 08:00:17.556168 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_cfr): 0.04s
[ OK ] RcfrTest.test_cfr
[ RUN ] RcfrTest.test_counterfactual_regrets_and_reach_weights
INFO:tensorflow:time(__main__.RcfrTest.test_counterfactual_regrets_and_reach_weights): 0.02s
I0413 08:00:17.572896 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_counterfactual_regrets_and_reach_weights): 0.02s
[ OK ] RcfrTest.test_counterfactual_regrets_and_reach_weights
[ RUN ] RcfrTest.test_counterfactual_regrets_and_reach_weights_value_error
INFO:tensorflow:time(__main__.RcfrTest.test_counterfactual_regrets_and_reach_weights_value_error): 0.02s
I0413 08:00:17.589344 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_counterfactual_regrets_and_reach_weights_value_error): 0.02s
[ OK ] RcfrTest.test_counterfactual_regrets_and_reach_weights_value_error
[ RUN ] RcfrTest.test_normalized_by_sum
INFO:tensorflow:time(__main__.RcfrTest.test_normalized_by_sum): 0.0s
I0413 08:00:17.589827 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_normalized_by_sum): 0.0s
[ OK ] RcfrTest.test_normalized_by_sum
[ RUN ] RcfrTest.test_num_features
INFO:tensorflow:time(__main__.RcfrTest.test_num_features): 0.0s
I0413 08:00:17.589981 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_num_features): 0.0s
[ OK ] RcfrTest.test_num_features
[ RUN ] RcfrTest.test_rcfr0 (False, False)
[ FAILED ] RcfrTest.test_rcfr0 (False, False)
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr0 (False, False)): 0.32s
I0413 08:00:17.912854 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr0 (False, False)): 0.32s
[ RUN ] RcfrTest.test_rcfr1 (False, True)
[ FAILED ] RcfrTest.test_rcfr1 (False, True)
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr1 (False, True)): 0.08s
I0413 08:00:17.996713 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr1 (False, True)): 0.08s
[ RUN ] RcfrTest.test_rcfr2 (True, False)
WARNING:tensorflow:5 out of the last 5 calls to <function feedforward_evaluate at 0x77618569f240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
W0413 08:00:18.054587 131263126016128 polymorphic_function.py:157] 5 out of the last 5 calls to <function feedforward_evaluate at 0x77618569f240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
WARNING:tensorflow:6 out of the last 6 calls to <function feedforward_evaluate at 0x77618569f240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
W0413 08:00:18.073678 131263126016128 polymorphic_function.py:157] 6 out of the last 6 calls to <function feedforward_evaluate at 0x77618569f240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
[ FAILED ] RcfrTest.test_rcfr2 (True, False)
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr2 (True, False)): 0.08s
I0413 08:00:18.079555 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr2 (True, False)): 0.08s
[ RUN ] RcfrTest.test_rcfr3 (True, True)
[ FAILED ] RcfrTest.test_rcfr3 (True, True)
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr3 (True, True)): 0.08s
I0413 08:00:18.163169 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr3 (True, True)): 0.08s
[ RUN ] RcfrTest.test_rcfr_functions
/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py:355: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
reach_probabilities[player] = next_reach_prob
/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py:353: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
reach_weights[sequence_idx] += next_reach_prob
/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py:364: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
action_values[action_idx] = action_value
[ FAILED ] RcfrTest.test_rcfr_functions
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr_functions): 0.08s
I0413 08:00:18.247504 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr_functions): 0.08s
[ RUN ] RcfrTest.test_rcfr_with_buffer
[ FAILED ] RcfrTest.test_rcfr_with_buffer
INFO:tensorflow:time(__main__.RcfrTest.test_rcfr_with_buffer): 0.09s
I0413 08:00:18.341853 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_rcfr_with_buffer): 0.09s
[ RUN ] RcfrTest.test_reservior_buffer_insert
INFO:tensorflow:time(__main__.RcfrTest.test_reservior_buffer_insert): 0.0s
I0413 08:00:18.344093 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_reservior_buffer_insert): 0.0s
[ OK ] RcfrTest.test_reservior_buffer_insert
[ RUN ] RcfrTest.test_reservior_buffer_insert_all
INFO:tensorflow:time(__main__.RcfrTest.test_reservior_buffer_insert_all): 0.0s
I0413 08:00:18.344485 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_reservior_buffer_insert_all): 0.0s
[ OK ] RcfrTest.test_reservior_buffer_insert_all
[ RUN ] RcfrTest.test_root_state_wrapper_num_sequences
INFO:tensorflow:time(__main__.RcfrTest.test_root_state_wrapper_num_sequences): 0.02s
I0413 08:00:18.360113 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_root_state_wrapper_num_sequences): 0.02s
[ OK ] RcfrTest.test_root_state_wrapper_num_sequences
[ RUN ] RcfrTest.test_root_state_wrapper_sequence_features
INFO:tensorflow:time(__main__.RcfrTest.test_root_state_wrapper_sequence_features): 0.02s
I0413 08:00:18.376622 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_root_state_wrapper_sequence_features): 0.02s
[ OK ] RcfrTest.test_root_state_wrapper_sequence_features
[ RUN ] RcfrTest.test_root_state_wrapper_sequence_indices
INFO:tensorflow:time(__main__.RcfrTest.test_root_state_wrapper_sequence_indices): 0.02s
I0413 08:00:18.392388 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_root_state_wrapper_sequence_indices): 0.02s
[ OK ] RcfrTest.test_root_state_wrapper_sequence_indices
[ RUN ] RcfrTest.test_root_state_wrapper_sequence_terminal_values
INFO:tensorflow:time(__main__.RcfrTest.test_root_state_wrapper_sequence_terminal_values): 0.02s
I0413 08:00:18.408025 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_root_state_wrapper_sequence_terminal_values): 0.02s
[ OK ] RcfrTest.test_root_state_wrapper_sequence_terminal_values
[ RUN ] RcfrTest.test_sequence_features
INFO:tensorflow:time(__main__.RcfrTest.test_sequence_features): 0.0s
I0413 08:00:18.413053 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_sequence_features): 0.0s
[ OK ] RcfrTest.test_sequence_features
[ RUN ] RcfrTest.test_sequence_weights_to_tabular_profile
INFO:tensorflow:time(__main__.RcfrTest.test_sequence_weights_to_tabular_profile): 0.02s
I0413 08:00:18.429589 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_sequence_weights_to_tabular_profile): 0.02s
[ OK ] RcfrTest.test_sequence_weights_to_tabular_profile
[ RUN ] RcfrTest.test_session
[ SKIPPED ] RcfrTest.test_session
[ RUN ] RcfrTest.test_with_one_hot_action_features_batch
INFO:tensorflow:time(__main__.RcfrTest.test_with_one_hot_action_features_batch): 0.01s
I0413 08:00:18.435335 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_with_one_hot_action_features_batch): 0.01s
[ OK ] RcfrTest.test_with_one_hot_action_features_batch
[ RUN ] RcfrTest.test_with_one_hot_action_features_error
INFO:tensorflow:time(__main__.RcfrTest.test_with_one_hot_action_features_error): 0.0s
I0413 08:00:18.436534 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_with_one_hot_action_features_error): 0.0s
[ OK ] RcfrTest.test_with_one_hot_action_features_error
[ RUN ] RcfrTest.test_with_one_hot_action_features_single_state_vector
INFO:tensorflow:time(__main__.RcfrTest.test_with_one_hot_action_features_single_state_vector): 0.01s
I0413 08:00:18.442354 131263126016128 test_util.py:2620] time(__main__.RcfrTest.test_with_one_hot_action_features_single_state_vector): 0.01s
[ OK ] RcfrTest.test_with_one_hot_action_features_single_state_vector
======================================================================
ERROR: test_rcfr0 (False, False) (__main__.RcfrTest)
RcfrTest.test_rcfr0 (False, False)
test_rcfr(False, False)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/venv/lib/python3.12/site-packages/absl/testing/parameterized.py", line 320, in bound_param_test
return test_method(self, *testcase_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 518, in test_rcfr
patient.evaluate_and_update_policy(_train)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py", line 779, in evaluate_and_update_policy
train_fn(regret_player_model, data)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 507, in _train
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
======================================================================
ERROR: test_rcfr1 (False, True) (__main__.RcfrTest)
RcfrTest.test_rcfr1 (False, True)
test_rcfr(False, True)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/venv/lib/python3.12/site-packages/absl/testing/parameterized.py", line 320, in bound_param_test
return test_method(self, *testcase_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 518, in test_rcfr
patient.evaluate_and_update_policy(_train)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py", line 779, in evaluate_and_update_policy
train_fn(regret_player_model, data)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 507, in _train
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
======================================================================
ERROR: test_rcfr2 (True, False) (__main__.RcfrTest)
RcfrTest.test_rcfr2 (True, False)
test_rcfr(True, False)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/venv/lib/python3.12/site-packages/absl/testing/parameterized.py", line 320, in bound_param_test
return test_method(self, *testcase_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 518, in test_rcfr
patient.evaluate_and_update_policy(_train)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py", line 779, in evaluate_and_update_policy
train_fn(regret_player_model, data)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 507, in _train
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
======================================================================
ERROR: test_rcfr3 (True, True) (__main__.RcfrTest)
RcfrTest.test_rcfr3 (True, True)
test_rcfr(True, True)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/venv/lib/python3.12/site-packages/absl/testing/parameterized.py", line 320, in bound_param_test
return test_method(self, *testcase_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 518, in test_rcfr
patient.evaluate_and_update_policy(_train)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py", line 779, in evaluate_and_update_policy
train_fn(regret_player_model, data)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 507, in _train
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
======================================================================
ERROR: test_rcfr_functions (__main__.RcfrTest)
RcfrTest.test_rcfr_functions
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 479, in test_rcfr_functions
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
======================================================================
ERROR: test_rcfr_with_buffer (__main__.RcfrTest)
RcfrTest.test_rcfr_with_buffer
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 579, in test_rcfr_with_buffer
patient.evaluate_and_update_policy(_train)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr.py", line 873, in evaluate_and_update_policy
train_fn(regret_player_model, data)
File "/home/lanctot/open_spiel/open_spiel/python/algorithms/rcfr_test.py", line 568, in _train
optimizer = tf.keras.optimizers.Adam(lr=0.005, amsgrad=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/adam.py", line 60, in __init__
super().__init__(
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/backend/tensorflow/optimizer.py", line 20, in __init__
super().__init__(*args, **kwargs)
File "/home/lanctot/venv/lib/python3.12/site-packages/keras/src/optimizers/base_optimizer.py", line 38, in __init__
raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.005}
----------------------------------------------------------------------
Ran 24 tests in 0.931s
FAILED (errors=6, skipped=1)
(venv) lanctot@nitro-exp:~/open_spiel/open_spiel/python/algorithms$
Tagging original author @dmorrill10, but we could use some community help to port this to Keras 3.
The argument to the Adam optimizer in Keras 3 is learning_rate, not lr. I'm guessing changing that will solve your problem.
The argument to the Adam optimizer in Keras 3 is
learning_rate, notlr. I'm guessing changing that will solve your problem.
I believe I tried it and it led to other problems. Hopefully someone who knows Keras 3 can fix this eventually. I doubt the problem is very deep.. just requires knowledge of Keras 3.