How to add tensorboard logging?
dtyugin opened this issue · comments
Dmitry Tyugin commented
I'm trying to add tensorboard logging to mask_rcnn.py train function
log_dir = "logs/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
callb = [
tensorboard_callback,
ModelCheckpoint(self.checkpoint_path,save_weights_only=True,save_best_only = True, monitor = "val_loss", verbose = 0),
lr_rate,
]
But training failed with error on start. Do you have any suggestions how to fix it? (I have tensorboard and tensorflow 2.5.0)
File "/home/greenfield/dl/pixllib_ws/pixellib/custom_train.py", line 163, in train_model
epochs=num_epochs,layers=layers)
File "/home/greenfield/dl/pixllib_ws/pixellib/mask_rcnn.py", line 2321, in train
verbose = 1
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_v1.py", line 814, in fit
use_multiprocessing=use_multiprocessing)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_generator_v1.py", line 590, in fit
steps_name='steps_per_epoch')
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_generator_v1.py", line 177, in model_iteration
mode=mode)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py", line 119, in configure_callbacks
callback_list.set_model(callback_model)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py", line 307, in set_model
callback.set_model(model)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/callbacks_v1.py", line 240, in set_model
self._make_histogram_ops(model)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/keras/callbacks_v1.py", line 231, in _make_histogram_ops
tf_summary.histogram('{}_out'.format(layer.name), layer.output)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/summary/summary.py", line 179, in histogram
tag=tag, values=values, name=scope)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 286, in histogram_summary
"HistogramSummary", tag=tag, values=values, name=name)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 630, in _apply_op_helper
param_name=input_name)
File "/home/greenfield/anaconda3/envs/pixellib/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 63, in _SatisfiesTypeConstraint
", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'values' has DataType bool not in list of allowed values: float32, float64, int32, uint8, int16, int8, int64, bfloat16, uint16, float16, uint32, uint64
HumanCallback commented
A small workaround would be setting the Histogram Frequency to 0. However if you need the histogram I don't know a solution.