Tensorboard plot ReduceLROnPlateau

Question

I keep failing to plot my learning rate in tensorboard because I am using the ReduceLROnPlateau as following:

tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=results_path, histogram_freq=1)

reduce_lr = ReduceLROnPlateau(monitor='loss', factor=0.5, verbose=1,
                              patience=100, min_lr=0.000001)
callbacks = [tensorboard_callback, reduce_lr]
# Compile VAE
vae.compile(optimizer='adam', loss=kl_reconstruction_loss,  metrics=["mse", metric_KL,binary_crossentropy])

# Train autoencoder
history = vae.fit(x_train, x_train, 
                  epochs = no_epochs, 
                  batch_size = batch_size, 
                  validation_data=(x_test,x_test,),
                  callbacks=callbacks)

After that I run this to plot the custom metrics to tensorboard log:

for epoch in range(len(history.history['mse'])):
    with train_summary_writer.as_default():
        tf.summary.scalar('metric_KL', history.history['metric_KL'][epoch], step=epoch)

With that setup. How can I plot my learning rate without writing my own custom ReduceLROnPlateau? Thx

Answer 1

The recommended way is to overwrite the TensorBoard callback.

You can see here how you can do that: Keras: how to output learning rate onto tensorboard.

You just need to adapt the code with the imports for tensorflow.keras instead of plain keras.