tf.keras.metrics.MeanIoU with sigmoid layer

Question

I have a network for semantic segmentation and the last layer of my model applies a sigmoid activation, so all predictions are scaled between 0-1. There is this validation metric tf.keras.metrics.MeanIoU(num_classes), which compares classified predictions (0 or 1) with validation (0 or 1). So if i make a prediction and apply this metric, will it automatically map the continuous predictions to binary with threshold = 0.5? Are there any possibilities to manually define the threshold?

Answer 1

No, tf.keras.metrics.MeanIoU will not automatically map the continuous predictions to binary with threshold = 0.5.

It will convert the continuous predictions to its binary, by taking the binary digit before decimal point as predictions like 0.99 as 0, 0.50 as 0, 0.01 as 0, 1.99 as 1, 1.01 as 1 etc when num_classes=2. So basically if your predicted values are between 0 to 1 and num_classes=2, then everything is considered 0 unless the prediction is 1.

Below are the experiments to justify the behavior in tensorflow version 2.2.0:

All binary result :

import tensorflow as tf

m = tf.keras.metrics.MeanIoU(num_classes=2)
_ = m.update_state([0, 0, 1, 1], [0, 0, 1, 1])
m.result().numpy()

Output -

1.0

Change one prediction to continuous 0.99 - Here it considers 0.99 as 0.

import tensorflow as tf

m = tf.keras.metrics.MeanIoU(num_classes=2)
_ = m.update_state([0, 0, 1, 1], [0, 0, 1, 0.99])
m.result().numpy()

Output -

0.5833334

Change one prediction to continuous 0.01 - Here it considers 0.01 as 0.

import tensorflow as tf

m = tf.keras.metrics.MeanIoU(num_classes=2)
_ = m.update_state([0, 0, 1, 1], [0, 0.01, 1, 1])
m.result().numpy()

Output -

1.0

Change one prediction to continuous 1.99 - Here it considers 1.99 as 1.

%tensorflow_version 2.x
import tensorflow as tf

m = tf.keras.metrics.MeanIoU(num_classes=2)
_ = m.update_state([0, 0, 1, 1], [0, 0, 1, 1.99])
m.result().numpy()

Output -

1.0

So ideal way is to define a function to convert the continuous to binary before evaluating the MeanIoU.

Hope this answers your question. Happy Learning.

Answer 2

Try this(remember to replace the space with tab):

def mean_iou(y_true, y_pred):
    th = 0.5
    y_pred_ = tf.to_int32(y_pred > th)
    score, up_opt = tf.metrics.mean_iou(y_true, y_pred_, 2)
    K.get_session().run(tf.local_variables_initializer())
    with tf.control_dependencies([up_opt]):
        score = tf.identity(score)
    return score