Label smoothing for binary cross entropy in tensorflow

4.8k views Asked by At

I am using the following function for label smoothing in tensorflow.

tf.keras.losses.BinaryCrossentropy(from_logits=False, label_smoothing=0, 
       reduction=losses_utils.ReductionV2.AUTO, name='binary_crossentropy')

If I assign label_smoothing = 0.1, does that mean it will generate random numbers between 0 and 0.1 instead of hard label of 0 for fake images and 0.9 to 1 instead of 1 for real images? I am trying to stabilize my generative adversarial network training. Thank you.

1

There are 1 answers

0
Hai Feng Kao On

label_smoothing = 0.1 , y_true = 0.95 , y_false = 0.05

label_smoothing = 1.0 , y_true = 0.5 , y_false = 0.5

  label_smoothing = ops.convert_to_tensor_v2(label_smoothing, dtype=K.floatx())

  def _smooth_labels():
    return y_true * (1.0 - label_smoothing) + 0.5 * label_smoothing

  y_true = smart_cond.smart_cond(label_smoothing,
                                 _smooth_labels, lambda: y_true)
  return K.mean(
      K.binary_crossentropy(y_true, y_pred, from_logits=from_logits), axis=-1)

https://github.com/tensorflow/tensorflow/blob/fcc4b966f1265f466e82617020af93670141b009/tensorflow/python/keras/losses.py#L1573