Using image_data_generator from R keras with probabilities like Albumentations

Question

Albumentations (in python) has image transformations that can take a probability argument p, for example:

transform = A.Compose([
        A.RandomRotate90(),
        A.Flip(),
        A.Transpose(),
        A.OneOf([
            A.IAAAdditiveGaussianNoise(),
            A.GaussNoise(),
        ], p=0.2),
        A.OneOf([
            A.MotionBlur(p=.2),
            A.MedianBlur(blur_limit=3, p=0.1),
            A.Blur(blur_limit=3, p=0.1),
        ], p=0.2),
        A.ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.2, rotate_limit=45, p=0.2),
        A.OneOf([
            A.OpticalDistortion(p=0.3),
            A.GridDistortion(p=.1),
            A.IAAPiecewiseAffine(p=0.3),
        ], p=0.2),
        A.OneOf([
            A.CLAHE(clip_limit=2),
            A.IAASharpen(),
            A.IAAEmboss(),
            A.RandomBrightnessContrast(),            
        ], p=0.3),
        A.HueSaturationValue(p=0.3),
    ])

Rstudio's Keras implementation has image_data_generator, which has image transformations, but no probability argument.

What's the easiest way to put together image transformations in Rstudio's Keras that are applied only at some level of probability?

One way would be to use Albumentations in python through reticulate, but practically speaking it's harder than it looks.

Ideas?

Answer 1

The recommended way to implement a dataset pipeline is to use {tfdatasets} in combination with keras preprocessing layers.

data_augmentation <- keras_model_sequential() %>%
  layer_random_flip("horizontal") %>%
  layer_random_rotation(0.1) %>%
  layer_random_zoom(0.2)

ds <- image_dataset_from_directory(image_size = c(180, 180)) %>%
  dataset_map(~ list(data_augmentation(.x), .y))

More examples can be found here: https://keras.rstudio.com/articles/new-guides/preprocessing_layers.html#quick-recipes-1

If you get into making custom transformations, you can also put then in a guard locally like this:

maybe_add_noise <- tf_function(function(x) {
  if(tf$random$uniform() > .5) {
    x <- x + tf$random$uniform(x$shape)
  }
  x
})

ds <- ds %>%
  dataset_map(~ list(maybe_add_noise(.x), .y))