Reverse Engineering to find where a given operation in a python library is executed

Question

There is an interesting notebook on https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/expectation_over_transformation_classification_rotation.ipynb to perform adversarial attacks when the image is also rotated.

However, I would like to edit the transformation to include other transformations.

From the code, it is seen that the following class is imported for the rotation:

from art.preprocessing.expectation_over_transformation import EoTImageRotationTensorFlow

eot_rotation = EoTImageRotationTensorFlow(nb_samples=eot_samples,
                                      clip_values=clip_values,
                                      angles=eot_angle)

Then, the attack uses the information about the EOTImageRotationTensorflow class to attack the image

classifier_eot = TensorFlowV2Classifier(model=model,
                                    nb_classes=nb_classes,
                                    loss_object=loss,
                                    preprocessing=preprocessing,
                                    preprocessing_defences=[eot_rotation],
                                    clip_values=clip_values,
                                    input_shape=input_shape)

attack_eot = ProjectedGradientDescent(estimator=classifier_eot,
                                  eps=eps,
                                  max_iter=num_steps,
                                  eps_step=eps_step,
                                  targeted=True)

when I checked such a class in /usr/local/lib/python3.8/dist-packages/art/preprocessing/expectation_over_transformation/image_rotation/tensorflow.py, I saw that there is a function that performs the rotation.

def _transform(
    self, x: "tf.Tensor", y: Optional["tf.Tensor"], **kwargs
) -> Tuple["tf.Tensor", Optional["tf.Tensor"]]:
    """
    Transformation of an input image and its label by randomly sampled rotation.

    :param x: Input samples.
    :param y: Label of the samples `x`.
    :return: Transformed samples and labels.
    """
    import tensorflow as tf  # lgtm [py/repeated-import]
    import tensorflow_addons as tfa

    # pylint: disable=E1120,E1123
    angles = tf.random.uniform(shape=(), minval=self.angles_range[0], maxval=self.angles_range[1])
    angles = angles / 360.0 * 2.0 * np.pi
    x_preprocess = tfa.image.rotate(images=x, angles=angles, interpolation="NEAREST", name=None)  
    x_preprocess = tf.clip_by_value(
        t=x_preprocess, clip_value_min=-self.clip_values[0], clip_value_max=self.clip_values[1], name=None
    )
    return x_preprocess, y

IN THEORY, I could simply change the _transform() function to make the transformation I want.

However, such a function is not used to rotate the images, I can even erase the function that nothing happens if I rerun the algorithm. I even created a python for loop to open all python files in the art library and search for a keyword rotate but even commenting these entries the algorithm runs without problems

Therefore, I believe the rotation happens somewhere I don't have the minimum idea where it is.. Is that any way of me doing reverse engineering and finding exactly where the rotation is performed?

Answer 1

One option is to add a deliberate error into the function, such as 0/0 (division by zero). That will tell you whether it's called, and where from...