How to compute GradCAM for keras regression model with multiple outputs

60 views Asked by At

I have build a DenseNet121 that has 5 regression output layers.

The code to build the model looks like this:

base_model = DenseNet121(include_top=False, input_shape=(224, 224, 3), weights="imagenet", pooling="avg")
x = base_model.output
F1 = Dense(1, activation=custom_activation, name='F1')(x)
F2 = Dense(1, activation=custom_activation, name='F2')(x)
...
model = Model(inputs=base_model.input, outputs=[F1, F2, F3, F4, F5])

Each output layer stands for a certain feature that we can observe in the image. The output layers will output values between 0 and 3. The prediction of the model has the following structure: [array([[1.2791169]], dtype=float32), array([[1.3784188]], dtype=float32), array([[0.6047046]], dtype=float32), array([[2.1661398]], dtype=float32), array([[1.6510094]], dtype=float32)]

How can we build a GradCam for this model? Furthermore, is it possible to only see the activation for just one feature?

I first tried to work with keras implementation of GradCam. However, I am not sure how to define the score. I tried:

gradcam = GradcamPlusPlus(model, model.layers[0])
results = model.predict(normalize(image))
score = [CategoricalScore([1]), InactiveScore(), InactiveScore(), InactiveScore(), InactiveScore()]
cam = gradcam(score, normalize(image), penultimate_layer=-1)

I used InactiveScore() so that we can only focus on one feature. But I get the error message "Invalid index value. indices: [1], output.shape: (1, 1)"

Can you help me?

0

There are 0 answers