The Tensorflow model graph built with keras.utils.plot_model does not show concatenations

Question

I'm trying to build a dense block, so I wrote a simple example like this:

input_layer = Input(shape=(HEIGHT, WIDTH, 3))

layer1 = Conv2D(1, (3, 3), activation="relu", padding="same")(input_layer)
layer2 = Conv2D(2, (3, 3), activation="relu", padding="same")(layer1)
layer3 = Conv2D(3, (3, 3), activation="relu", padding="same")(layer2)
layer4 = Conv2D(4, (3, 3), activation="relu", padding="same")(layer3)

concatenate([layer3, layer2])

concatenate([layer4, layer3])
concatenate([layer4, layer2])

model = keras.Model(inputs=input_layer, outputs=layer4, name="Dense_block")
keras.utils.plot_model(model, "info.png", show_shapes=True)

But the graph I got does not contain any concatenations:

May be something is wrong with my code?

Answer 1

Layers 1 till 4 are connected and part of your model due to the call model = keras.Model(inputs=input_layer, outputs=layer4, name="Dense_block") with outputs=layer4. Your concatenate operations are not connected.

You could fix this by defining a new layer, for example layer5 = concatenate([layer3, layer2]) and pass that layer to outputs in your model statement.

For the concatenating itself, I would advice you to use Keras' Concatenate layer: tf.keras.layers.Concatenate. Reference here.

layer5 = tf.keras.layers.Concatenate()([layer3, layer2])
model = keras.Model(inputs=input_layer, outputs=layer5, name="Dense_block")