Joining/Combining two models for Transfer Leaning in KERAS

Question

How can we join/combine two models in Transfer Leaning in KERAS?

I have two models: model 1 = My Model model 2 = Trained Model

I can combine these models by putting the model 2 as input and then passed its output to the model 1, which is the conventional way.

However, I am doing it in other way. I want to put the model 1 as input and then passed its output to the model 2 (i.e. trained model one).

Answer 1

It's exactly the same procedure, just make sure that your model's output has the same shape as the other model's input.

from keras.models import Model

output = model2(model1.outputs)
joinedModel = Model(model1.inputs,output)

Make sure (if that's what you want), to make all layers from model 2 have trainable=False before compiling, so the training will not change the already trained model.

---

Test code:

from keras.layers import *
from keras.models import Sequential, Model

#creating model 1 and model 2 -- the "previously existing models"
m1 = Sequential()
m2 = Sequential()
m1.add(Dense(20,input_shape=(50,)))
m1.add(Dense(30))
m2.add(Dense(5,input_shape=(30,)))
m2.add(Dense(11))

#creating model 3, joining the models 
out2 = m2(m1.outputs)
m3 = Model(m1.inputs,out2)

#checking out the results
m3.summary()

#layers in model 3
print("\nthe main model:")
for i in m3.layers:
    print(i.name)

#layers inside the last layer of model 3
print("\ninside the submodel:")
for i in m3.layers[-1].layers:
    print(i.name)

Output:

Layer (type)                 Output Shape              Param #   
=================================================================
dense_21_input (InputLayer)  (None, 50)                0         
_________________________________________________________________
dense_21 (Dense)             (None, 20)                1020      
_________________________________________________________________
dense_22 (Dense)             (None, 30)                630       
_________________________________________________________________
sequential_12 (Sequential)   (None, 11)                221       
=================================================================
Total params: 1,871
Trainable params: 1,871
Non-trainable params: 0
_________________________________________________________________

the main model:
dense_21_input
dense_21
dense_22
sequential_12

inside the submodel:
dense_23
dense_24

Answer 2

The issue has been resolved.

I used the model.add()function and then added all the required layers of both Model 1 and Model 2.

The following code would add the first 10 layers of Model 2 just after the Model 1.

for i in model2.layers[:10]: model.add(i)