Keras, Theano - Cropping2D introduces "Input dimension mis-match" on GPU

Question

I tried to create an autoencoder using keras. (ref: https://blog.keras.io/building-autoencoders-in-keras.html)

the following test code works just fine on my mac:

import numpy as np
from keras.layers import Input, Dense, Convolution2D, MaxPooling2D, UpSampling2D, Cropping2D, ZeroPadding2D
from keras.models import Model

input_img = Input(shape=(1, 30, 32))

testx = np.random.rand(10,1,30,32)

x = ZeroPadding2D((1, 0))(input_img)
x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)

x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Cropping2D(cropping=((1,1),(0,0)))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)

autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
autoencoder.fit(testx, testx, nb_epoch = 5, batch_size =1)

However, when I test this piece of code on AWS EC2 g2.2xlarge (my configuration info is on the end of this post), I run into the following Error:

Using Theano backend.
Using gpu device 0: GRID K520 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5005)
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_1 (InputLayer)             (None, 1, 30, 32)     0                                            
____________________________________________________________________________________________________
zeropadding2d_1 (ZeroPadding2D)  (None, 1, 32, 32)     0           input_1[0][0]                    
____________________________________________________________________________________________________
convolution2d_1 (Convolution2D)  (None, 16, 32, 32)    160         zeropadding2d_1[0][0]            
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D)    (None, 16, 16, 16)    0           convolution2d_1[0][0]            
____________________________________________________________________________________________________
convolution2d_2 (Convolution2D)  (None, 8, 16, 16)     1160        maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
maxpooling2d_2 (MaxPooling2D)    (None, 8, 8, 8)       0           convolution2d_2[0][0]            
____________________________________________________________________________________________________
convolution2d_3 (Convolution2D)  (None, 8, 8, 8)       584         maxpooling2d_2[0][0]             
____________________________________________________________________________________________________
maxpooling2d_3 (MaxPooling2D)    (None, 8, 4, 4)       0           convolution2d_3[0][0]            
____________________________________________________________________________________________________
convolution2d_4 (Convolution2D)  (None, 8, 4, 4)       584         maxpooling2d_3[0][0]             
____________________________________________________________________________________________________
upsampling2d_1 (UpSampling2D)    (None, 8, 8, 8)       0           convolution2d_4[0][0]            
____________________________________________________________________________________________________
convolution2d_5 (Convolution2D)  (None, 8, 8, 8)       584         upsampling2d_1[0][0]             
____________________________________________________________________________________________________
upsampling2d_2 (UpSampling2D)    (None, 8, 16, 16)     0           convolution2d_5[0][0]            
____________________________________________________________________________________________________
convolution2d_6 (Convolution2D)  (None, 16, 16, 16)    1168        upsampling2d_2[0][0]             
____________________________________________________________________________________________________
upsampling2d_3 (UpSampling2D)    (None, 16, 32, 32)    0           convolution2d_6[0][0]            
____________________________________________________________________________________________________
cropping2d_1 (Cropping2D)        (None, 16, 30, 32)    0           upsampling2d_3[0][0]             
____________________________________________________________________________________________________
convolution2d_7 (Convolution2D)  (None, 1, 30, 32)     145         cropping2d_1[0][0]               
====================================================================================================
Total params: 4,385
Trainable params: 4,385
Non-trainable params: 0
____________________________________________________________________________________________________
Epoch 1/5
Traceback (most recent call last):
  File "testtest.py", line 32, in <module>
    autoencoder.fit(testx, testx, nb_epoch = 5, batch_size =1)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1144, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 844, in _fit_loop
    outs = f(ins_batch)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 953, in __call__
    return self.function(*inputs)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 886, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 873, in __call__
    self.fn() if output_subset is None else\
ValueError: Input dimension mis-match. (input[0].shape[3] = 32, input[1].shape[3] = 0)
Apply node that caused the error: Elemwise{mul,no_inplace}(convolution2d_7_target, Elemwise{log,no_inplace}.0)
Toposort index: 703
Inputs types: [TensorType(float32, 4D), TensorType(float32, 4D)]
Inputs shapes: [(1, 1, 30, 32), (1, 1, 30, 0)]
Inputs strides: [(3840, 3840, 128, 4), (120, 120, 4, 4)]
Inputs values: ['not shown', array([], shape=(1, 1, 30, 0), dtype=float32)]
Outputs clients: [[Elemwise{add,no_inplace}(Elemwise{mul,no_inplace}.0, Elemwise{mul,no_inplace}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "testtest.py", line 30, in <module>
    autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 619, in compile
    sample_weight, mask)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 307, in weighted
    score_array = fn(y_true, y_pred)
  File "/usr/local/lib/python2.7/dist-packages/keras/objectives.py", line 49, in binary_crossentropy
    return K.mean(K.binary_crossentropy(y_pred, y_true), axis=-1)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 1235, in binary_crossentropy
    return T.nnet.binary_crossentropy(output, target)

I can run the code on GPU if the Cropping2D function was removed:

import numpy as np
from keras.layers import Input, Dense, Convolution2D, MaxPooling2D, UpSampling2D, Cropping2D, ZeroPadding2D
from keras.models import Model

input_img = Input(shape=(1, 30, 32))

testx = np.random.rand(10,1,30,32)
testy = np.random.rand(10,1,32,32)

x = ZeroPadding2D((1, 0))(input_img)
x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)

# at this point the representation is (8, 4, 4) i.e. 128-dimensional

x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
# x = Cropping2D(cropping=((1,1),(0,0)))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)

autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
autoencoder.summary()
autoencoder.fit(testx, testy, nb_epoch = 5, batch_size =1)

In a similar scenario, Cropping3D also raise the similar error... :-(

My question is:

How can I solve this and run Cropping2D/Cropping3D on GPU?

ps. My AWS EC2 g2.2xlarge configuration is as followed:

• Ubuntu 14.04
• Cuda 7.5
• Cudnn v5
• Keras (1.2.0)
• Theano (0.9.0dev4)

Thank you!

Answer 1

There appears to be a bug in Keras. It has been taking care of. Case closed!