Tensorflow with Keras: ValueError - expected dense_84 to have 2 dimensions, but got array with shape (100, 9, 1)

Question

I am trying to use Tensorflow through Keras to build a network that uses time-series data to predict the next value, but I'm getting this error:

ValueError: Error when checking target: expected dense_84 to have 2 dimensions, but got array with shape (100, 9, 1)

What is causing this? I've tried reshaping the data as other posts have suggested, but to no avail so far. Here is the code:

import keras
import numpy as np
import os
from keras import losses
from keras.models import Sequential
from keras.layers import Dense, Activation, Flatten, Dropout
from keras.layers.convolutional import Conv1D, Conv2D

# add the desktop to our path so we can access the data
os.path.join("C:\\Users\\user\\Desktop")
# import data
data = np.genfromtxt("C:\\Users\\user\\Desktop\\aapl_blocks_10.csv",
                     delimiter=',')
# separate into inputs and outputs
X = data[:, :9]
X = np.expand_dims(X, axis=2) # reshape (409, 9) to (409, 9, 1) for network
Y = data[:, 9]

# separate into test and train data
X_train = X[:100]
X_test = X[100:]
Y_train = Y[:100]
Y_test = Y[100:]

# set parameters
batch_size = 20;

# define model
model = Sequential()
model.add(Conv1D(filters=20,
                 kernel_size=5,
                 input_shape=(9, 1),
                 padding='causal'))
model.add(Flatten())
model.add(Dropout(rate=0.3))
model.add(Dense(units=10))
model.add(Activation('relu'))
model.add(Dense(units=1))
model.compile(loss=losses.mean_squared_error,
              optimizer='sgd',
              metrics=['accuracy'])

# train model
model.fit(X_train, Y_train, epochs=10, batch_size=batch_size)

# evaluate model
model.evaluate(X_test, Y_test, batch_size=batch_size)

And here is the model summary:

Layer (type)                 Output Shape              Param #
=================================================================
conv1d_43 (Conv1D)           (None, 9, 20)             120
_________________________________________________________________
flatten_31 (Flatten)         (None, 180)               0
_________________________________________________________________
dropout_14 (Dropout)         (None, 180)               0
_________________________________________________________________
dense_83 (Dense)             (None, 10)                1810
_________________________________________________________________
activation_29 (Activation)   (None, 10)                0
_________________________________________________________________
dense_84 (Dense)             (None, 1)                 11
=================================================================
Total params: 1,941
Trainable params: 1,941
Non-trainable params: 0

If there's a proper way to be formatting the data, or maybe a proper way to stack these layers, I would love to know.

Answer 1

I suspect you need to squeeze the channel dimension from the output, i.e. the labes are shape (batch_size, 9) and you're comparing that against the output of a dense layer with 1 channel which has size (batch_size, 9, 1). Solution: squeeze/flatten before calculating the loss.

...
model.add(Activation('relu'))
model.add(Dense(units=1))
model.add(Flatten())
model.compile(loss=losses.mean_squared_error,
              optimizer='sgd',
              metrics=['accuracy'])

A note on squeeze vs Flatten: in this case, the result of squeezing (removing an axis of dimension 1) and flattening (making something of shape (batch_size, n, m, ...) into shape (batch_size, nm...) will be the same. Squeeze might be slightly more appropriate in this case, since if you accidentally squeeze an axis without dimension 1 you'll get an error (a good thing), as opposed to having your program run with unexpected behaviour. I don't use keras much though and couldn't find a 'Squeeze' layer - just a squeeze function - and I'm not entirely sure how to integrate it.