Error with adding a CNN layer to LSTM pytorch model for time series forecasting

Question

I have a Pytorch LSTM model for multivariate timeseries forecasting. It works fine. But I want to add a CNN layer to the model. My problem that I get an error when I add the CNN layer as I couldn't figure out how to prepare the diminsions correctly. This is the class:

class TorchCNN_LSTM(nn.Module):
    def __init__(self, input_dim, hidden_dim, h2 ,layer_dim, output_dim, dropout_prob):
        super(TorchCNN_LSTM, self).__init__()
        .....
        # CNN layer
        self.conv = nn.Conv1d(input_dim, output_dim,
                              kernel_size=(3,), padding=2)
        # LSTM layers
        self.lstm = nn.LSTM(
            input_dim, hidden_dim, layer_dim, batch_first=True, dropout=dropout_prob
        )
        # Fully connected layer
        self.fc_1 = nn.Linear(hidden_dim, h2)  # fully connected
        self.fc_2 = nn.Linear(h2, output_dim)  # fully connected last layer
        self.relu = nn.ReLU()

    def forward(self, x):

        #this is my trying to reshap. but id doesn't work.
        #x_out= x.view(len(x), 1, -1)
        #and
        #x = x.permute(0,2,1)

        # Initializing hidden state for first input with zeros
        h0 = torch.zeros(self.layer_dim, x.size(0), self. hidden_dim , device = x.device ). 
             requires_grad_ ( ) 
        # Initializing cell state for first input with zeros
        c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim, 
             device=x.device).requires_grad_()

        out = self.conv(x_out)
        out = self.lstm(out, (h0.detach(), c0.detach()))
        out = out[:, -1, :]
        out = self.fc_1(out)  # first dense
        out = self.relu(out)  # relu
        out = self.fc_2(out)  # final output
        return out

The x input is a torch of Size([16, 15, 2]), where 16 is the batch, 15 is the timestep, and the 2 is the feature numbers. Suppose I want to have 1 or 3 horizon future prediction. I got this error: RuntimeError: Given groups=1, weight of size [1, 2, 3], expected input[16, 15, 2] to have 2 channels, but got 15 channels instead. I have tried to add some reshape code like:

#x_out= x.view(len(x), 1, -1)
#or
#x = x.permute(0,2,1)

Can anyone help please?

Answer 1

I think you're misunderstanding the dimensions in 1D convolutions.

For a tensor of size (bs, ch, n), the axes denote batch size, channels, and number of features. A 1D conv operates over the channel dimension. If your input is of size (16, 15, 2), this means your input has 15 channels with 2 features per channel.

In this case, the appropriate 1D convolution would have a input_dim size of 15.

For example:

layer = nn.Conv1d(15, 30, kernel_size=(3,), padding=1, stride=2)
input = torch.randn(16, 15, 32)
output = layer(input)
output.shape
>torch.Size([16, 30, 16])

Note that the parameters for kernel size, padding and stride control the output size of the final dimension. In this case, kernel_size=(3,), padding=1, stride=2 reduces the feature dimension from 32 to 16

Note also that for time series, the ch dimension tends to be the number of timesteps. 1D convolutions are fixed in the number of timesteps they can take in. You will need to account for this in your data processing.