tensorflow multivariable seq 2 seq model return only lagged forcast

22 views Asked by At

Using this simple model, I try to predict the missing values (future data).

past_input = tf.keras.Input(shape=(None, x_past.shape[2]), name='past_inputs')
forcast_input = tf.keras.Input(shape=(None, x_forcast.shape[2]), name='forcast_inputs')
forcast_masked = tf.keras.layers.Masking(mask_value=-9999)(forcast_input)

encoder_last_h1, encoder_last_h2, encoder_last_c = tf.keras.layers.LSTM(60,  return_sequences=False, 
                          return_state=True,
                          name='encoder')(past_input) 
 
decoder = tf.keras.layers.LSTM(60, activation='tanh',
                          kernel_initializer= tf.keras.initializers.glorot_uniform(),
                          return_state=False, 
                          return_sequences=False,
                          name='decoder')(forcast_masked, initial_state=[encoder_last_h1, encoder_last_c])

out = tf.keras.layers.Dense(NB_OUTPUTS, 
                            activation='linear',
                            kernel_initializer= tf.keras.initializers.he_uniform(), 
                            name='Output')(decoder) 


model = tf.keras.Model(inputs=[past_input, forcast_input], outputs=out)

tf.keras.utils.plot_model(model, show_shapes=True, show_layer_names=True, to_file='architecture.png')

optimizer = tf.keras.optimizers.legacy.Adam(learning_rate=0.001)

early_stop = tf.keras.callbacks.EarlyStopping(monitor=['loss', 'val_loss'], 
                                              patience=10, 
                                              verbose=1,
                                              mode='min', 

    
model.compile(optimizer=optimizer, loss='mae')

enter image description here

my input and output looklike this:

|--past inputs--|--futur/forcast input---|
x-3  x-2  x-1  x  x+1  x+2  x+3 ... x+10
.3   .2   .2  .1  .15  .2  .2   ...  .5
.3   .3  .35  .1  .2   .25  .15 ...  .3
.12  .13 .2   .2  -99  -99  -99      -99 <---- only this line as output too
                 |---------output-------|
where -99 are masked and must be replace by the predict values

When I try to predict after trainning, I only get a 10x lagged of the same inference.

enter image description here What is missing in my code

0

There are 0 answers