Accuracy/Model issue with Tensorflow/Keras DNN Regression Model

Question

I’m new to TF/Keras and have setup my first model with my own data to train it but am having an issue with the loss and accuracy. My model is a DNN using regression that is attempting to use 400 features to predict 1 label.

From the trials that I have ran, I have noticed that my epoch loss and val accuracy level out early which obviously results in my val accuracy being poor. I have run a range of epochs from 100-3000 with same result so my theory is that I have a bug in my code/poor model setup or I do not have enough data to train this model. Any help is appreciated. Here are my versions followed by the data sample and code

Versions are: Tensorflow 2.9.1 Python 3.9.12 Numpy 1.23.1 Pandas 1.4.3

Rawdata sample - Feature data are columns 0-400. Labels are 400-599, but this model is will only predict label "lower_pos_0" (column 400).

My code

import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.callbacks import TensorBoard
import datetime

print(tf.__version__)
# Make NumPy printouts easier to read.
np.set_printoptions(precision=3, suppress=True)

#read in the csv file into a dataframe. Sample file path "C:\\...\\...\\...\\csv name.csv"
filelocation = "C:\\(place the path to the csv here"
raw_dataset = pd.read_csv(filelocation)

#drop all columns that this model will NOT be used in training this model. Those columns are "lower_pos_1" through "lower_load_99"
raw_dataset = raw_dataset.drop(raw_dataset.loc[:, 'lower_pos_1':'lower_load_99'].columns, axis=1)
print('shape of raw_dataset:', raw_dataset.shape)
print(raw_dataset.head())

#convert the data frame to array
dataset = raw_dataset.copy()
dataset.tail()

#create a training and test set
train_dataset = dataset.sample(frac=0.8, random_state=0)
test_dataset = dataset.drop(train_dataset.index)

#check the data
train_dataset.describe().transpose()

#split features from labels
train_features = train_dataset.copy()
test_features = test_dataset.copy()

#print the shape
print('train features shape:',train_features.shape)
print(train_features.head())
print('test features shape:',test_features.shape)
print(test_features.head())

#drop the labels from the features. The "lower_pos_0" column is the label that the model is trying to predict.
train_labels = train_features.pop('lower_pos_0')
test_labels = test_features.pop('lower_pos_0')

#print the shape
print('train labels shape:',train_labels.shape)
print(train_labels.head())
print('test labels shape:',test_labels.shape)
print(test_labels.head())

#normalize the data using keras
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(np.array(train_features))
print(normalizer.mean.numpy())
first = np.array(train_features[:1])

with np.printoptions(precision=2, suppress=True):
  print('First example:', first)
  print()
  print('Normalized:', normalizer(first).numpy())

#build the model
def build_and_compile_model(norm):
  model = keras.Sequential([
      norm,
      layers.Dense(401, activation='relu'),
      layers.Dense(401, activation='relu'),
      layers.Dense(1)
  ])

  model.compile(loss='mean_absolute_error',
                optimizer=tf.keras.optimizers.Adam(0.001),
                 metrics='accuracy')
  return model

#display the models summary
dnn_model = build_and_compile_model(normalizer)
dnn_model.summary()

#setup tensorboard for viewing data during training

log_folder = "Traininglogs/"  + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")

callbacks = [TensorBoard(log_dir=log_folder,
                         histogram_freq=1,
                         write_graph=True,
                         write_images=True,
                         update_freq='epoch',
                         profile_batch=2,
                         embeddings_freq=1)]

#train the model with model.fit()
dnn_model.fit(train_features, train_labels, epochs=125, validation_split=0.2, callbacks=callbacks)

#evaluate the model with model.evaluate()
loss, mae = dnn_model.evaluate(test_features, test_labels, verbose=2)

#saving the model once completed
dnn_model.save('dnn_model')

Answer 1

I read through your coding. I made several observations and solutions which MAY help you out.

1. shuffling of data is needed in order to train the model to become more generic toward data. Therefore I request to look at the shuffling of data.

2. As you said the epochs I was around 100-3000. Actually, epochs were increased when you have large data.

3. Adding more epochs may make your model overfit. I request you to add a dropout layer

   tf.keras.layers.Dropout(rate, noise_shape=None, seed=None, **kwargs)

This may help you out.:)