I'm training my model using Keras, here is my code:

    model = keras.Sequential([
      keras.layers.Dense(len(COLUMN_NAMES), activation=tf.nn.sigmoid, use_bias=True),
      keras.layers.Dense(256, activation=tf.nn.relu),
      keras.layers.Dense(64, activation=tf.nn.relu),
      keras.layers.Dense(split_num, activation=tf.nn.softmax)
    ])
    model.compile(optimizer=tf.train.AdamOptimizer(),
          loss=focal_loss(gamma=2, alpha=0.85714),
          metrics=['accuracy'])
    model.fit(x_train.values, y_train.values, epochs=5, batch_size=2000, verbose=1)
    test_loss, test_acc = model.evaluate(x_test.values, y_test.values)

My y_test has 0:85%; 1:15%, It's output:

    32321/32321 [==============================] - 1s 40us/sample - loss: 0.0608 - acc: 1.0000

But when I checked it using:predictions = model.predict(x_test.values, verbose=1), it predicted nearly all cases to 0. So, in my opinion the ['accuracy'] could be 85% instead of "acc: 1.0000". What's going wrong ?

1 Answers

0
Zaccharie Ramzi On

I think there are several things you are confused about here:

  • Firstly, "acc: 1.0000" indicates an accuracy of 1.0 which is equivalent to 100%.
  • Secondly, it is the accuracy on your training data, so it doesn't necessarily relate to the accuracy on your test data. Several things might have gone wrong. You may be overfitting because the model you are using is too big compared to the amount of available training data. You may also be fitting something that doesn't have the same distribution, i.e. y_train is all 0s. We can't help you further if you don't specify more what your data is