!!NOT ABLE TO EXPLAIN WITHOU PICTURES!! I have a neural network, which i feed with text news, encoded with BagOfWOrds and labels {0,1}.I classify the text with convolutions. Everything seems to be working, but, in case that i have quoit a little data(20 000 news), the accuracy on train is not converging and accuracy on test is wierd itself(X axis is batches, green is test, blue is train):
Then, I visualized the wieghts of every layer in my NN and it suprised and confused me even more(X axis is batches):
And
1. I really cant explain, why weights converges at 20th batch and dont change later, while, according to accuracy, they were supposed to!!
2. And why the test accuracy's behavior is so strange(greeen line) I hope this is just code...
topic_input = lasagne.layers.InputLayer(shape=(None, v_train.shape[1]), input_var=v_t)
embedding = lasagne.layers.EmbeddingLayer(topic_input, input_size=len(token_to_id)+1, output_size=32)
WHAT = lasagne.layers.DimshuffleLayer(embedding, [0,2,1])
conv_1 = lasagne.layers.Conv1DLayer(WHAT, num_filters=15, filter_size=4)
conv_2 = lasagne.layers.Conv1DLayer(conv_1, num_filters=5, filter_size=3)
dense_1 = lasagne.layers.DenseLayer(conv_2, 30)
dense_2 = lasagne.layers.DenseLayer(dense_1, 5)
dense_3 = lasagne.layers.DenseLayer(dense_2, 1, nonlinearity=lasagne.nonlinearities.sigmoid)
weights = lasagne.layers.get_all_params(dense_3,trainable=True)
prediction = lasagne.layers.get_output(dense_3)
loss = lasagne.objectives.binary_crossentropy(prediction, target).mean()
updates = lasagne.updates.adam(loss, weights, learning_rate=0.01)
accuracy = lasagne.objectives.binary_accuracy(prediction, target).mean()
train_func = theano.function([v_t, target], [loss, prediction, accuracy, weights[7]], updates=updates)
acc_func = theano.function([v_t, target], [accuracy, prediction])