Python randomly crashes when using theano

Question

I am new to python and try to use lasagne and theano to enhance performance of machine learning by computing it on GPU (installed numpy+mlk). I call NeuralNet.fit(X, y) which sometimes proceeds successfully, but sometimes (and more often) crashes with open an alert window saying following:

"Python.exe has stopped working"

I can explore "problem details", where I can find "Fault Module Name: m16938b3c266c80e639557073d4dbcc9b.pyd"

Also, sometimes the trn loss and val loss are very huge (100 digits long number) and in some epoch, they turn to be "nan".

I use MNIST as the demo and for speed pick only the first 50 digits. Here the neural network structure:

## Layer information

  #  name     size
---  -------  --------
  0  input    1x28x28
  1  conv1    16x26x26
  2  conv2    16x24x24
  3  hidden4  500
  4  output   10

any ideas or suggestions? Thank you very much