Trying to add more hidden layers to my neural network so I can compare the precision score of different layers using the same learn rates and momentum etc.
So I have the following:
from nolearn.dbn import DBN
from sklearn.cross_validation import train_test_split
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score
numFeat = 50 * 64
labs = ["BC", "FL", "PRO1", "PRO2", "RP", "SD"] # Defect types
hiddenAr = [numFeat, 6] # initial params for dbn
pos = len(hiddenAr) - 1
with open("all_defects.csv") as f:
reader = csv.reader(f)
for row in reader:
input = [int(x) for x in row[:numFeat]]
labels += [labs.index(row[-1])] # categorical labels in numerical form
data += [input]
X_train, X_test, y_train, y_test = train_test_split(
data, labels, test_size=0.25)
for i in range (0, layer):
hiddenAr.insert(pos, 300) # Add hidden layer
dbn = DBN(
hiddenAr,
# Learning rate of algorithm
learn_rates = 0.03,
# Decay of learn rate
learn_rate_decays=1,
# Iterations of training data (epochs)
epochs=10,
# Verbosity level
verbose=1,
momentum= 0.03,
use_re_lu=True
)
What I have done there, is just added another 300 nodes to another hidden layer each time. However, it spits out a number of npmat.py errors when I add the layers to it for some reason.
Is there an obvious reason for this? I really would like to just automate it to add hidden layers itself so I can generate graphs and csv files for evaluation easily.
The errors happen after the hidden array has the second layer added to it, i.e. when i = 1 and then for each layer added after:
npmat.py:433: RuntimeWarning: inv alid value encountered in add target.numpy_array[:] = vec.numpy_array + self.numpy_array
RuntimeWarning: inv alid value encountered in less target.numpy_array[:] = self.numpy_array < val
RuntimeWarning: inv alid value encountered in greater
npmat.py:969: RuntimeWarning: inv alid value encountered in multiply
Data: https://drive.google.com/file/d/0B12vhoNivII6My1GQ3E3T3JxQm8/view?usp=sharing