nolearn auto add hidden layers

101 views Asked by At

Trying to add more hidden layers to my neural network so I can compare the precision score of different layers using the same learn rates and momentum etc.

So I have the following:

from nolearn.dbn import DBN 
from sklearn.cross_validation import train_test_split

from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score

numFeat = 50 * 64 
labs = ["BC", "FL", "PRO1", "PRO2", "RP", "SD"]  # Defect types

hiddenAr = [numFeat, 6] # initial params for dbn
pos = len(hiddenAr) - 1

with open("all_defects.csv") as f:
reader = csv.reader(f)
for row in reader:
    input = [int(x) for x in row[:numFeat]]  
    labels += [labs.index(row[-1])]  # categorical labels in numerical form
    data += [input]

X_train, X_test, y_train, y_test = train_test_split(
data, labels, test_size=0.25) 

for i in range (0, layer):
    hiddenAr.insert(pos, 300) # Add hidden layer

    dbn = DBN(
    hiddenAr,
    # Learning rate of algorithm
    learn_rates = 0.03,
    # Decay of learn rate
    learn_rate_decays=1,
    # Iterations of training data (epochs)
    epochs=10,
    # Verbosity level
    verbose=1,
    momentum= 0.03,
    use_re_lu=True
    ) 

What I have done there, is just added another 300 nodes to another hidden layer each time. However, it spits out a number of npmat.py errors when I add the layers to it for some reason.

Is there an obvious reason for this? I really would like to just automate it to add hidden layers itself so I can generate graphs and csv files for evaluation easily.

The errors happen after the hidden array has the second layer added to it, i.e. when i = 1 and then for each layer added after:

npmat.py:433: RuntimeWarning: inv alid value encountered in add target.numpy_array[:] = vec.numpy_array + self.numpy_array

RuntimeWarning: inv alid value encountered in less target.numpy_array[:] = self.numpy_array < val

RuntimeWarning: inv alid value encountered in greater

npmat.py:969: RuntimeWarning: inv alid value encountered in multiply

Data: https://drive.google.com/file/d/0B12vhoNivII6My1GQ3E3T3JxQm8/view?usp=sharing

0

There are 0 answers