I am trying to create a simple neural network program/module. Because I am using Pythonista on an iPad Pro, the speed could use some improvement. My understanding is that for loops have a lot of overhead, so I was wondering if it is possible to train 50000 input:target sets using some form of vectorization.

I have been trying to find resources on how numpy arrays pass through functions, but it’s very difficult to wrap my head around. I have been trying to create one large array that holds the “train” function inputs as small lists, but the function is operating on the entire array, not the smaller ones individually.

# train function that accepts inputs and targets
# currently called inside a for loop
# input: [[a], [b]], target: [[c]]
def train(self, input_mat, target_mat):
        # generate hidden layer neuron values
        hidden = self.weights_in_hid.dot(input_mat)
        hidden += self.bias_hid
        hidden = sigmoid(hidden)
        # generate output neuron values
        output_mat = self.weights_hid_out.dot(hidden)
        output_mat += self.bias_out
        # activation function
        output_mat = sigmoid(output_mat)
# more of the function continues 

# ...
# Datum converts simple lists into numpy matrices, so they don’t have to be reinstantiated 50000 times
training_data = [
        Datum([0, 0], [0]), 
        Datum([0, 1], [1]), 
        Datum([1, 0], [1]), 
        Datum([1, 1], [0]), 
# ...
# XXX Does this for loop cause a lot of the slowdown?
for _ in range(50000):
        datum = rd.choice(training_data)        
        brain.train(datum.inputs, datum.targets)

In the shown state everything works, but somewhat slowly. Whenever I try to pass all the data in one matrix, the function cannot vectorize it. It instead attempts to operate on the matrix as a whole, obviously raising an error on the first step, as the array is way too big.

0 Answers