I'm creating a neural network using a sigmoid activation function and my weight adjusting function is not working. Weights are randomly initialized from -1 to 1 and as the network trains the values are getting out of range.

Weights are stored in an adjacency matrix and each weight is adjusted with a loop and this formula my professor gave me to calculate the change in the weight(deltaW). When deltaW is added to the current weight I tried making it negative because my prof suggested trying that but there was no such luck. I have tested all other functions and they work fine so my issue must lie in here.

void adjustWeights(struct neuralNetwork* network_ptr, struct dataSet data){
    float stepsize = 0.1,
        summation = 0,
        deltaW = 0;

    for(int i=0; i<2; i++){//adjust weights between intermediate and output
        for(int j=0; j<3; j++){
           deltaW =stepsize*(network_ptr->outputNodes[i] - data.target[i]) * network_ptr->outputNodes[i]* (1 -network_ptr->outputNodes[i]) * network_ptr->intermediateNodes[j];
            //cout << "weight: " << network_ptr->intermediateToOutput[j][i] << "  deltaW:  " << deltaW;
            network_ptr->intermediateToOutput[j][i] += deltaW;
            //cout << "  new weight: " << network_ptr->intermediateToOutput[j][i] << endl;

    for(int i=0;i<3; i++){ //adjust weights between input and intermediate
        for(int j=0;j<3; j++){
            for(int k=0; k<2; k++)//this does the summation portion of the weight adjusment.
                summation =(network_ptr->outputNodes[k] - data.target[k]) * network_ptr->outputNodes[k]* (1 -network_ptr->outputNodes[k]) * network_ptr->intermediateNodes[j]*network_ptr->intermediateToOutput[j][k]
                            * network_ptr->intermediateNodes[j]* (1- network_ptr->intermediateNodes[j]) * network_ptr->inputNodes[i];
                network_ptr->inputToIntermediate[i][j] += stepsize*summation;

As the weights are adjusted they just keep growing for some reason. One time I ran it and a weight got up to 7.8. I sent the entire project to my Prof and He said everything appears to be correct. So I am completely dumbfounded.

Any advice is greatly appreciated!

1 Answers

Shieldfoss On

Line 10:

network_ptr->intermediateToOutput[j][i] += deltaW;

should be [i][j] instead.