How to evaluate a recurrent connection in an artificial neural network?

Question

I just can't understand how should I compute the output of a neural network, which contains a recurrent connection.

So here is an example (I can't post images yet..): https://i.stack.imgur.com/ngEbq.png

(i_1,2 are the input values, w_1,2,3,r are the connection weights, and o_1 is the output value.)

For the sake of simplicity, let's say that there are no activation or transfer functions. If I understand the workings of ANNs correctly, then in case of not taking the red recurrent connection into consideration, the output is calculated as

o_1=(w_1*i_1+w_2*i_2)*w_3

However, what is the case when the red connection is taken into account? Would it be

o_1=((w_1*i_1+w_2*i_2)+(w_1*i_1+w_2*i_2)*w_r)*w_3

maybe? But that's just my guess.

Thanks in advance.

Answer 1

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs.

To me, it seems like :

o_1=(w_1*i_1+w_2*i_2)*w_r*w_3

Note: Please note if this is homework.

Answer 2

The RNN is not a usual network. The usual network has no time, but RNN has time. The digitalized signals go to the input of the net. So for example, for i_1 we have not one value, but signal i_1[t=0], i_1[t=1],i_1[t=2], … The red connection has delay inside itself and the delay is one unit of time. Thus, to calculate the output of the H1 you need to use the following recurrent formula:

o[t]=w_1*i_1[t]+w_2*i_2[t])+o[t-1]*w_r

You see here o[t-1] that means delay at one unit of time.

Speaking about recurrent neural networks, you may find many examples of using it. Recently we have participated in machine learning contest and tried to use the RNN for classification of EEG signals, but faced with some obstacles. Here are the details: http://rnd.azoft.com/classification-eeg-signals-brain-computer-interface/.