Hi I am a starter of using Torch's 'nn' package. In the past two weeks, I am extremely confused about the meaning of gradInput and gradOutput in the Torch's 'nn' library. I believe the 'grad' here means gradient, but what exactly are those two variables refers to?
Thanks for anyone's help!
gradOutput: gradient w.r.t. the output of the module. This is passed in either from the loss function, or from the module next to the current module. it is used to compute the gradient w.r.t. the input (gradInput) and gradient w.r.t. the parameters of the module (gradWeight / gradBias)