According to Microsoft, CNTK includes automatic differentiation. For better understanding the source (which I've successfully built) I'd like to know which C++ classes implement AD and how it is implemented in CNTK?
CNTK class Function implements the AD (via Gradients method, to be precise). Neural networks are represented as multiple Function compositions like g(f(x)). Then derivative of function g is computed with respect to f like this:
CNTK class
Functionimplements the AD (via Gradients method, to be precise). Neural networks are represented as multipleFunctioncompositions like g(f(x)). Then derivative of function g is computed with respect to f like this: