Microsoft CNTK Automatic Differentiation

123 views Asked by At

According to Microsoft, CNTK includes automatic differentiation. For better understanding the source (which I've successfully built) I'd like to know which C++ classes implement AD and how it is implemented in CNTK?

1

There are 1 answers

0
papadoble151 On

CNTK class Function implements the AD (via Gradients method, to be precise). Neural networks are represented as multiple Function compositions like g(f(x)). Then derivative of function g is computed with respect to f like this:

                                                                     derivative