Automatic differentiation — forward vs reverse

Forward (tangent) mode

Reverse (adjoint) mode