Prev Next

Second Order Reverse Mode

Syntax
dw = f.Reverse(2, w)

Purpose
We use  F : B^n \rightarrow B^m to denote the AD function corresponding to f. Reverse mode computes the derivative of the Forward mode Taylor coefficients with respect to the domain variable  x .

x^(k)
For  k = 0, 1 , the vector  x^{(k)} \in B^n is defined as the value of x_k in the previous call (counting this call) of the form
     
f.Forward(kx_k)
If there is no previous call with  k = 0 ,  x^{(0)} is the value of the independent variables when the corresponding AD of Base operation sequence was recorded.

W
The functions  W_0 : B^n \rightarrow B and  W_1 : B^n \rightarrow B are defined by  \[
\begin{array}{rcl}
W_0 ( u ) & = & w_0 * F_0 ( u ) + \cdots + w_{m-1} * F_{m-1} (u)
\\
W_1 ( u ) & = & 
w_0 * F_0^{(1)} ( u ) * x^{(1)} 
     + \cdots + w_{m-1} * F_{m-1}^{(1)} (u) * x^{(1)}
\end{array}
\] 
This operation computes the derivatives  \[
\begin{array}{rcl}
W_0^{(1)} (u) & = & 
     w_0 * F_0^{(1)} ( u ) + \cdots + w_{m-1} * F_{m-1}^{(1)} (u)
\\
W_1^{(1)} (u) & = & 
     w_0 * \left( x^{(1)} \right)^\T * F_0^{(2)} ( u ) 
     + \cdots + 
     w_{m-1} * \left( x^{(1)} \right)^\T F_{m-1}^{(2)} (u)
\end{array}
\] 
at  u = x^{(0)} .

f
The object f has prototype
     const ADFun<
Basef
Before this call to Reverse, the value returned by
     
f.size_taylor()
must be greater than or equal two (see size_taylor ).

w
The argument w has prototype
     const 
Vector &w
(see Vector below) and its size must be equal to m, the dimension of the range space for f.

dw
The result dw has prototype
     
Vector dw
(see Vector below). It contains both the derivative  W^{(1)} (x) and the derivative  U^{(1)} (x) . The size of dw is equal to  n \times 2 , where  n is the dimension of the domain space for f.

First Order Partials
For  j = 0 , \ldots , n - 1 ,  \[
dw [ j * 2 + 0 ] 
=  
\D{ W_0 }{ u_j } \left( x^{(0)} \right) 
=
w_0 * \D{ F_0 }{ u_j } \left( x^{(0)} \right)
+ \cdots + 
w_{m-1} * \D{ F_{m-1} }{ u_j } \left( x^{(0)} \right)
\] 
This part of dw contains the same values as are returned by reverse_one .

Second Order Partials
For  j = 0 , \ldots , n - 1 ,  \[
dw [ j * 2 + 1 ] 

\D{ W_1 }{ u_j } \left( x^{(0)} \right) 
=
\sum_{\ell=0}^{n-1} x_\ell^{(1)} \left[
w_0 * \DD{ F_0 }{ u_\ell }{ u_j } \left( x^{(0)} \right)
+ \cdots + 
w_{m-1} * \DD{ F_{m-1} }{ u_\ell }{ u_j } \left( x^{(0)} \right)
\right]
\] 


Vector
The type Vector must be a SimpleVector class with elements of type Base. The routine CheckSimpleVector will generate an error message if this is not the case.

Hessian Times Direction
Suppose that  w is the i-th elementary vector. It follows that for  j = 0, \ldots, n-1  \[
\begin{array}{rcl}
dw[ j * 2 + 1 ] 
& = & 
w_i \sum_{\ell=0}^{n-1} 
\DD{F_i}{ u_j }{ u_\ell } \left( x^{(0)} \right) x_\ell^{(1)} 
\\
& = &
\left[ F_i^{(2)} \left( x^{(0)} \right) * x^{(1)} \right]_j
\end{array}
\] 
Thus the vector  ( dw[1], dw[3], \ldots , dw[ n * p - 1 ] ) is equal to the Hessian of  F_i (x) times the direction  x^{(1)} . In the special case where  x^{(1)} is the l-th elementary vector ,  \[
dw[ j * 2 + 1 ] = \DD{ F_i }{ x_j }{ x_\ell } \left( x^{(0)} \right)
\] 


Example
The files reverse_two.cpp and HesTimesDir.cpp contain a examples and tests of reverse mode calculations. They return true if they succeed and false otherwise.
Input File: omh/reverse.omh