Introduction
ADiMat
Example and Results
Wrap up and outlook
On the way to compute derivatives in M ATLAB
efficiently
Andre Vehreschild
[email protected]
Institute for Scientific Computing, RWTH Aachen University,
Aachen, Germany
Seventh European Workshop on Automatic Differentiation
November 24 - 25, 2008
Oxford-Man Institute of Quantitative Finance
Oxford University, Oxford, UK
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Outline
1
Introduction
2
ADiMat
Class mode
Scalar directional derivatives mode
3
Example and Results
Lighthouse example
Performance comparison
4
Wrap up and outlook
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
M ATLAB
AD for M ATLAB
Introduction
M ATLAB
M ATLAB:
high-level objects
matrices, vectors, scalars
multi-dimensional arrays
classes, . . .
high-level operators
matrix-matrix-product
backslash operator, . . .
implicitly typed
static analysis of precise type is complicated
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
M ATLAB
AD for M ATLAB
Introduction
AD for M ATLAB
AD for M ATLAB
operator overloading techniques
implemented in ADMIT-1/ADMAT, TOMLAB/MAD, myAD
a priori computation of types is unnecessary, because
types are available at runtime
source transformation techniques
implemented in ADiMat, MSAD
analyses structure of the program
computes variables dependencies
adds new statements or rewrites old statements to
efficiently compute derivatives
may apply code optimizations techniques
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
M ATLAB
AD for M ATLAB
Introduction
AD for M ATLAB
AD for M ATLAB
operator overloading techniques
implemented in ADMIT-1/ADMAT, TOMLAB/MAD, myAD
a priori computation of types is unnecessary, because
types are available at runtime
source transformation techniques
implemented in ADiMat, MSAD
analyses structure of the program
computes variables dependencies
adds new statements or rewrites old statements to
efficiently compute derivatives
may apply code optimizations techniques
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Class mode
Scalar directional derivatives mode
ADiMat
Class mode
Compute multiple directional derivatives during one evaluation
of the differentiated program:
+ evaluate differentiated program once only
+ datatype for storing directional derivatives may be
forced to sparse
- additional indirection induced by runtime system
- many additional files are needed
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Class mode
Scalar directional derivatives mode
ADiMat
Scalar directional derivatives mode
Compute one directional derivative during multiple evaluations
of the differentiated program:
+ no additional indirection induced, because no
class needed
(JIT compiler may optimize computation)
- evaluate differentiated program multiple times
- computes just one directional derivative during an
execution of the program
+ number of additional files is small
(May be reduced further by more sophisticated
support in source transformation component)
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example
f u n c t i o n y= l i g h t h o u s e ( nu , gamma, omega , t )
% An o p t i m i z e d v e r s i o n o f t h e l i g h t h o u s e example from t h e book :
% A . Griewank ” E v a l u a t i n g D e r i v a t i v e s ” SIAM 2000 p . 17
v= t a n ( omega∗ t ) ;
y ( 1 ) = ( nu ∗ v ) / ( gamma−v ) ;
y ( 2 ) = gamma∗ y ( 1 ) ;
Differentiate to obtain Jacobian.
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example
f u n c t i o n y= l i g h t h o u s e ( nu , gamma, omega , t )
% An o p t i m i z e d v e r s i o n o f t h e l i g h t h o u s e example from t h e book :
% A . Griewank ” E v a l u a t i n g D e r i v a t i v e s ” SIAM 2000 p . 17
v= t a n ( omega∗ t ) ;
y ( 1 ) = ( nu ∗ v ) / ( gamma−v ) ;
y ( 2 ) = gamma∗ y ( 1 ) ;
Differentiate to obtain Jacobian.
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Differentiated lighthouse example
f u n c t i o n [ g y , y ] = g l i g h t h o u s e ( g nu , nu , g gamma , gamma, . . .
g omega , omega , g t , t )
% A . Griewank ” E v a l u a t i n g D e r i v a t i v e s ” SIAM 2000 p . 17
tmp 0= omega∗ t ;
g v = ( g omega∗ t + omega∗ g t ) . ∗ sec ( tmp 0 ) . ˆ 2 ;
v= t a n ( tmp 0 ) ;
c l e a r tmp 0 ;
g tmp 1= g nu ∗ v+ nu∗ g v ;
g tmp 2= g gamma− g v ;
tmp 2= gamma− v ;
y ( 1 ) = ( nu∗ v ) / tmp 2 ;
g y ( 1 ) = ( ( tmp 2 ) ’ \ ( ( g tmp 1 ) ’ − ( g tmp 2 ) ’ ∗ y ( 1 ) ’ ) ) ’ ;
c l e a r g tmp 2 tmp 2 g tmp 1 ;
g y ( 2 ) = g gamma∗ y ( 1 ) + gamma∗ g y ( 1 ) ;
y ( 2 ) = gamma∗ y ( 1 ) ;
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Driver for class mode
n=
g=
o=
t=
1 0 . 0 ; % (m)
0.375∗ p i ; % ( rad )
0.0001∗ p i ; % ( rad )
2.0; % (s)
[ g n , g g , g o , g t ]= createFullGradients ( n , g , o , t ) ;
[ g y , y ]= g lighthouse ( g n , n , g g , g , g o , o , g t , t ) ;
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example using class mode
...
tmp 0= omega∗ t ;
g v = ( g omega∗ t + omega∗ g t ) . ∗ sec ( tmp 0 ) . ˆ 2 ;
v= t a n ( tmp 0 ) ;
c l e a r tmp 0 ;
g tmp 1= g nu ∗ v+ nu∗ g v ;
g tmp 2= g gamma− g v ;
tmp 2= gamma− v ;
...
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example using class mode
...
tmp 0= omega∗ t ;
g v = ( g omega∗ t + omega∗ g t ) . ∗ sec ( tmp 0 ) . ˆ 2 ;
v= t a n ( tmp 0 ) ;
c l e a r tmp 0 ;
g tmp 1= g nu ∗ v+ nu∗ g v ;
g tmp 2= g gamma− g v ;
tmp 2= gamma− v ;
...
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example using class mode
...
tmp 0= omega∗ t ;
g v = ( g omega∗ t + omega∗ g t ) . ∗ sec ( tmp 0 ) . ˆ 2 ;
v= t a n ( tmp 0 ) ;
c l e a r tmp 0 ;
g tmp 1= g nu ∗ v+ nu∗ g v ;
g tmp 2= g gamma− g v ;
tmp 2= gamma− v ;
...
Each operator ◦ executes:
function r=◦(u,v)
for c=1:ndd
r{c}=u{c}◦v; % or u◦v{c}
end
Andre Vehreschild
% or u{c}◦v{c}
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example using scalar directional derivatives mode
n= 1 0 . 0 ; g= 0.375∗ p i ; o= 0.0001∗ p i ; t = 2 . 0 ;
ndd =4;
seeding=eye ( ndd ) ; g y = zeros ( 2 , ndd ) ;
f o r d =1: ndd
[ g y ( : , d ) , y ] = g l i g h t h o u s e ( seeding ( 1 , d ) , n , seeding ( 2 , d ) , g , . . .
seeding ( 3 , d ) , o , seeding ( 4 , d ) , t ) ;
end
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Example
Lighthouse example using scalar directional derivatives mode
n= 1 0 . 0 ; g= 0.375∗ p i ; o= 0.0001∗ p i ; t = 2 . 0 ;
ndd =4;
seeding=eye ( ndd ) ; g y = zeros ( 2 , ndd ) ;
f o r d =1: ndd
[ g y ( : , d ) , y ] = g l i g h t h o u s e ( seeding ( 1 , d ) , n , seeding ( 2 , d ) , g , . . .
seeding ( 3 , d ) , o , seeding ( 4 , d ) , t ) ;
end
g lighthouse.m remains unchanged!
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Results
Performance comparison
Time impact
TAD
Torig ∗ndd
problem
brusselator
geo. modes
swirling flow
vanderm.
Andre Vehreschild
ndd
20
8
56
420
#diff
108
13
93
5
#undiff
260
78
295
32
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Results
Performance comparison
Time impact
TAD
Torig ∗ndd
problem
brusselator
geo. modes
swirling flow
vanderm.
Andre Vehreschild
ndd
20
8
56
420
#diff
108
13
93
5
#undiff
260
78
295
32
On the way to compute derivatives in M ATLAB efficiently
#diff
#undiff
0.41
0.17
0.32
0.16
Introduction
ADiMat
Example and Results
Wrap up and outlook
Lighthouse example
Lighthouse example using class mode
Lighthouse example using scalar directional derivatives mode
Performance comparison
Results
Performance comparison
Memory impact
MAD
Morig
problem
brusselator
geometric modes
swirling flow
vandermonde solve
Andre Vehreschild
ndd
20
8
56
420
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Wrap up and outlook
Wrap up:
Class mode and scalar directional derivatives mode can be
used to compute derivatives of M ATLAB programs.
If the ratio of differentiated to undifferentiated statements is
close to one or above, then the scalar directional
derivatives mode performs better.
Future work:
Better support for scalar directional derivatives:
create driver automatically, and
remove dependency on runtime functions.
Get rid of class and directly insert the “loops” in the
generated program.
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently
Introduction
ADiMat
Example and Results
Wrap up and outlook
Wrap up and outlook
Wrap up:
Class mode and scalar directional derivatives mode can be
used to compute derivatives of M ATLAB programs.
If the ratio of differentiated to undifferentiated statements is
close to one or above, then the scalar directional
derivatives mode performs better.
Future work:
Better support for scalar directional derivatives:
create driver automatically, and
remove dependency on runtime functions.
Get rid of class and directly insert the “loops” in the
generated program.
Andre Vehreschild
On the way to compute derivatives in M ATLAB efficiently