ADAPTIVE QUADRATURE IN THE DYNAMICS PROGRAM
Given sample data X, M factors, and model parameters Γ, consider the conditional
likelihood for a single agent’s experience:
L(Γ | X)  
R
M
K
 H

f
(
θ
)

k

  g h (θ) d M d M 1...d 2 d1 .
 k 1
  h1

(1.0)
The dynamics program minimizer navigates the likelihood function surface, changing
model parameters Γ at each step. Driven by surface geometry, model parameters vary in
ways that are unpredictable ex ante. We shall investigate evaluating the agent likelihood
along a path in parameter space in a case simplified for easy comprehension.
Step 1: Simplify the model for pedagogy.
1. Three states — start-state S is the lone decision point with two exits, states Z
and W. The zero-cost exit state is Z, the cost-exit state is W.
2. A single normal factor θ with density f1 (θ ) 
1    1 
.

 1   1 
(4.1)
(1)
3. Start-state S is a null state; i.e., no outcome is defined for state S. Let i in {2,3}
index states {W,Z}. States 2 and 3 each specify one outcome, earnings yi . Earnings
are specified by a linear equation including earnings shock  i ~ N(i , i ) .
Define f i (θ ) 
1  yi  xi  βi  αi θ  i 
 .

 i 
i

(4.2a)
Assume factor-loading αi is non-zero. Exploiting symmetry of  () :
y  x i  β i  i

  i
i
1
f i (θ ) 

 
i
i i 
i 
i
where  i 
yi  x i  β i   i
i
and  i 


  1 1      i  ,
  i  i   i 


(4.2b)
i
.
i
(2)
Adaptive Quadrature
1
7/13/2017
4. A single test T, outcome 4, is specified to determine the scale of the factor. The
test score designated y4 is specified by a linear equation with disturbance
 4 ~ N(4 , 4 ) . Assume α4  0 and define:
f 4 (θ ) 
where  4 
1  y 4  x 4  β 4  α4 θ   4  1 1     4 
 
 ,

 
 4 
4



4
4 
4


y4  x 4  β 4   4
4
and  4 
(4.3)
4
.
4
(3)
5. Finally, consider the transition probability for exit from state S. Since W and Z
are both absorbing states, the difference in exit-state systematic values is
R( )  VW  VZ  (xW  βW  αW θ )  (xC  βC  αC θ )  (x Z  β Z  αZ θ )
 (xW  βW  xC  βC  x Z  β Z )  (αW  αC  αZ )θ .
Define vectors x5  [xW xC x Z ] and β5  [βW  βC  β Z ] .
Define scalar α5  αW  αC  αZ . Then R( )  x5  β5  α5θ .
Psychic costs for exit to state W have disturbance  5 ~ N(5 , 5 ) .
The probability of exit to W is Pr{VW   5  VZ }  Pr{ 5  R( )} .
Transition probabilities for agent exit from start state S are:
  x 5  β 5  α5 θ   5 

if exit to W ;
 
5
 

g (θ )  
1   x 5  β 5  α5θ  5  if exit to Z ;



5


(4.4)
(4)
Adaptive Quadrature
2
7/13/2017
Given the five model specifications, the likelihood in (1.0) is written
4
 4 1  θ  k
1
L(Γ | X)   R  f k (θ ) g (θ )d 
  

 2 3 4 R  k 1  k   k
 k 1


 g (θ )d

(4.5)
whenever the outcome equation factor loadings {αk : k  2,3,4} are non-zero.
(5)
Step 2: Simplify the integrand
Lemma 1:
The product of two normal pdf’s is a constant times a normal pdf.
That is, given constants μ1 and μ2 and positive constants σ1 and σ2,
there exists constant μ* and positive constants σ* and γ such that
1    1  1     2      * 
 
  
.

 1   1   2   2   *   * 
Corollary 1: A finite product of normal densities is a constant times a normal
density. That is, given K > 1 constants { k : k  1,..., K } and K
positive constants { k : k  1,..., K } , there exists constant * and
positive constants  * and  such that
K
1
k 1
k

 θ   k      * 
   
 .
  k  *  * 
 
Proofs of both lemmas specify formulae for μ*, σ*, and γ.
(6)
When K = 4,
where
and
and
4
1
k 1
k

 θ   k      * 
   




k
* 
*



 
(6.0)
 22 32 42 1   12 32 42  2   12 22 42 3   12 22 32  4
,
* 
 12 22 32   12 22 42   12 32 42   22 32 42
 1 2 3 4
* 
,
 12 22 32   12 22 42   12 32 42   22 32 42


exp  12  3  4 ( 1  2 )
2
2
2
 22 42 ( 1  3 ) 2  22 32 ( 1   4 ) 2  12 42 (  2  3 ) 2  12 32 (  2   4 ) 2  12 22 ( 3   4 ) 2
 12 22 32  12 22 42  12 32 42  22 32 42
8π (               )
3
2
1
2
2
2
3
2
1
2
2
2
4
2
1
2
3
2
4
2
2
2
3
(6.1)
(6.2)
.
(6.3)
2
4
whenever the outcome equation factor loadings {αk : k  2,3,4} are non-zero.
(7)
Adaptive Quadrature
3
7/13/2017
Remove dependence on non-zero factor loadings. (Superscript A denotes agent-dep.)
Let kA  ykA  x kA  β k  the data residual for outcomes k = 2,3,4.
*A 
* 
 22 32 42 1   12 32 42 2 (2A  2 )   12 22 42 3 (3A  3 )   12 22 32 4 (4A  4 )
.
 12 22 32 42   12 22 42 32   12 32 42 22   22 32 42
 1 2 3 4
(6.8)
(6.9)
 12 22 32 42   12 22 42 32   12 32 42 22   22 32 42
 1

NA
exp

 2  2 2 2 2   2 2 2 2   2 2 2 2   2 2 2 
A

1 2 3 4
1 2 4 3
1 3 4 2
2 3 4 

A 

3
2
2
2
2
2
2
2
2
2
2
2
2
2
 2 3 4
8π ( 1  2 3  4   1  2 4  3   1  3  4  2   2 32 42 )
(6.10)
where N A   32 42 [ 2 1  (2A   2 )]2   22 42 [ 3 1  (3A  3 )]2   22 32 [ 4 1  (4A   4 )]2
  12 42 [ 3 (2A   2 )   2 (3A  3 )]2   12 32 [ 4 (2A   2 )   2 (4A   4 )]2
  12 22 [ 4 (3A  3 )   3 (4A   4 )]2 .
(6.11)
Notice that *A ,  * ,  A are well-defined for all values of {αk}.
(8)
Then (4.5) is: L(Γ | X A )   A 
1  θ  *A  A
 g ( )d
 
R
 * 
* 

1  θ  *A   x 5A  β 5  α5θ  5 
d
 
 R  
 *   *  
5
A

 
A
A



1  1   θ  *  x 5  β 5  α5θ  5 d

 R  *   *  
5

if exit to W ;
(6.16)
if exit to Z .
(9)
Adaptive Quadrature
4
7/13/2017
Challenge: Calculate I A  
1  θ  *A   x5A  β5  α5θ  5 
d .

 
R
 *  
5
* 

Gauss-Hermite quadrature is a natural choice for integration on R:
Let HN  {(n,Wn) : n = 1,…,N} denote the set of points and weights, resp., for
N-point Gauss-Hermite quadrature. For any function F(y)  C2N (R) that satisfies
2
e  y F ( y) is integrable on R, there exists   R such that

where


N
e  y F ( y )dy   Wn F ( n )   N ( | F )
2
(7.0)
n 1
 N ( | F ) 
N! π
F [ 2 N ] ( ) .
N
2 (2 N )!
Whenever  N ( | F ) is smaller than needed precision, then
Proceedingly naively, let F A ( ) 
2
e
*



N
e  y F ( y )dy   Wn F ( n ) .
2
n 1
 θ     x  β5  α5θ  5 
 .

5
 *  

A
*
 
A
5
2
1  θ  *A 
  e  F A ( ) is integrable on R;
 
*  * 
Then
0  e  F A ( ) 
so
I A   e  F A ( )d   Wn F A ( n ) whenever  N ( | F A ) is small enough.
2
N
2
R
n 1
(10)
Problem: Consider the quadrature approximation for the complete integral of the
normal density with mean zero:
50
1  
en  n  1.00000000000000 if   0.50,
(to 15-digits).

d


W
   


n
      
    0.03767003775858 if   0.05,
n 1
2

where {(Θn,Wn) : n = 1,…,50} are parameters for 50-point Gauss-Hermite quadrature.
See Numerical Recipes method gauher().
For σ = 0.05, the result is worse with 24 points and only slightly better with 100 points.
But also consider (to 15-digits):
1 
a    
b
50
wn  1  b  a
b  a  1.0 if   0.50 and [a, b]  [4.50,4.50],

d


 
n 

 

2  1.0 if   0.05 and [a, b]  [0.45,0.45],

n 1 
  2
where {(λn,wn) : n = 1,…,50} are parameters for 50-point Gauss-Legendre quadrature on
the interval [−9σ,9σ] — quadrature is adapted to the support of the integrand in this case.
See Numerical Recipes method gaulen().
(11)
Adaptive Quadrature
5
7/13/2017
What is an appropriate adaptation for the likelihood integral in (6.16)?
Intuition: Suppose exactly one k, say 2 ≡ τ2/|α2|, is very small relative to the other
three standard deviations. From (6.1–3) it is easy to see that:


0
0
*A 
 2A and  * 
0
2
and
and
2
 1 (   A )2 ( A   A )2 ( A   A )2 
exp   1 2 2  3 2 2  4 2 2 

2 
1
3
4

 2 0 

A
 
3
8π  1 3 4
  A  θ   A 
A
 2 0 
*


max   


  .


2π *
  *   * 
When any one k is very small, the integrand in (6.16) is a spike with significant
support in a small interval containing  kA — a primary insight for identifying an
adaptive strategy. What happens if the integrand support is “spread out” to include
more evaluation points?
(12)
Lemma 2: The Adaptive Rule for Gauss-Hermite quadrature of the normal pdf.
HN  {(n,Wn) : n = 1,…,N} denotes parameters of N-point Gauss-Hermite quadrature.

If F(y) is a polynomial of degree < 2N, then F[2N ](y) = 0 
If F( y)  1,
N
N
 W   W F (
n 1
n
n 1
n
For each N > 0, define: n 

n



N
e  y F ( y )dy   Wn F ( n ) .
2
n 1
)   e  y F ( y )dy   e  y dy  π
Wn
π
2

2
for all N.

for n = 1,…,N. Then
N

n 1
n

1




e  y dy  1 .
2
(7.1)
Pick any constant  and positive constant  and define the adaptation map
AH ( y |  ,  )  2 y  
for all y  R.
1    
Pick any function G( )  C2N(R) satisfying:  
G ( ) is integrable on R.
   
Then, there exists   R such that
(7.2)
(7.3)
N
1    

G
(

)
d


 n ( G  AH )( n |  ,  )   N ( | G,  ,  ) ,

R    
n 1
where
Proof:
(13)
 N ( y | G,  ,  ) 
 2 N N!
(2 N )!
(G [ 2 N ]  AH )( y |  ,  ) 
(7.4)

 2 N N! d 2 N
 2 N G ( )
(2 N )!  d
  A
. (7.5)
H ( y |  ,
)
Apply change of variable  = AH(y | , ) defined in (7.2) to the integral in (7.4).
Adaptive Quadrature
6
7/13/2017
By Lemma 2, the quadrature sum is an approximation of the integral;
that is,
N
N
1    

G
(

)
d



(
G

A
)(

|

,

)

 nG ( 2  n   )


n
H
n
R    
n 1
n 1
(7.7)
whenever the error term  N ( | G,  ,  ) defined by (7.5) is small enough.
 x A  β  α θ  5 

In particular,   *A ,    * as defined at (6.8,9) and G( )   5 5 5
5


A
specify the agent likelihood integrand of (6.16). Further, AH ( y | * ,  * )  2 * y *A ,
so
(G  AH )( y | *A ,  * )  ( 2t5 y  u A )
where
t5 
Define
F ( y) 
 5 *
5
1
π
and u A 
x5A  β5   5 *A  5
5
e y (G  AH )( y | *A ,  * ) 
2
1
π
.
(8.3)
e y ( 2t5 y  u A )  F ( y | u A , t5 ) .
2
(8.4)
Apply change of variable   AH ( y | *A ,  * ) to the integral in (6.16),
y
 e
1  θ  *A 
A


R    G( )d   π (G  AH )( y | * , * )dy
*
*
2
N

  F ( y | u A , t5 )dy   n ( 2t5 n  u A ) .

(8.2)
n 1
In calculating the integral of (6.16) by approximation formula (8.2), we see that
 All agent dependence in the calculation is in the agent-specific constants *A ,  * .
Naïve Gauss-Hermite points and weights HN, a set of universal constants for
each N, suffice for approximating the integral for all agents.

Evaluation of the integral does not require calculating the integrand density at each
quadrature point n. Given *A ,  * , only the function ( 2t5 n u A ) requires
calculation at each n.
Given integer N, a computer algorithm can calculate Gauss-Hermite points and
weights HN once at the start of program execution. Then, to evaluate agent
likelihood in (6.16), the algorithm calculates agent-specific *A ,  * ,  A by (6.8–11)
and then approximates the likelihood using (8.2).
(14)
Adaptive Quadrature
7
7/13/2017
But what is the error of approximation (8.2)? When is  N ( | G, *A ,  * ) small enough?
Some mathematical results eliminate special cases:
Lemma 3: Approximation (8.2) is exact when  5  0 :



F ( y | u A , t5 )dy  (u A ) .
Lemma 4: In (8.3), if uA = 0, then, for all t5  R\{0},

(a)
R
F ( y)dy  12 and (b) approximation (8.2) is exact for all N > 0.
So, in general consideration of approximation error, we may assume  5  0  u A .
Importantly, this assumption implies t5  0. In fact, we may also assume t5 > 0:
Remark 8.1.3.

R
F ( y | u A ,t5 )dy   F ( y | u A , t5 )dy .
R
By (8.3), the sign of t5 depends only on 5, the factor loading in the cost equation
for exit to state W; so t5 can be positive or negative. By (8.4),
F ( y | u A ,t5 ) 
1
π
e y ( 2t5 y  u A )  F ( y | u A , t5 ) .
2
Geometrically, a plot of F( y | uA, t5) is the mirror image of the plot for F( y | uA, t5).
Furthermore, with s  y,

R






F ( y | u A ,t5 )dy   F ( y | u A , t5 )dy   F (s | u A , t5 )ds   F (s | u A , t5 )ds .
The likelihood integral is an even function of t5. Given (uA, t5), approximation (8.2) is
identical for either sign of t5; thus, it is sufficient to characterize approximation (8.2)
for t5 > 0.
In contrast, it is not sufficient to consider only one sign for uA, as discussed below.
From this point, general consideration can proceed in two ways —
1. Investigate bounds on the error term, which, by (7.5), (7.2), and (8.3), is:
 N ( y | G,  ,  * ) 
A
*
 *2 N N!
(2 N )!
(G
[2 N ]
2N

t5 N !  d 2 N
 AH )( y |  ,  * ) 
 2 N ( )
(2 N )!  d
  A
A
*
. (9.0)
A
H ( y | * , * )
2. Proceed empirically by calculating the likelihood for a representative range of values
(uA, t5) in an effort to discover regularities of accuracy and inaccuracy.
Investigation of the error term is a complicated analysis detailed elsewhere. For now,
let’s proceed empirically.
(15)
Adaptive Quadrature
8
7/13/2017
In empirical investigation of an integral, it is very helpful to develop intuition about the
geometry of the integrand. Figure 8.0 presents the central case, t5 = 1 and uA = 0:
1.0
F ( y) 
 
1
π
 
0.9
e y  2 y
2
0.8
 2y
1
π
e y
0.7
0.6
2
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.0:
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of integrand F ( y) and its components in (8.4) with parameters t5 = 1, uA = 0.
Note: RF( y) = 0.5 for all t5 when uA = 0. See Lemma 4.
(16)
The multiplied components of integrand F( y) are:
1
π
e y 
2
1
2π
1
2
e
1 y 
  1 

2
 2
2

 2 ( 2 y ) = pdf of the N 0,
1
2
 distribution,

 uA 


y   

 uA

2
t
t
1
5 


and  ( 2t5 y  u A )   5
=
cdf
of
N
,


1
2t5 2 t5
 t5



2 t5



(8.5)

 oriented by sgn(t5).


Since t5 is assumed positive, we consider only the usual orientation of the cdf component.
Characteristics of F( y) components are primary features of adaptation to changing model
parameters and data varying by agent. The original integrand density for N(*A ,  * ) in
(6.16) is transformed in (8.5) to a fixed density centered at the origin and independent of
all parameter values and agent data. To compensate, the original cdf in (6.16) is
translated and rescaled by parameters and data. By (8.3), a small standard deviation *,
which causes inaccuracy with naïve quadrature, may imply a small t5 which implies the
cdf component of F( y) has large standard deviation. Untransformed density with narrow
peak is replaced by a pdf of fixed peak-width > 2 2 ; quadrature points thereby populate
the expanded support of the transformed integrand.
(17)
Adaptive Quadrature
9
7/13/2017
Figures 8.1a–f below illustrate the integrand F( y) in (8.4) for selected values of t5 and uA.
When uA and t5 are positive, the cdf component mean is left of the origin by (8.5):
F ( y) 


1
π
2
2
e
1
π
e
 y2



2
2
y

1
2
y  12 , for N 
1
2

2
2
,
1.0
0.9

0.8
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
-2.0
-1.5
F ( y) 
1
π

e
 y2
0.0
0.5
1.0
1.5
2.0
2.5
3.0


1.0
 2y 1


 2 y  1 , for N 
e
-0.5
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 0.5 = uA.
Note: The integral of F( y)  0.672639576990712 with N = 15. See Table 8.1a below.
FIGURE 8.1a:
1
π
-1.0
1
2
,
1
2
0.9

0.8
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.1b:
F ( y) 

-2.0
-1.5
-1.0

e
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 1.0 = uA.
Note: The integral of F( y)  0.760249938906524 with N = 28. See Table 8.1b below.
1
π
e
 y2


 2 2y  2

 2 2 y  2 , for N 
1
-0.5
1
2

, 2 12
1.0
0.9

0.8
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.1c:
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 2.0 = uA.
Note: The integral of F( y)  0.814453315238651 with N = 78. See Table 8.1c below.
Adaptive Quadrature
10
7/13/2017
When uA < 0 and t5 > 0, the cdf component mean is right of the origin by (8.5):

2
2
y  12 , for N

F ( y) 


1
π
2
2
e
1
π
e
 y2


y
1
2


1
2
2
2
,
1.0
0.9
0.8
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.1d:
F ( y) 

-2.0
-1.5
e
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 0.5 = uA.
Note: The integral of F( y)  0.327360423009288 with N = 15. See Table 8.1d below.
1
π



1.0
e y  2 y 1
2

 2 y  1 , for N
1
π
-1.0

1
2
1
2
,
0.9
0.8
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.1e:
F ( y) 

-2.0
-1.5
e
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 1.0 = uA.
Note: The integral of F( y)  0.239750061093477 with N = 29. See Table 8.1e below.
1
π
e
 y2


 2 2y  2
 2 2 y  2 , for N
1
π
-1.0

1
2


1.0
0.9
0.8
, 2 12
0.7
 y2
0.6
0.5
0.4
0.3
0.2
0.1
y
0.0
-3.0
-2.5
FIGURE 8.1f:
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Plot of transformed likelihood integrand components in (8.4) with parameters t5 = 2.0 = uA.
Note: The integral of F( y)  0.185546684761349 with N = 81. See Table 8.1f below.
Adaptive Quadrature
11
7/13/2017
For each selected pair of parameters (uA,t5) in Figures 8.1a–f, Tables 8.1a–f below describe
accuracy of the quadrature sum in (8.2) as a function of the number of quadrature points N.
Calculations for each table below were used to position the vertical lines under the curve of
F( y) in each figure above. For the N of the last row in each table, quadrature points in HN
that fall in the interval [3,+3] are at the position of those vertical lines and small y-axisticks in the figure corresponding to the table.
In the tables, the quadrature sum in (8.2) that approximates F( y)dy is denoted:
N
H N ( | u A , t5 )   n ( 2t5 n  u A ) .
(8.13)
n 1
The error of approximation (8.2) is simply the difference
EN (u A , t5 )   F ( y)dy  H N ( | u A , t5 )
(8.14)
R

1
R π
N
e  y ( 2t5 y  u A )dy   n ( 2t5 n  u A ) .
2
n 1
For each table row, F( y)dy was calculated by Romberg integration on the interval [9,+9].
Romberg calculation was conducted in 19-digit hardware precision. Iterative refinement of
the interval partition terminated with the relative error of polynomial interpolation < 1016.
Since approximation error EN (uA,t5) does not directly reveal the number of significant
digits in the approximation, define the relative error at N points of approximation (8.2):
RE N (u A , t5 ) 

1
R π
N
e  y  ( 2t5 y  u A )dy   n  ( 2t5 n  u A )
2
n 1

1
R π
e
 y2
.
 ( 2t5 y  u A )dy
(8.15)
Then the decimal significance at N points of approximation (8.2) is:
S N (u , t5 )   log 10 RE N (u , t5 )  log 10
A

A

1
R π
1
R π
e  y  ( 2t5 y  u A )dy
2
N
e  y  ( 2t5 y  u A )dy   n  ( 2t5 n  u A )
2
n 1
In the tables, SN (uA,t5) rounded down to the nearest integer is the number of significant
decimal digits in HN ( | uA,t5) relative to the Romberg value for F( y)dy.
For given (uA,t5), each table lists values of HN ( | uA,t5) and the signed error (8.14) for
selected increasing N until REN (uA,t5) < 1015. Thus, table lengths vary with the efficiency
of Gauss-Hermite quadrature at each (uA,t5). The last two columns show REN (uA,t5) with
SN (uA,t5) rounded down to the nearest tenth. When HN ( | uA,t5) equals the Romberg
value, both rounded to 15 digits, the value of the quadrature sum is shown in bold-face.
(18)
Adaptive Quadrature
12
7/13/2017
When uA and t5 are positive:
N
4
8
12
15
HN ( | 0.5,0.5)
0.672618686869557
0.672639574765081
0.672639576990494
0.672639576990712
TABLE 8.1a:
EN (0.5,0.5)
REN (0.5,0.5)
2.08901211543  10
2.2256304  109
2.172  1013
2  1016
5
SN (0.5,0.5)
3.10569313327  10
3.3088008  109
3.229  1013
3  1016
5
4.5
8.4
12.4
15.5
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 0.5 = uA.
In H15, the extreme points are 4.49999070730939 with weight 8.58964989963327  1010.
N
HN ( | 1.0,1.0)
EN (1.0,1.0)
REN (1.0,1.0)
SN (1.0,1.0)
4
8
12
16
20
24
28
0.758944432021307
0.760251305281224
0.760250049464872
0.760249940494693
0.760249938922564
0.760249938906643
0.760249938906524
1.3055068852163  103
1.3663747004  106
1.105583487  107
1.5881696  109
1.60404  1011
1.196  1013
4  1016
1.7172074845465  103
1.7972703849  106
1.454236865  107
2.0890098  109
2.10989  1011
1.573  1013
6  1016
2.7
5.7
6.8
8.6
10.6
12.8
15.2
TABLE 8.1b:
N
4
8
12
16
20
24
28
32
36
40
44
48
52
56
60
64
68
72
76
78
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 1.0 = uA.
In H28, the extreme points are 6.59160544236774 with weight 6.43254743880186  1020.
Note: Romberg value = 0.760249938906523 at relative tolerance 1016 rounded to 15 digits.
HN ( | 2.0,2.0)
EN (2.0,2.0)
REN (2.0,2.0)
0.816631474537427 2.1781592987761  10
0.818015114411898 3.5617991732470  103
0.814804231765483 3.509165268317  104
0.814430926294541
2.23889441103  105
0.814438558202192
1.47570364595  105
0.814450180176825
3.1350618260  106
0.814452900521139
4.147175118  107
0.814453293777743
2.14609082  108
0.814453322229591
6.9909400  109
0.814453318135879
2.8972278  109
0.814453315930169
6.915176  1010
0.814453315365336
1.266851  1010
0.814453315256783
1.81315  1011
0.814453315240331
1.6799  1012
0.814453315238586
6.56  1014
0.814453315238569
8.20  1014
0.814453315238625
2.61  1014
0.814453315238645
6.2  1015
0.814453315238650
1.2  1015
5  1016
0.814453315238651
TABLE 8.1c:
3
SN (2.0,2.0)
2.6743820155460  10
4.3732392104063  103
4.308614382998  104
2.74895364675  105
1.81189470083  105
3.8492836451  106
5.091974015  107
2.63500778  108
8.5835981  109
3.5572668  109
8.490576  1010
1.555462  1010
2.22622  1011
2.0627  1012
8.05  1014
1.006  1013
3.20  1014
7.6  1015
1.5  1015
7  1016
3
2.5
2.3
3.3
4.5
4.7
5.4
6.2
7.5
8.0
8.4
9.0
9.8
10.6
11.6
13.0
12.9
13.4
14.1
14.8
15.1
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 2.0 = uA.
In H78, the extreme points are 11.7257979195159 with weight 7.6920667316977  1061.
Adaptive Quadrature
13
7/13/2017
When uA is negative and t5 is positive:
N
HN ( | 0.5,0.5)
EN (0.5,0.5)
4
8
12
15
0.327381313130443
0.327360425234919
0.327360423009506
0.327360423009288
2.08901211543  105
2.2256304  109
2.172  1013
2  1016
TABLE 8.1d:
N
4
8
12
16
20
24
28
29
4
8
12
16
20
24
28
32
36
40
44
48
52
56
60
64
68
72
76
80
81
EN (1.0,1.0)
REN (1.0,1.0)
0.241055567978693 1.3055068852163  10
0.239748694718776
1.3663747004  106
0.239749950535128
1.105583487  107
0.239750059505307
1.5881696  109
0.239750061077436
1.60404  1011
0.239750061093357
1.196  1013
0.239750061093476
4  1016
0
0.239750061093477
3
4.1
8.1
12.1
15.2
SN (1.0,1.0)
5.4452828051931  10
5.6991630956  106
4.611400232  107
6.6242717  109
6.69048  1011
4.988  1013
1.8  1015
2  1016
3
2.2
5.2
6.3
8.1
10.1
12.3
14.7
15.6
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 1.0 = uA.
In H29, the extreme points are 6.72869519860885 with weight 1.02934180872194  1020.
HN ( | 2.0,2.0)
EN (2.0,2.0)
0.183368525462573
0.181984885588102
0.185195768234517
0.185569073705459
0.185561441797808
0.185549819823175
0.185547099478861
0.185546706222257
0.185546677770409
0.185546681864121
0.185546684069831
0.185546684634664
0.185546684743217
0.185546684759669
0.185546684761414
0.185546684761431
0.185546684761375
0.185546684761355
0.185546684761350
0.185546684761349
0.185546684761349
2.1781592987761  10
3.5617991732470  103
3.509165268317  104
2.23889441103  105
1.47570364595  105
3.1350618260  106
-4.147175118  107
2.14609082  108
6.9909400  109
2.8972278  109
6.915176  1010
1.266851  1010
1.81315  1011
1.6799  1012
6.56  1014
8.20  1014
2.61  1014
6.2  1015
1.2  1015
2  1016
1  1016
TABLE 8.1f:
SN (0.5,0.5)
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 0.5 = uA.
In H15, the extreme points are 4.49999070730939 with weight 8.58964989963327  1010.
Note: Romberg value = 0.327360423009289 at relative tolerance 1016 rounded to 15 digits.
HN ( | 1.0,1.0)
TABLE 8.1e:
N
REN (0.5,0.5)
6.38138262476  105
6.7987154  109
6.635  1013
6  1016
REN (2.0,2.0)
3
SN (2.0,2.0)
1.17391442567546  10
1.91962425943005  102
1.8912573257942  103
1.206647488156  104
7.95327411995  105
1.68963505334  105
2.2351114078  106
1.156631186  107
3.76775258  108
1.56145489  108
3.726921  1010
6.827669  1010
9.77195  1011
9.0540  1012
3.536  1013
4.417  1013
1.405  1013
3.33  1014
6.6  1015
1.1  1015
7  1016
2
1.9
1.7
2.7
3.9
4.0
4.7
5.6
6.9
7.4
7.8
8.4
9.1
10.0
11.0
12.4
12.3
12.8
13.4
14.1
14.9
15.1
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5 = 2.0 = uA.
In H81, the extreme points are 11.9681194448687 with weight 2.45280551389805  1063.
Adaptive Quadrature
14
7/13/2017
For each (uA,t5), define quadrature inefficiency ineff(uA,t5)  min{N : REN (uA,t5) < 1015}.
Define (uA,t5) as a problem point if ineff((uA,t5) > 40. There are two problem point cases.
Problem 1: uA < 10. (The cdf component mean is positive.)
uA
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
HN ( | uA,1.0)
N
29
30
30
30
33
34
35
35
38
39
41
42
44
47
49
51
54
56
59
60
TABLE 8.2c:
EN (uA,1.0) REN (uA,1.0) SN (uA,1.0)
0.239750061093477
0.786496035251426  101
0.169474267623446  101
0.233886749052363  102
0.203476008722479  103
0.110452484992927  104
0.371549186170706  106
0.770862895014000  108
0.983080220771443  1010
0.768729897214017  1012
0.367892395898720  1014
0.107598683562495  1016
0.192107416356032  1019
0.209191280388971  1022
0.138832469301529  1025
0.561214858649146  1029
0.138116203566689  1032
0.206851587325691  1036
0.188460724282744  1040
0.104424379188127  1044
6  10
4  1018
4  1018
1  1018
1  1019
7  1021
9  1023
7  1024
3  1026
0
1  1030
4  1033
1  1035
7  1039
6  1042
2  1045
1  1048
6  1054
1  1056
2  1061
17
2  10
6  1017
2  1016
6  1016
7  1016
7  1016
2  1016
9  1016
3  1016
0
4  1016
4  1016
5  1016
3  1016
4  1016
4  1016
8  1016
3  1017
6  1016
2  1016
16
15.6
16.2
15.6
15.1
15.1
15.1
15.6
15.0
15.4
 16.0
15.4
15.3
15.2
15.4
15.3
15.4
15.0
16.5
15.2
15.6
Romberg Value
0.239750061093477
0.786496035251426  101
0.169474267623446  101
0.233886749052363  102
0.203476008722479  103
0.110452484992927  104
0.371549186170706  106
0.770862895014001  108
0.983080220771444  1010
0.768729897214017  1012
0.367892395898720  1014
0.107598683562495  1016
0.192107416356032  1019
0.209191280388971  1022
0.138832469301528  1025
0.561214858649146  1029
0.138116203566689  1032
0.206851587325691  1036
0.188460724282744  1040
0.104424379188127  1044
Efficiency of N-point adaptive Gauss-Hermite quadrature with t5  1.0 and uA increasingly negative.
In H60, the extreme points are 10.1591092461801 with weight 6.2601756734114  1046.
Note: Romberg values were calculated on interval [16,+16] with tolerance < 1016.
A quadrature sum equal to the Romberg value rounded to 15 digits is in bold-face.
Compare Table 8.2c with Table 8.2a for uA increasingly positive (negative cdf means).
uA
1
2
3
4
5
6
7
8
9
10
11
12
N
28
28
28
27
25
24
22
19
17
14
12
1
TABLE 8.2a:
HN ( | uA,1.0)
EN (uA,1.0)
REN (uA,1.0)
0.760249938906524
0.921350396474858
0.983052573237656
0.997661132509476
0.999796523991277
0.999988954751500
0.999999628450814
0.999999992291372
0.999999999901693
0.999999999999231
0.999999999999997
1.000000000000000
4  10
5  1016
4  1016
6  1016
1  1016
6  1016
6  1016
7  1016
8  1016
5  1016
4  1016
0
6  10
6  1016
4  1016
6  1016
1  1016
6  1016
6  1016
7  1016
8  1016
5  1016
4  1016
0
16
16
SN (uA,1.0)
15.2
15.2
15.4
15.2
15.8
15.2
15.2
15.1
15.0
15.3
15.4
 16.0
Romberg Value
0.760249938906523
0.921350396474857
0.983052573237655
0.997661132509476
0.999796523991277
0.999988954751501
0.999999628450814
0.999999992291371
0.999999999901692
0.999999999999231
0.999999999999996
1.000000000000000
Efficiency of N-point adaptive Gauss-Hermite quadrature with t5  1.0 and uA increasing.
Note: A quadrature sum equal to the Romberg value rounded to 15 digits is in bold-face.
Adaptive Quadrature
15
7/13/2017
In both Tables 8.2a,c above, the table entry for each uA is for N  ineff(uA,1). Table 8.2c
shows the effect on quadrature efficiency as the magnitude of uA < 0 increases with t5
fixed at 1. As a positive cdf mean increases with constant standard deviation, the cdf
component converges to the constant function 0 on significant support of the integrand
F( y), so F( y) converges to zero. In contrast, as a negative cdf mean decreases with
constant standard deviation, the cdf component converges to the constant function 1 on
significant support of the integrand F( y), so F( y) converges to the pdf component.
It is clear that adaptive quadrature maintains 15-digit accuracy across 46 orders of
magnitude for the likelihood integral in the two tables. It is also clear that quadrature
efficiency improves with increasingly positive uA in 8.2a and worsens with increasingly
negative uA in 8.2c.
One might expect that efficiency would improve in both cases because F( y) converges to
a function for which quadrature is exact in both cases. Intuitively, inefficiency increases,
in 8.2c because, with increasingly negative uA, the interval of significant support widens
as the integral value decreases. In Table 8.2a, quadrature evaluation points n with
decreasing N are located in narrower intervals centered at the origin with larger weights
Wn. In 8.2c, larger N locates quadrature evaluation points farther from the origin in the
wider interval of significant support where, for each n, the contribution nF(n | uA, t5) to
the quadrature sum, though small because n and F(n | uA, t5) are both small, is
nonetheless significant because the sum is small. This intuition is one way of interpreting
the character of polynomial approximation for F( y) at each degree N when uA is negative.
Problem 2: t5 > 1. (The cdf component standard deviation is small.)
This problem is evident in both Tables 8.1c and 8.1f. As t5 increases beyond 2, efficiency
worsens rapidly. Indeed, ineff(4,4) > 180. This problem with small cdf component
standard deviations is the analogue of the problem with small pdf component standard
deviations. In the simple model, there is a work-around for this problem:
Lemma 7: For all (uA, t5)  R  R\{0}, define u A* 
N




uA
1
and t5* 
 0 . Then
t5
t5
N
n( 2t5n  u A )   F ( y | u A , t5 )dy   F ( y | u A* , t5* )dy  n( 2t5*n  u A* ) .
n 1
n 1
Lemma 7 establishes (uA, t5) and (uA*, t5*) as calculation duals. F( y | uA, t5) is said to be
dual to F*( y | uA, t5)  F( y | uA*, t5*). This dualism is very affective. For example, as
remarked above, ineff(4,4) > 180, but (1,0.25) is dual to (4,4) and ineff(1,0.25) = 10:
N
4
8
10
HN ( | 1.0,0.25)
0.834012223599186
0.834012266458670
0.834012266458632
TABLE 8.4b:
EN (1.0,0.25)
REN (1.0,0.25)
4.28594457  10
3.82  1014
0
8
SN (1.0,0.25)
5.13894668  10
4.58  1014
0
8
7.2
13.3
 16.0
Accuracy of N-point adaptive Gauss-Hermite quadrature with parameters t5* = 0.25 and uA* = 1.0.
(19)
Adaptive Quadrature
16
7/13/2017
To describe the region of adaptive Gauss-Hermite quadrature inaccuracy more fully,
relative approximation errors at N = 40 were determined on a 400  400 grid of 160,000
points {(uiA , t5 j ) : i  1,...,400 and j  1,...,400} in a subset of the (uA, t5)-plane. The 400
uA-coordinate values were regularly spaced in the interval [20.0,+20.0] separated by 0.1,
excluding zero because quadrature is exact when uA = 0. The 400 t5-coordinate values
were regularly spaced in the interval [0.01,4.00] separated by 0.01. No grid point had a
negative t5-coordinate because the likelihood function and the approximation formulae
are even functions of t5. Calculations on the grid for t5j  1 used approximation formula
(8.2), while calculations for t5j > 1 used the dual function F*( y | uA, t5) of Lemma 7.
With this specification, calculation on the grid was conducted to locate the subset region
where relative error of adaptive quadrature approximation with N = 40 is greater than or
equal to   1015. Importantly, all 80,000 relative errors calculated with uiA  0 were less
than 1015 at N = 40. Results for the remaining 80,000 grid points with coordinate uiA  0
are plotted in Figure 8.6a below.
Three characteristics of Figure 8.6a are noteworthy:

To characterize approximation error at a fixed number M of quadrature points (uA, t5),
e.g. at M = 40, we may define two classes of quadrature inaccuracy.
Inaccuracy at (uA, t5) is in the first class if 1  N  M  REN (uA, t5)  accuracy . This
class of inaccuracy is labeled Underdone in Figure 8.6a.
Inaccuracy at (uA, t5) is of second class if there exists m < M such that REm(uA, t5) < ;
but REM (uA, t5)  . This class of inaccuracy is labeled Overdone in Figure 8.6a.
To classify the grid points, the value of the quadrature sum for all N = 1,2,…,40 was
calculated for all 160,000. If the result is accurate at all m  N  40 points, then white
space is plotted in Figure 8.6a. If the result is inaccurate at all N  40 points, then
Underdone inaccuracy is plotted. Otherwise Overdone inaccuracy is plotted.

The closed trapezoidal region T in the lower quarter of the figure highlights the use of
Lemma 7 for calculations of quadrature sums at grid points in rectangle A  {(uA, t5) :
20  uA < 0 and 1 < t5  4}. Every point in T is (uA*, t5*), dual to a unique (uA, t5)  A
under the dual map of Lemma 7. For example, the left boundary of A is a segment of
the line uA = 20 which maps to the diagonal boundary of T.

Figure 8.6a presents three fairly distinct regions containing points of quadrature
inaccuracy — Analytic, Unstable, and Indeterminate. The Analytic region is the
unbounded steadily-darkening roughly-triangular subset at lower left, distinct from
the Unstable region immediately below where points of Overdone inaccuracy are
more abundant. The Indeterminate region lies mostly above t5 = 1 dotted with isolated
points where 1014  RE40 (uiA , t5j) < 1013. Inaccuracy in each region is of interest.
The Analytic region is the site of points with Problem 1.
At points of inaccuracy in the Unstable Region, REN (uiA , t5j) oscillates about ε.
Points of inaccuracy in the Indeterminate Region are sites of Romberg inaccuracy.
Adaptive Quadrature
17
7/13/2017
t5
4.0
3.9
3.8
3.7
3.6
3.5
3.4
3.3
3.2
3.1
3.0
2.9
2.8
2.7
2.6
Underdone
2.5
White Space: RE40(uA, t5) < 1015
2.4
1015  RE40(uA, t5) < 1014
 RE40(u , t5) < 10
2.3
13
2.2
1013  RE40(uA, t5) < 1012
2.1
1012  RE40(uA, t5) < 1011
2.0
10
14
A
1.9
1011  RE40(uA, t5)
1.8
1.7
1.6
Overdone
+

1.5
1015  RE40(uA, t5) < 1014
10
14
 RE40(u , t5) < 10
A
1.4
13
1.3
1.2
1.1
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
uA
0.0
-20
-19
-18
-17
-16
-15
-14
-13
-12
-11
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
1
FIGURE 8.6a:
Inaccuracy regions for adaptive 40-point Gauss-Hermite quadrature.
At each grid point, the Romberg value was calculated on the interval [16,+16].
Note 1: RE40(uA, t5) is plotted whenever t5  1; RE40(uA*, t5*) is plotted whenever t5 > 1.
The trapezoidal region is the image of {t5  1} under the dual map (uA, t5)  (uA*, t5*).
Note 2: “Underdone” means REN(uA, t5) < 1015 is not achieved with N  40.
“Overdone” means REN(uA, t5) < 1015 is achieved at some N < 40 but not at N = 40.
Note 3: In H40, extreme points are 8.09876113925085 with weight 1.46183987386942  1029.
Adaptive Quadrature
18
7/13/2017
Oscillation of the quadrature sum in the Unstable Region is exemplified by Figures 8.7a,b.
EN (17.4,0.40)  1073 (Romberg value = 0.519411553148810  1058 )
7
15
 REN (17.4,0.40)  10
6
 REN (17.4,0.40) < 1015
5
4
3
2
1
N
0
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85
-1
-2
-3
FIGURE 8.7a:
Approximation error oscillation yielding Underdone inaccuracy at N = 40 in Table 8.7a.
EN (17.9,0.24)  1082 (Romberg value = 0.373224040928404  1067 )
15
 REN (17.9,0.24)  10
7
 REN (17.9,0.24) < 1015
6
5
4
3
2
1
N
0
-1 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
-2
-3
-4
-5
-6
-7
-8
FIGURE 8.7b:
Approximation error oscillation yielding Overdone inaccuracy at N = 40 in Table 8.7b.
Compare oscillatory inaccuracy with Underdone inaccuracy in the Analytic Region.
EN (17.5,1.00)  1049 (Romberg value = 0.179855786432354  1034 )
2
1
N
0
-1 51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
-2
-3
-4
-5
-6
15
 REN (17.5,1.0)  10
-7
 REN (17.5,1.0) < 1015
-8
-9
-10
-11
-12
-13
-14
FIGURE 8.7c:
Approximation error sequence at t5 = 1.0 and uA = 17.5. At N = 40, inaccuracy is Underdone.
In H54, the extreme points are 11.4786603187226 with weight 2.39641805232478  1058.
Adaptive Quadrature
19
7/13/2017
Conclusion: Figure 8.6a answers the question “When is  N ( | G, *A ,  * ) small enough?”
How do we use the information in Figure 8.6a?
We should identify the region S of parameter space where the strategy is suitable and the
complementary region U where accuracy and/or efficiency are unsuitable. Calculations
for construction of Figures 8.6a and 8.6b provide a foundation for identifying S and U. A
specific algorithm is presented later after discussion of the approximation error function
N ( y | G,, ) defined by (7.5). Here we assert two general conclusions to set the
direction of further developments.
First, the Gauss-Hermite strategy is suitable for accurate approximation at most points of
the parameter space R  R\{0}. Specifically, 40-point quadrature was demonstrated to be
competent for 15-digit accuracy at (uA, t5) at more than 95% (by Euclidean measure) of
the disconnected set G  [20,+20]  [4,+4]  R  R\{0}. Points of accuracy in G are
parameters for values of the likelihood integral spanning 89 orders of magnitude. It was
also demonstrated that the minimum integer m satisfying REm(uA, t5) < 1015 often also
satisfies m < 40. Because evaluation of likelihood integrand F(n | uA, t5) is expensive, for
the sake of algorithm efficiency we must identify S and classify its points by m.
Second, the Gauss-Hermite strategy is inefficient for accurate calculation at many points
in the Analytic region. Many points of inaccuracy in the Unstable region might be less
worrisome because approximation error in the Unstable region of G is observed beyond
the 12th digit. However, it is not clear that Unstable region inaccuracy is bounded with
increasingly negative uA. For an accurate approximation at all points of the parameter
space, U must be identified (say, as the complement of G), perhaps with some flexibility
depending on user specification of tolerance or even automated by tracking the
magnitude of function value change across minimizer steps. Naturally, however U is
identified, an alternative strategy for approximating the likelihood integral at points in U
should be developed.
One might contemplate avoiding the challenge by considering two effects. First, at points
in the Analytic and Unstable regions, the likelihood is very small. Thus, since the
minimizer is maximizing the likelihood (by minimizing the negative log-likelihood),
minimizer trial values for the parameters are biased away from points in U. Second, one
might also consider scaling agent data so that minimizer trial values for the parameters —
observed and unobserved predictor loadings and distribution parameters — are small
enough to ensure that uA is never sufficiently negative for the minimizer path to traverse U.
In this point of view, the challenge for accuracy is simply a data-scaling problem.
It is easy to see that scaling the data cannot ensure an empty intersection between U
and the minimizer path. Recall the definitions of uA and t5 in (8.3):
uA 
where
*A 
x5A  β5   5 *A  5
5
and t5 
 5 *
,
5
 22 32 42 1   12 32 42 2 (2A  2 )   12 22 42 3 (3A  3 )   12 22 32 4 (4A  4 )
 12 22 32 42   12 22 42 32   12 32 42 22   22 32 42
Adaptive Quadrature
20
7/13/2017
and
* 
 1 2 3 4
 12 22 32 42   12 22 42 32   12 32 42 22   22 32 42
as defined at (6.8,9). Scaling the data and astutely choosing estimation starting
values can ensure that the numerators and denominators of the above expressions
are small along the minimizer path. However, scaling cannot ensure that the ratio
defining uA is bounded away from points in U.
Nonetheless, the first sensibility is correct — the minimizer is biased for trial
parameter values such that path points (uA, t5) are outside U. An alternative strategy
for calculating the likelihood integral can rely on this bias by relaxing concern for
efficiency of the calculation at points in U. Because accuracy is harder to achieve
on U, more work is required and lower efficiency is achieved. However, because
the measure of U is a small fraction of the measure of S, and because the minimizer
is biased away from U, it is reasonable to expect that the fraction of calculation on
U is small and decreasing as an estimation sequence proceeds to the likelihood
maximum. Thus, lower efficiency is experienced on a small and decreasing fraction
of the total calculation. We shall quantify approximation efficiency and accuracy on
U in discussion of the alternative strategy for approximating the likelihood integral
with parameters (uA, t5) in U.
So the direction of next developments is two-fold. First, we will investigate the
approximation error function to discover mathematical characterization of GaussHermite quadrature accuracy. Then we will present an algorithm for specifying S
and classifying its point by m. This is the most crucial need because, as explained
above, we expect most of the calculation to occur at points in S where efficiency is
highest.
Then, having specified S and calculation on S, we will define U as the complement
of S and specify a less efficient but sufficiently accurate strategy for approximating
the likelihood integral at points in U.
From there we will broaden discussion to models of greater complexity — multiple
factors and multiple agent decisions. We shall see that more complex non-linearity
in the likelihood integrand for multiple decisions requires more complex analysis
and development. However, inituition developed by investigation of the simple
model will inform our approach to calculation for more complex models.
Adaptive Quadrature
21
7/13/2017
Lagniappe: The following is useful for analysis of likelihoods with normal probability.
Let g ( z ) 
Lemma:
z 
and h( z )  z( z )   ( z ) and k ( z )  12 [( z 2  1)( z )  z ( z )] .
 z 


Then (a) g ( z ) z
  , g ( z ) z
 0 .
 
 0 for all non-negative integers n.
(b) z n  ( z ) z
z
z


(c) h( z)   ( y)dy and k ( z)   h( y)dy
(d) g (z ) is a positive, increasing, convex function.
 ( z)
 ( z)
( z )
(e) z  0  0 
 ( z ) 
 lim
 1.
z


1
 ( z) 

z

z


z
 z 
 ( z)
( f ) z 0
approximates (z) with decimal significance > 2 log 10 z  .
z
Proof:
We argue that (a)  (b)  (c)  (d)  (e)  ( f ).
(a) The first limit in (a) is obvious. As z   , the limits of numerator and denominator
in g(z) are both zero, so the second limit in (a) is easily proved by l’Hopital’s Rule:
 z 
 z 
1
 lim
  lim  0 .
z    z 
z   z  z 
z  z
lim
(b) Pick any  > 0. From l’Hopital’s Rule for n = 1 and then induction, it is well-known
 
 0 for all non-negative integers n.
that z n  z  z
Thus, there exists M1 such that z  M 1  z n  z    .
From the second limit in (a), there exists M2  0 such that z  M 2  z n  ( z )  z n  z   .
Let M ≡ min{M1,M2}. Then z  M  z n ( z )  z n  z     .
(c) For the first equality in (c), note that h( z )  ( z )  z ( z )  z ( z )  ( z ) .
So h(z) is the anti-derivative of (z), whence
z
 ( y)dy  z( z)   ( z)  a(a)   (a)
a
for all a < z.
Now take the limit as a   on both sides of the above equation and apply (b):

z

( y)dy  lim
a 
z
 ( y)dy  z( z)   ( z)  lim [a(a)   (a)]  h( z) .
a 
a
The argument for the second equality in (c) is analogous. Note that
k ( z )  12 [2 z( z )  ( z 2  1) ( z )   ( z )  z 2 ( z )]  12 [2 z( z )  2 ( z )]  h( z ) .
z
So k(z) is the anti-derivative of h(z). By the argument for h(z), k ( z)   h( y)dy .

Adaptive Quadrature
22
7/13/2017
(d) That g(z) is everywhere positive is obvious. That g(z) is everywhere increasing is a
consequence of (c). Consider:
g ( z ) 
 ( z ) ( z )
( z )

[ z ( z )]  1  z
 1  zg ( z ) .
2
 ( z)  ( z)
 ( z)
Thus, h( z )   ( z ) g ( z ) .
z
Then ( z)  0   ( y)dy  0   ( z) g ( z)  0  g ( z)  0 .

So g(z) is everywhere increasing. That g(z) is everywhere convex is analogous.
g ( z )  g ( z )  zg ( z ) 
( z ) 
( z ) 
( z )
 z 1  z
 ( z 2  1)
z.

 ( z)
 ( z) 
 ( z)

Thus, k ( z )  12  ( z ) g ( z ) . Then h( z )  0  k ( z )  0  12  ( z ) g ( z )  0  g ( z )  0 .
(e) The necessary inequalities have already been proved in (d). Specifically,
h( z )  0  z( z )   ( z )   z( z )   ( z ) so z  0   ( z ) 
 ( z)
z
;
z ( z )
 ( z)

(for all z  R).
2
z 1  z  1
z
z
z
z

 ( z )   zg ( z )  1 .
For the limit, multiply the inequality by
0:
1  z 
 z 
z
z
z
 lim [ zg ( z )]  1 .
Take the limit on all sides of this inequality: 1  lim
z  
1 z 
z
z
 ( z)  ( z)

 z 2 1  z 2
 ( z)  ( z)
1
1
z

z
( z ) 
 ( z )
z
 
2
z  z
z  z z 2  1  z ( z  1)  1 .
( f ) (e) 

 ( z)
z
z2
( z )
( z )
z2
 2

1
z 1
z ( z 2  1)
z
z
and k ( z )  0  ( z 2  1) ( z )   z ( z )   ( z )  
Thus, the decimal significance is  log 10
 ( z)
 z  log z 2  2 log z . ▌
10
10
( z )
z = 10
Examples:
and
( z ) 
z = 100
(10) = 7.619853024160526+  1024
(100) = 1.344179076744198+  102174
  10 
  100 
10
 7.694598626706419+  1024
Decimal significance = 2.008361
Adaptive Quadrature
100
 1.344313467781723+  102174.
Decimal significance = 4.000087
23
7/13/2017
Corollary 5.1: Let q( z ) 
1   z 
and r ( z )   z[1  ( z )]   ( z )
 z 
and s( z )  12 {( z 2  1)[1  ( z )]  z ( z )} .


Then (a) q( z ) z
 0 , q( z ) z
  .
(b)
(c)

z n [1  ( z )] z
 0 for all non-negative integers n.


z
z
r ( z )   [1  ( y)]dy and s( z )   r ( y)dy .
(d) q (z ) is a positive, decreasing, convex function.
(e)
z 00
 ( z)
 1  ( z ) 
 ( z)
.
1
z
z
z
 ( z)
( f ) z 0
approximates 1  (z) with significance > 2 log 10 z  .
z
Proof:
{1  (z) = (z) and (z) = (z)}  {q(z) = g(z), r(z) = h(z), s(z) = k(z)}
as defined in Lemma 5. Thus the geometry of Corollary 5.1 is the reflection
z  z of the geometry of Lemma 5. Assertions (a–f) above are the reflection of
(a–f) in Lemma 5. Argue (a)  (b)  (c)  (d)  (e)  (f ) accordingly. ▌
Adaptive Quadrature
24
7/13/2017