Chapter 3
Nonlinear Equation Methods
3.1
Nonlinear Models: Adiabatic CSTR
Consider the well-insulated (adiabatic) Continuous-flow Stirred-Tank Reactor (CSTR) where the single,
exothermic, mole-conserving reaction A → B takes place. The material and energy balances on the
reactor give
Quantity In − Quantity Out
=
Quantity Consumed + Accumulated
dC
= V k0 e−Ea /RT C + V
dt
dT
= −V ∆HR k0 e−Ea /RT C + V ρCp
dt
QCin − QC
QρCp (Tin − Tref ) − QρCp (T − Tref )
Q,
Tin ,
Cin
Parameter
Cin
Q
ρ
Cp
V
∆HR
k0
Ea
R
Tin
Q, T, C
Description
inlet conc
feed/effluent flow rate
liquid density
heat capacity
reactor volume
exothermic heat of rxn
preexp constant
activation energy
gas constant
inlet temp
Units
kgmol/m3
m3 /s
kg/m3
J/kg K
m3
J/kgmole
1/s
J/gmol
J/gmole/K
K
Value
3.0
6 × 10−5
1000
4190
1.8 × 10−2
2.09 × 108
4.48 × 106
62,800
8.314
300
Figure 3.1: Adiabatic CSTR with representative parameter values.
Defining the dimensionless variables and constants
x=
C
,
Cin
y=
T
,
Tin
τ=
Q
t,
V
γ=
57
Ea
,
RTin
α=
V k0
,
Q
β=
∆HR Cin
ρCp Tin
58
CHAPTER 3. NONLINEAR EQUATION METHODS
with data taken from [16] and presented in Fig. 3.1, we find that the modeling equations can be written
in much simpler form
dx
dτ
dy
dτ
=
1 − x − αxe−γ/y
=
1 − y + αβxe−γ/y
with
γ = 25.2,
α = 1.344 × 109 ,
β = 0.5
and subject to specified initial conditions.
In this Chapter, we focus on numerical techniques for computing solutions to nonlinear algebraic equations. One example of this class of problems is defined by
1 − x − αxe−γ/y
= 0
(3.1)
−γ/y
= 0
(3.2)
1 − y + αβxe
Solutions of the set of coupled nonlinear equations define the steady-state solutions to this CSTR model.
While not generally possible for sets of nonlinear equations, (3.1)-(3.2) can be simplified to a set of
two equivalent (one nonlinear and one linear) equations with the property that the nonlinear equation
is decoupled from the second equation; furthermore, because the second equation is linear, it can be
solved exactly given the solution to the first. This transformation effectively performs the equivalent of
the forward elimination factorization to the set of nonlinear equations.
This transformation is performed by multiplying (3.1) by β and subsequently subtracting (3.2) to find
β(1 − x) + 1 − y = 0
or
y = β(1 − x) + 1.
This function can be substituted into the material balance equation (3.1) to find
−γ
= g(x) = 0
1 − x − αx exp
β(1 − x) + 1
(3.3)
The steady-state solutions to the CSTR model are now defined in terms of the roots of the nonlinear
equation g(x) = 0; because this is a nonlinear equation, it is possible there are no, one, several, or
infinite solutions. It is possible to gain some insight into the behavior of g(x) = 0 by considering how
the solution(s) change as α is varied; because α is inversely proportional to Q at steady state, and by
using (3.3), we can see that
α → 0 (Q → ∞)
α → ∞ (Q → 0)
=⇒
=⇒
x→1
x→0
no conversion of reactant
complete conversion
It is possible to take advantage of another unique property of (3.3) to determine all possible solutions
to (3.3): while it is not possible to immediately assess the number of solutions to (3.3), one method for
computing all solutions is to switch the solution procedure around: instead of computing x for some α,
compute α for some x:
γ
1−x
.
(3.4)
exp
α=
x
β(1 − x) + 1
This takes advantage of the range of x being limited to the unit interval. We then plot x as a function
of α (Fig. 3.2).
59
3.2. NEWTON’S METHOD
1
0.8
x
0.6
0.4
0.2
0
0
1
2
α
3
4
9
x 10
Figure 3.2: Steady-states for the adiabatic CSTR as a function of α.
3.1.1
Turning-point bifurcations
It is interesting to consider the solutions which satisfy the modeling equation (3.3) and that make the
first derivative condition (3.5) vanish. This gives two equations from which potentially either a variable
or a parameter can be eliminated. For example, by using the modeling equation in the form (3.4), if
we set the derivative computed in (3.5) equal to zero and substitute in the equation for α, we find the
zero-derivatives can be calculated from the algebraic equation
#
"
x−1
−γβ
= 0
−1 +
+ (x − 1)
2
x
[(1 − x)β + 1]
2
2
−x [(1 − x)β + 1] + (x − 1) [(1 − x)β + 1] − γβ(x − 1)x =
(1 − x)2 β 2 + 2(1 − x)β + 1 + γβ(x2 − x) =
0
0
(γβ + β 2 )x2 − (2β 2 + 2β + γβ)x + β 2 + 2β + 1 =
0
so
p
β 2 (2β + 2 + γ)2 − 4β(γ + β)(β + 1)2
x=
= 0.9035, 0.1938.
2β(γ + β)
Compare these points to the values observed in left figure of Fig. 3.2. We note that multiple solutions
will not be found when
β 2 (2β + 2 + γ)2 − 4β(γ + β)(β + 1)2 < 0.
β(2β + 2 + γ) ±
This illustrates an important characteristic of nonlinear systems: that multiple solutions can exist and
can correspond to true physical solutions – for the adiabatic CSTR, only two of these are normally seen
due to the saddle-type stability of the intermediate state (the dynamics of the system near the steady
states will be discussed in the next chapter). We note that the Newton procedure can compute all of the
steady states, regardless of their stability characteristics.
3.2
Newton’s Method
We now solve for solutions to g(x) = 0 using the iterative Newton’s method for a specified set of α and
other parameters. If x0 is our guess for a solution, a more refined solution x1 can be computed using the
60
CHAPTER 3. NONLINEAR EQUATION METHODS
Taylor’s series of g evaluated at the current solution estimate x0 :
1 d2 g dg 1
0
(x1 − x0 )2 + h.o.t.
(x
−
x
)
+
g(x) ≈ g(x0 ) +
dx x0
2! dx2 x0
where h.o.t. refers to order three and higher derivative terms. To compute the more refined solution
estimate x1 , a linear curve f (x) tangent to g(x) at the current solution estimate g(x0 ) is defined from
the Taylor’s series expansion by
dg (x − x0 )
f (x) = g(x0 ) +
dx x0
so setting the right side of the equation above equal to zero and solving for the corresponding x = x1
gives
g(x0 )
x1 = x0 −
dg/dx|x0
with
dg
= −1 − α exp
dx
−γ
β(1 − x) + 1
− αx exp
−γ
β(1 − x) + 1
"
−γβ
[β(1 − x) + 1]2
#
(3.5)
for this example.
Produced by this computation is a (hopefully) better estimate x1 of a true solution to g(x) = 0; the
word hopefully is used because the actual convergence behavior of the Newton procedure can be difficult
to predict. We can continue to refine the solution estimate by replacing the previous estimate x0 with
the newer estimate x1 and then repeating the procedure until a sufficiently accurate solution is reached.
3.2.1
Convergence
The convergence of Newton’s method (for cases where the first g ′ and second g ′′ derivatives do not
vanish and for sufficiently-good initial guesses) is said to be quadratic: if x = r is the numerical value of
the root we are attempting to find, in the neighborhood of x = r with ǫ = x − r,
g(x)
1
≈ g(r) + (x − r)g ′ (r) + (x − r)2 g ′′ (r)
2
1 2 ′′
′
= 0 + ǫg (r) + ǫ g (r)
2
Differentiating each term of the right side of the equation above with respect to x (i.e., with respect to
ǫ) gives
g ′ (x) = g ′ (r) + ǫg ′′ (r).
Thus we can analyze our Newton scheme to find
x1
x1 − r
ǫ1
g(x0 )
g ′ (x0 )
g(x0 )
= x0 − r − ′ 0
g (x )
0 ′
ǫ g (r) + 21 (ǫ0 )2 g ′′ (r)
= ǫ0 −
g ′ (r) + ǫ0 g ′′ (r)
= x0 −
61
3.2. NEWTON’S METHOD
=
=
ǫ0 g ′ (r) + (ǫ0 )2 g ′′ (r) − ǫ0 g ′ (r) − 12 (ǫ0 )2 g ′′ (r)
g ′ (r) + ǫ0 g ′′ (r)
1 0 2 ′′
2 (ǫ ) g (r)
′
g (r) + ǫ0 g ′′ (r)
so for sufficiently small ǫ0 ,
ǫ1 ≈ (ǫ0 )2
g ′′ (r)
= C(ǫ0 )2 .
2g ′ (r)
Thus, we see that after six iterations, the error becomes
ǫ6 = ǫ0 Cǫ0
63
which normally will be sufficiently accurate if |Cǫ0 | < 1. It is important to point out, however, that
choosing initial guesses based on |Cǫ0 | < 1 do not necessarily have to converge to the fixed points used
to calculate the values of C.
The analysis above illustrates the quadratic convergence behavior of the Newton method when the
estimate of the solution approaches the true value. Because the next value of the error ǫ1 is proportional
to the square of the previous error ǫ0 , the accuracy of the solution estimates improves with each iteration
in such a way that the number of accurate digits doubles with each iteration, i.e., if ǫ0 = 0.001, then
ǫ1 = 0.00001 in the neighborhood of the converged solution. We note that explicit computation of g ′
and g ′′ may be used to gain further insight into the true accuracy of the solution.
Casting the Newton procedure in terms of updates
u1
=
u2
g(x0 )
g ′ (x0 )
(x0 − r)g ′ + (1/2)(x0 − r)2 g ′′
−
g ′ + (x0 − r)g ′′
g(x1 )
− ′ 1
g (x )
g(x0 + u1 )
− ′ 0
g (x + u1 )
(x0 + u1 − r)g ′ + (1/2)(x0 + u1 − r)2 g ′′
−
g ′ + (x0 + u1 − r)g ′′
(x0 − r)g ′ + (1/2)(x0 − r)2 g ′′ + u1 g ′ + (1/2)[(u1 )2 + 2u1 (x0 − r)]g ′′
−
g ′ + (x0 + u1 − r)g ′′
−u1 [g ′ + (x0 − r)g ′′ ] + u1 g ′ + (1/2)[(u1 )2 + 2u1 (x0 − r)]g ′′
−
g ′ + (x0 + u1 − r)g ′′
(1/2)(u1 )2 g ′′
− ′
g + (x0 + u1 − r)g ′′
g ′′
because x0 + u1 − r = x1 − r = small.
− ′ (u1 )2
2g
= −
=
=
=
=
=
=
≈
Because this analysis was done in the neighborhood of the true solution point r, we see that convergence
of u to zero implies convergence of x to r, and that the (negative!) exponent of the update will double
during each iteration. This is demonstrated in the following computational example.
62
3.2.2
CHAPTER 3. NONLINEAR EQUATION METHODS
Computational example: Quadratic convergence of Newton’s method
We consider the following portion of a Matlab script written to solve the single-equation version of the
adiabatic CSTR problem (3.3)
for iters = 1:8
g
= 1 - x - alph*x*exp(-gam/(bet*(1-x)+1));
gp = - 1 - alph*exp(-gam/(bet*(1-x)+1)) ...
- alph*x*exp(-gam/(bet*(1-x)+1))*(-gam*bet/(bet*(1-x)+1)^2);
update = g/gp;
x = x - update;
end
Given an initial guess of x0 = 0.4, the updates converge with the following behavior
Update
Iteration
1
-1.58e-01
2
-8.42e-02
3
-3.84e-02
4
-8.84e-03
5
-4.51e-04
-1.15e-06
6
7
-7.41e-12
8
3.76e-16
and so the exponent of the update indeed nearly doubles during each iteration as the algorithm approaches
the true solution (note that the update does now go below 1 × 10−16 in magnitude because this value is
approximately machine roundoff). A graphical view of the Newton iterations is shown in Fig. 3.3, where
the red-line segments are plots of the linear function f (x) while the blue curve is g(x).
0.5
g(x)
0
−0.5
−1
−1.5
0.4
0.45
0.5
0.55
x
0.6
0.65
0.7
Figure 3.3: Convergence of the Newton scheme.
Before concluding this section, several important general points on the Newton method must be made:
63
3.3. SETS OF NONLINEAR EQUATIONS: THE NEWTON-RAPHSON METHOD
• The numerical technique will not converge when the derivative vanishes at the root;
• The solution computed can depend strongly on the initial guess: for this problem, the initial guess
x0 = 0.2 actually converges to the lowest conversion steady state (x ≈ 1), while the guess x0 = 0.8
converges to the highest conversion steady state (x ≈ 0);
3.3
Sets of Nonlinear Equations: The Newton-Raphson Method
Returning to the adiabatic CSTR modeling equations (3.1, 3.2) and writing them in vector form
0 = g(z)
with
z=
x
y
(3.6)
with
g1 (x, y)
g2 (x, y)
= 1 − x − αxe−γ/y
= 1 − y + αβxe−γ/y
we can write the Taylor’s series expansion of g at (x0 , y 0 ) as
g1 (x, y) ≈
g2 (x, y) ≈
∂g1 (x − x0 ) +
g1 (x , y ) +
∂x x0 ,y0
∂g2 (x − x0 ) +
g2 (x0 , y 0 ) +
∂x 0 0
0
0
x ,y
or, neglecting the second and higher-order terms
∂g1 (y − y 0 ) +
∂y x0 ,y0
∂g2 (y − y 0 ) +
∂y 0 0
x ,y
∂ 2 g1 (x − x0 )(y − y 0 ) + . . .
∂x∂y x0 ,y0
∂ 2 g2 (x − x0 )(y − y 0 ) + . . .
∂x∂y 0 0
x ,y
g(z) ≈ g(z0 ) + J(z − z0 ).
(3.7)
For the adiabatic CSTR model, the Jacobian matrix J is found to be
#
"
0
0
−1 − αe−γ/y
−αx0 e−γ/y (γ/(y 0 )2 )
.
J=
0
0
αβe−γ/y
−1 + αβx0 e−γ/y (γ/(y 0 )2 )
This gives us the Newton-Raphson procedure,
z1 = z0 − J−1 g(z0 ).
(3.8)
Of course, we do not actually invert the Jacobian J; instead, we use the Gaussian elimination procedure
described in the previous section on linear systems to solve
J0 u1 = −g(z0 )
with
u1 = z1 − z0
for the update vector u, which is used to compute the refined solution estimate z1 from
z1 = z0 + u1 .
64
3.4
CHAPTER 3. NONLINEAR EQUATION METHODS
Computing Derivatives with Finite-Differences
We can compute approximate values of the derivatives needing for Newton’s method with finite differences: if δ is a small number (but significantly larger than the machine roundoff error), g is a vector of
nonlinear equations, and x is the vector of variables, for centered finite differences
gi (x0j + δ) − gi (x0j − δ)
∂gi ≈
∂xj x0
2δ
and for forward differences
gi (x0j + δ) − gi (x0j )
∂gi ≈
∂xj x0
δ
The Jacobian array can be constructed using either of these approaches
∂g ∂g
···
J|x0 =
∂x1 ∂x2
3.4.1
Computational example: Finite-difference Jacobian Newton-Raphson
We now consider the problem of finding all solutions to (3.1)-(3.2) using a Newton-Raphson method
NRfd.m and a grid of initial guesses
for x0 = 0.1:0.2:0.9
for y0 = 0.1:0.2:1.4
z0 = [x0; y0];
[z cFlag] = NRfd(’adcstrFun1’,z0);
...
Plotting the results of the converged solutions reveals the three different solutions to this model for the
parameter values used throughout this Chapter (see Fig. 3.4).
3.5
Jacobian-Free Solution Methods
The Newton-Raphson update can be computed using
−1 M×1
uM×1 = − JM×M
g
and is used to refine an estimate of the solution zM×1 :
z1 = z0 + u
If V is an orthonormal basis for u, we can write the update as
uM×1 = VM×N yN ×1
N ≤M
and so computing the update in the Newton-Raphson procedure becomes
JM×M VM×N yN ×1 = −gM×1
3.8. LINEARIZATION AND THE GEOMETRY OF PHASE SPACE
3.8
69
Linearization and the Geometry of Phase Space
As a first step in understanding the dynamical behavior of nonlinear systems, let us put some of the facts
we already understand together: if we calculate the three steady-state solutions zs i , i = 1, 2, 3 of the
nonadiabatic CSTR model
dz
= g(z)
dτ
for the parameter values given previously, and linearize the modeling equations [see (3.7)] at these
solutions
dz
dτ
dζ
dτ
=
g|zs i + J|zs i (z − zsi )
=
J|zs i (z − zs i ) because g|zs i = 0
=
J|zs i ζ
by defining the deviation variable ζ = z − zsi . The solutions, subject to the sets of initial conditions z0
are written as
z = zs i + d1 eλ1 τ u1 + d2 eλ2 τ u2
with the coefficients di computed from
Ud = z0 − zsi .
The three steady-state solutions and the corresponding parameter values for the ODE solutions valid in
the neighborhood of the steady solution points are given in the following table.
Variable
xs
ys
λ1 , λ2
U
Low Conv
0.9810
1.0095
-1.00,
-0.78 −0.99
0.89
0.04 −0.44
Medium Conv
0.6902
1.1549
-1.00,
1.47 −0.99
0.89
0.07 −0.44
High Conv
0.0158
1.4921
-57.59,
-1.00 −0.89
0.17
0.44 −0.98
At this point, we make the following observations and pose a question to be answered in the next section:
1. The three steady states demonstrate different aspects of stability and instability – the upper and
lower steady states attracting all initial conditions, and the middle demonstrating saddle stability,
where only initial conditions on the “stable eigenvector” reach the steady state;
2. The issues of stability raise the question of how one determines which initial conditions are attracted
to each of the stable steady states. This requires numerical integration of the fully nonlinear system.
Specifically, we compute numerically the invariant manifolds which form the skeleton organizing
the dynamics in the phase space.
Stable Manifold W s : Points, which when followed forward in time, asymptotically approach
the fixed point – W s is tangent to the space spanned by the stable eigenvectors of the fixed
point.
Unstable Manifold W u : Points, which when followed in reverse time, asymptotically approach
the fixed point – W u is tangent to the space spanned by the unstable eigenvectors of the
fixed point.
70
CHAPTER 3. NONLINEAR EQUATION METHODS
Med Conv
Low Conv
1.06
Hi Conv
1.2
1.6
−57.59
−1
−1
1
1.15
y
1.02
y
y
1.04
0.98
0.96
0.9
1
x
−0.7844
1.1
1.1
1.5
1.4
1.478
0.6
0.7
0.8
x
−0.1
−1
0
0.1
x
Figure 3.7: The three phase portraits of the three linearized systems. The smaller points indicate
the initial conditions of the trajectories computed as exact solutions to the linearized equations; the
eigenvectors are plotted, and the magnitude of the associated eigenvalues are also given.
3.9
Numerical Integration Background
Consider the single differential equation model (2.6) of the quenching problem with the steel sphere
temperature variable redefined as x = Ts for clarity in the following derivations:
dx
= g(x) = γx + δ
dt
subject to the initial condition x(t = 0) = 80o C and the corresponding particular solution for αw = 0.02
and αs = 0.1 sec−1 :
x(t) = 30 + 50e−0.12t
We wish to compare the exact solution computed over 0 ≤ t ≤ 30sec to the solution approximated with
the (forward) Euler method:
dx = g(xn )
(exact)
dt t=tn
xn+1 − xn
≈
tn+1 − tn
and so
xn+1 = xn + (tn+1 − tn )g(xn )
= xn + ∆t g(xn ).
If we consider xn = x(tn ) to be a point on the exact solution to the differential equation, we can
write the Taylor’s series approximation to the time-dependent behavior of x (the exact solution) in the
neighborhood of xn as
dx 1 d2 x x(t) = x(tn ) +
(t − tn ) +
(t − tn )2 + h.o.t.
dt tn
2 dt2 tn
d2 x 1
dx x(tn+1 ) = x(tn ) + ∆t + ∆2t 2 + h.o.t.
dt tn
2
dt tn
1
dg(x) = x(tn ) + ∆t g(xn ) + ∆2t
+ h.o.t.
2
dt tn
71
3.10. TIME-STEPPING METHODS
and so subtracting the results from one step of the Euler integrator and retaining only terms of order ∆2t
or less gives an estimate of the error dn+1 generated by one step of the Euler integrator:
dn+1
=
=
=
x(tn+1 ) − xn+1
1 2 dg(x) x(tn ) + ∆t g(xn ) + ∆t
− xn − ∆t g(xn )
2
dt tn
1 2 dg(x) ∆t
2
dt tn
=
=
<
so
dn+1
<
1 2 2
∆ [γ xn + γδ]
2 t
1 2 2
∆t γ 30 + 50e−0.12t + γδ
2
1 2 2
∆ γ (80 + γδ)
2 t
1.146∆2t
for our particular problem.
This means halving the step size decreases the local error by a factor of 1/4, however, twice as many steps
are required to integrate the system over the original time step and so if we assume errors accumulate in
an additive manner over short time intervals, we see the net effect of halving the step size when using
the Euler method is to halve the error. This is why the Euler method is called a first order method,
even though it is accurate to order 2. A plot of the true solution to our scalar ODE problem and the
predictions of the Euler integrator are shown in Fig. 3.8. We notice that for this particular example, both
the local and global error decrease with time – the first is due to the exponential term e−0.12tn shrinking
with time and the second is due to the Euler approximation and the true system being attracted to the
same, steady-state solution.
There are other issues that have not be discussed, but should be mentioned:
• The Euler method works for sets of equations by simply employing vector notation:
zn+1 = zn + ∆t g(zn )
subject to z0
This approach was used to compute the trajectories shown in Fig. 3.9.
• The method works with mixed algebraic/ODE systems provided the AEs are solved at each time
step for some subset y the state variables:
zn+1
=
f (zn , yn ) =
zn + ∆t g(zn , yn )
0
subject to z0 ;
• Stability issues set an upper practical limit on the time step sizes for most numerical integration
techniques, because instabilities can cause “runaway” to infinitely large global errors in systems
which in reality display bounded, long-time behavior.
3.10
Time-Stepping Methods
Use of built-in Matlab numerical integrators with the AEModel and other classes we have defined wrapping the existing matlab methods.
72
CHAPTER 3. NONLINEAR EQUATION METHODS
80
Exact
∆ =5
t
∆t=2.5
75
70
65
o
Ts, x ( C)
60
55
50
45
40
35
30
0
5
10
15
time (sec)
20
25
30
Figure 3.8: Exact solution and two approximate solutions to the quenching problem computed using
a forward Euler method.
3.11
Review Problems
1. Solve the 8-equation, 7-unknown flash drum material balance below subject to zM = 0.1 and
zO = 0.9 directly using a Newton-Raphson method.
xM + xO
yM
yM
yO
+ yO
V +L
yM V + xM L
zM F
yO V + xO L
= 1
= 220xM
= 0.5xO
= 1
= F
= zM F
= 10
= zO F
Prove your computed solution is a true solution to the modeling equations; is this an overdetermined (least-squares) system, or is there a redundant equation?
73
3.11. REVIEW PROBLEMS
−57.59
1.5
1.4
−1
y
1.3
1.2
−1
1.478
1.1
−1
1
−0.7844
0
0.2
0.4
0.6
0.8
1
x
Figure 3.9: Representative trajectories and saddle stable/unstable manifolds computed using a numerical integrator.