COMO 201: Numerical Methods and Structural Modelling, 2003
Computational Modelling 201
(Numerical Methods)
* DRAFT *
Lecture Notes
Dr John Enlow
Last modified July 22, 2003.
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
1
Contents
1 Preliminary Material
4
2 Introduction to Numerical Analysis
2.1 What is Numerical Analysis? . . . . . . . . .
2.1.1 Numerical Algorithms . . . . . . . . .
2.1.2 The Importance of Numerical Methods
2.2 Unavoidable Errors in Computing . . . . . . .
2.2.1 Floating Point Numbers . . . . . . . .
2.2.2 Roundoff (Machine) Error . . . . . . .
2.2.3 Truncation Error . . . . . . . . . . . .
2.2.4 Overflow and Underflow . . . . . . . .
2.2.5 Error Propagation . . . . . . . . . . .
5
5
5
7
7
8
8
9
9
9
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3 Iteration and Root Finding
3.1 The Standard Iteration Method . . . . . . . . . . . . .
3.2 Aitken’s Method for Acceleration (∆2 Method) . . . .
3.2.1 The Mean Value Theorem (MVT) . . . . . . . .
3.2.2 Aitken’s Approximate Solution . . . . . . . . .
3.2.3 The ∆2 Notation . . . . . . . . . . . . . . . . .
3.2.4 Aitken’s Algorithm . . . . . . . . . . . . . . . .
3.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . .
3.3.1 Graphical Derivation of Newton’s Method . . .
3.3.2 Pathological Failure of Newton’s Method . . . .
3.3.3 Convergence Rate . . . . . . . . . . . . . . . . .
3.4 Newton’s Method and Fractals . . . . . . . . . . . . . .
3.5 Systems of Equations . . . . . . . . . . . . . . . . . . .
3.5.1 Two Dimensions . . . . . . . . . . . . . . . . .
3.5.2 Higher Dimensions . . . . . . . . . . . . . . . .
3.5.3 Newton-Raphson Method for Nonlinear Systems
3.5.4 Example: Newton’s Method for Two Equations
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
11
. . . . . . . 11
. . . . . . . 13
. . . . . . . 13
. . . . . . . 13
. . . . . . . 14
. . . . . . . 15
. . . . . . . 15
. . . . . . . 16
. . . . . . . 17
. . . . . . . 18
. . . . . . . 18
. . . . . . . 20
. . . . . . . 20
. . . . . . . 22
of Equations 22
. . . . . . . 23
4 Interpolation and Extrapolation
4.1 Lagrange (Polynomial) Interpolation . . . . . . . . . . . . . . . . .
4.1.1 Examples of Lagrange Interpolation . . . . . . . . . . . . . .
4.2 Least Squares Interpolation . . . . . . . . . . . . . . . . . . . . . .
2
25
25
27
28
3
COMO 201: Numerical Methods and Structural Modelling, 2003
4.2.1
4.2.2
4.2.3
Fitting a Straight Line to Data (“Linear Regression”) . . . .
Least Squares from an Algebraic Perspective . . . . . . . . .
General Curve Fitting with Least Squares . . . . . . . . . .
5 Numerical Differentiation
5.1 Finite Differencing . . . . . . . . . . . . .
5.1.1 Second Derivative . . . . . . . . . .
5.2 Ordinary Differential Equations . . . . . .
5.2.1 High Order ODEs . . . . . . . . . .
5.2.2 Example: An Initial Value Problem
28
29
31
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
32
32
33
33
33
33
6 Numerical Integration
6.1 The Trapezoid Rule . . . . . . . . . . . . . . . .
6.2 Simpson’s Rule . . . . . . . . . . . . . . . . . .
6.3 Gauss-Legendre Integration . . . . . . . . . . .
6.3.1 Computational Determination of Weights
. . . . . . .
. . . . . . .
. . . . . . .
and Nodes
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
35
35
36
37
7 Fundamentals of Statics and Equilibrium
7.1 Forces and Particles . . . . . . . . . . . . .
7.2 Newton’s Laws . . . . . . . . . . . . . . .
7.3 Static Equilibrium . . . . . . . . . . . . .
7.4 Solution of Mechanics Problems . . . . . .
7.4.1 SI Units in Mechanics . . . . . . .
7.5 More on Forces . . . . . . . . . . . . . . .
7.5.1 Coordinates . . . . . . . . . . . . .
7.5.2 Example I . . . . . . . . . . . . . .
7.5.3 Example II . . . . . . . . . . . . .
7.6 Equilibrium of a Particle . . . . . . . . . .
7.6.1 Example . . . . . . . . . . . . . . .
7.7 Free Body Diagrams . . . . . . . . . . . .
7.7.1 Example . . . . . . . . . . . . . . .
7.8 Rigid Bodies: Force Equivalents . . . . . .
7.8.1 Moment . . . . . . . . . . . . . . .
7.8.2 Varigon’s Theorem . . . . . . . . .
7.8.3 Equivalent Forces Theorem . . . .
7.8.4 Beam Corollary . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
40
40
40
40
41
41
41
42
43
43
44
44
45
45
46
46
47
48
49
. . . .
. . . .
. . . .
. . . .
(IVP)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 1
Preliminary Material
Lecturer
First half:
Second half:
Dr John Enlow, Room 518, Science III ([email protected] ).
Dr R.L. Enlow, Room 220, Science III.
Course structure:
• Two lectures (Tuesday and Thursday at 1pm, Seminar Room),
• One one-hour tutorial (not at the computers, Thursday 2pm, Seminar Room).
• One two-hour laboratory (using computers, Friday 10am-12pm, Lab A).
The tutorial exercise and the laboratory exercise are both due at 1pm on Friday.
Assessment:
The internal assessment is comprised of the mid-semester test (40%) and the weekly
exercises (60%). The final grade is either your final exam score or one third of your
internal assessment plus two thirds of your final exam score, whichever is greater.
Computer Packages Used (First Half of Course)
All problems will be solved in, at your preference, MATLAB or C. It is assumed
you have good knowledge of one of these.
Books
There is no prescribed text for this half of the course. Useful references include:
• Numerical Recipes in C, Press, Teukolsky, Vetterling and Flannery.
• Numerical Methods with MATLAB, Recktenwald.
• Fundamentals of Computer Numerical Analysis, Freidman and Kandel.
4
Chapter 2
Introduction to Numerical
Analysis
2.1
What is Numerical Analysis?
Numerical analysis is a mathematical discipline. It is the study of numerical methods, which are mathematical techniques for generating approximate solutions to
problems (for example by using an iterative process or a series approximation).
Numerical methods can be suitable for problems that are very difficult or impossible to solve analytically.
2.1.1
Numerical Algorithms
Numerical methods are implemented via algorithms. An algorithm is a set of instructions, each of which is composed of a sequence of logical operations. These
instructions use the techniques described by the numerical method to solve a problem. A desired accuracy for the answer is generally specified, so that the algorithm
terminates with an approximate solution after a finite number of steps.
An example √
of a numerical method is a scheme based on Newton’s method
to calculate√ a for any given a > 0. We assume that we are given an initial
estimation a ≈ x0 . Each step of the algorithm
consists of calculating xn+1 ,
√
a (hopefully) better approximation to a, from the previous approximation
xn . This is done using the rule
1
a
xn +
,
n ∈ N.
2
xn
√
It can be shown that xn → a as n → ∞. To enable an algorithm (which
implements the above method) to terminate after a finite number of steps,
we might test xn after each step, stopping when
xn+1 =
|xn − xn−1 | < 10−6 .
5
COMO 201: Numerical Methods and Structural Modelling, 2003
6
There is more to numerical analysis than just development of methods and
algorithms. We also need to know how to predict the behaviour of both the method
and its associated algorithms. Common questions include:
• Under what conditions will the method/algorithm produce a correct approximate solution to the problem? (e.g. convergence issues, stability).
• What can we say about the errors in the approximate solutions? (error analysis).
• How long does it take to find a solution to a given accuracy? (O(n log(n))
etc...).
Answering these questions may well be more difficult than developing either the
method or algorithm, and the answers are often non-trivial.
For example, consider the standard iterative method for solving x = f (x)
on a ≤ x ≤ b (where at each step we calculate xn+1 = f (xn )). Sufficient
conditions for both a unique solution to exist and for the method to be
guaranteed to converge to this unique solution are:
1. f (x) continuous on [a, b],
2. x ∈ [a, b] ⇒ f (x) ∈ [a, b],
3. ∃L ∈ R, ∀x ∈ [a, b],
y
|f 0 (x)| ≤ L < 1.
g(x) ≡ x
f (x)
x
This course will focus on numerical methods and algorithms rather than on numerical analysis. MATH 361, which some of you will take in first semester 2003,
focuses on numerical analysis. It provides a greater depth of understanding of the
derivation of the methods and of their associated errors and limitations. Numerical
analysis does not require the use of a computer because it focuses on theoretical
COMO 201: Numerical Methods and Structural Modelling, 2003
7
ideas. In contrast we will make regular use of computers in this course, spending
roughly 32 of the exercise time in Lab A.
2.1.2
The Importance of Numerical Methods
Computers are used for a staggering variety of tasks, such as:
• accounting and inventory control,
• airline navigation and tracking systems,
• translation of natural languages,
• monitoring of process control and
• modelling of everything from stresses in bridge structures during earthquakes
to possum breeding habits.
One of the earliest and largest uses of computers is to solve problems in science
and engineering. The techniques used to obtain such solutions are part of the
general area called scientific computing, which in turn is a subset of computational
modelling, where a problem is modelled mathematically then solved on computer
(the solution process being scientific computing).
Nearly every area of modern industry, science and engineering relies heavily on
computers. In almost all applications of computers, the underlying software relies
on mathematically based numerical methods and algorithms. While examples done
in this course focus on solving seemingly simple mathematical problems with numerical methods, the importance of these same methods to aspects of an enormous
range of real world problems can not be overstated. An understanding of these
fundamental ideas will greatly assist you in using complex and powerful computer
packages designed to solve real problems. They are also a crucial component of
your ability, as computational modelers, to solve modelled systems.
2.2
Unavoidable Errors in Computing
Representation of modelled systems in a computer introduces several sources of
error, primarily due to the way numbers are stored in a computer. These sources
of error include:
• Roundoff or Machine error.
• Truncation error.
• Propagated errors.
COMO 201: Numerical Methods and Structural Modelling, 2003
2.2.1
8
Floating Point Numbers
Definition
The representation of a number f as
f = s · m · Mx
where
• the sign, s, is either +1 or −1,
• the mantissa, m, satisfies 1/M ≤ m ≤ 1,
• the exponent, x, is an integer,
is called the floating point representation of f in base M .
The decimal number f = 8.75, written in its floating point representation
in base 10 is 0.875 · 101 , and has s = 1, m = 0.875 and x = 1. Its binary
equivalent is
f = (1000.11)2 = +(0.100011)2 · (2)(100)2
where s = 1, m = (0.100011)2 and x = (100)2 .
2.2.2
Roundoff (Machine) Error
In a computer the mantissa m of a number x is represented using a fixed number of
digits. This means that there is a limit to the precision of numbers represented in a
computer. The computer either truncates or rounds the mantissa after operations
that produce extra mantissa digits.
The absolute error in storing a number on computer in base 10 using d digits
can be calculated as follows. Let m̂ be the truncated mantissa, then the magnitude
of the absolute error is
= |s · m · 10x − s · m̂ · 10x | = |m − m̂| · 10x < 10−d 10x
Roundoff error can be particularly problematic when a number is subtracted from
an almost equal number (subtractive cancellation). Careful algebraic rearrangement
of the expressions being evaluated may help in some cases.
The expression
1015 ∗ (pi − 3.14159265359) = −206.7615373566167...
However when evaluated on my calculator the answer given is 590, 000!!
Clearly the calculator’s answer is catastrophically incorrect.
COMO 201: Numerical Methods and Structural Modelling, 2003
2.2.3
9
Truncation Error
This type of error occurs when a computer is unable to evaluate explicitly a given
quantity and instead uses an approximation.
For example, an approximation to sin(x) might be calculated using the
first three terms of the Taylor series expansion,
x3 x5
sin(x) = x −
+
+ E(x)
3!
5!
where
x7
|E(x)| ≤ .
7!
If x is in the interval [−0.2, 0.2] then the truncation error for this operation
is at most
0.27
< 3 · 10−9
7!
2.2.4
Overflow and Underflow
Underflow and overflow errors are due to a limited number of digits being assigned
to the exponent, so that there is a largest and smallest magnitude number that can
be stored (N and M say). Attempts to store a number smaller than M will result
in underflow, generally causing the stored number to be set to zero. Overflow errors
are more serious, occurring when a calculated or stored number is larger than N .
These generally cause abnormal termination of the code.
Consider the modulus of a complex number a + ib. The obvious way to
calculate this is using
√
|a + ib| = a2 + b2 ,
however this method is clearly prone to overflow errors for large a or b. A
much safer approach is with the relation




|a + ib| = 


2.2.5
q
|a| 1 + (b/a)2 , |a| ≥ |b|
q
|b| 1 + (a/b)2 , |a| < |b|
Error Propagation
Errors in calculation and storage of intermediate results propagate as the calculations continue. It is relatively simple to derive bounds on the propagated errors
after a single operation, but combining many operations can be problematic and
may lead to complicated expressions.
COMO 201: Numerical Methods and Structural Modelling, 2003
Consider quantities x̂ and ŷ approximating ideal values x and y. Let x and
y be bounds on the magnitudes of the absolute errors in these approximations, so that
|x − x̂| ≤ x ,
|y − ŷ| ≤ y .
Then, using the triangle inequality, the propagated error under the addition
operation is
|(x + y) − (x̂ + ŷ)| ≤ |x − x̂| + |y − ŷ| ≤ x + y .
10
Chapter 3
Iteration and Root Finding
Solving nonlinear equations is a major task of numerical analysis and the iteration
method is a prototype for many methods dealing with these equations.
3.1
The Standard Iteration Method
The standard iteration method is designed to solve equations of the form
x = f (x)
where f (x) is a real valued function defined over some interval [a, b].
An algorithm for the standard iteration method.
1. Choose an initial approximate solution x0 , a tolerance and a maximal number of iterations N . Set n = 1.
2. Create the next iteration using the relation
xn = f (xn−1 ).
3. If |xn − xn−1 | > and n < N then increment n and go to step 2.
4. If |xn − xn−1 | > then the maximal number of iterations has been
reached without the desired convergence. Raise a warning that this
is the case.
5. Output xn , the approximate solution to x = f (x).
Clearly the algorithm finishes when successive approximations are less than apart.
Ideally we have
lim xn = s,
where
s = f (s)
n→∞
11
12
COMO 201: Numerical Methods and Structural Modelling, 2003
and |xn − s| monotonically decreasing as n increases. In fact we have already stated
a theorem that gives us sufficient conditions for behaviour similar to this - see the
example in section 2.1.1.
Example where |f 0 (x)| < 1 on the interval. Good convergence.
f (x) = E −x +
1
sin(10x),
10
x0 =
1
5
1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
1
Example where ∃x ∈ [0, 1] s.t. |f 0 (x)| > 1. Poor convergence!
f (x) = E −x +
1
sin(20x),
5
x0 =
1
5
1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
1
13
COMO 201: Numerical Methods and Structural Modelling, 2003
3.2
Aitken’s Method for Acceleration (∆2 Method)
Aitken’s ∆2 method improves on the convergence rate of the S.I.M. in cases where
the S.I.M. converges to a unique solution. It does this by generating an approximate
expression for f 0 (s), then using this information to predict what the S.I.M. is going
to converge to. In deriving Aitken’s method we’ll need to use the mean value
theorem.
3.2.1
The Mean Value Theorem (MVT)
Recall the Mean Value Theorem (M.V.T):
Let f (x) be differentiable on the open interval (a, b) and continuous on the closed
interval [a, b]. Then there is at least one point c in (a, b) such that
f 0 (c) =
f (b) − f (a)
b−a
y
f (b)
f (x)
f (a)
a
3.2.2
c1
c2
b
x
Aitken’s Approximate Solution
Let f be a ‘nice’ function, with both f (x) and f 0 (x) continuous and differentiable in
the domain of interest. Suppose that we have an approximation xa to the solution
s (where f (s) = s). Assume that the S.I.M., starting at xa will converge to s. Let
xb and xc represent two iterations of the S.I.M., so that xb = f (xa ) and xc = f (xb ).
Then for some ca between xa and s,
xb − s = f (xa ) − f (s) = (xa − s) f 0 (ca ) by the M.V.T.
(3.1)
COMO 201: Numerical Methods and Structural Modelling, 2003
14
and similarly, for some cb between xb and s,
xc − s = (xb − s) f 0 (cb )
(3.2)
Both xa and xb are close to s, since xa is an approximate solution, so that ca and cb
are also close to s (because |ca − s| < |xa − s| and |cb − s| < |xb − s|). With this in
mind, we approximate f 0 (ca ) and f 0 (cb ) by f 0 (s) in equations 3.1 and 3.2, giving:
xb − s ≈ (xa − s) f 0 (s)
(3.3)
xc − s ≈ (xb − s) f 0 (s)
(3.4)
Subtracting Eq. 3.3 from Eq. 3.4 yields:
f 0 (s) ≈
xc − x b
xb − x a
Substituting this expression back into Eq. 3.3 and rearranging yields the following
approximate solution to f (x) = x.
(xb − xa )2
s ≈ x ≡ xa −
xc − 2xb + xa
∗
In general x∗ will be much closer to s than xa , xb and xc . We use this expression
roughly as follows:
• Start at an estimated solution xa .
• At each iteration calculate xb = f (xa ), xc = f (xb ) and x∗ (using the boxed
equation above).
• Set xa for the next iteration to be the current value of x∗ .
Before developing a full algorithm we’ll explain why this method is called Aitken’s
∆2 method.
3.2.3
The ∆2 Notation
This expression can be written more compactly using the forward difference operator
∆, which is defined by the three relations:
∆xn ≡ xn+1 − xn ,

∆
m
X
j=1

α j xn j  ≡
m
X
αj ∆xnj ,
j=1
n≥0
m ≥ 1, αj ∈ R, (∀j) nj ≥ 0
∆n E ≡ ∆ ∆n−1 E ,
n≥2
The more compact form of the boxed equation, which gives the method its name,
is given by
(∆xa )2
x∗ = x a −
.
∆2 x a
15
COMO 201: Numerical Methods and Structural Modelling, 2003
3.2.4
Aitken’s Algorithm
Aitken’s algorithm is as follows. x∗a represents Aitken’s accelerated approximate
solution, xb and xc are two iterations of the S.I.M. starting at x∗a .
1. Choose an initial approximate solution x∗a , a tolerance and a maximal number of iterations N . Set n = 0.
2. Compute xb = f (x∗a ) and xc = f (xb ).
3. Update x∗a with x∗a = x∗a −
(xb − x∗a )2
xc − 2xb + x∗a
4. Increment n.
5. If |f (x∗a ) − x∗a | > and n < N then go to step 2.
6. If |f (x∗a ) − x∗a | > then raise a warning that the maximal number of iterations
has been reached without the desired convergence.
7. Output the approximate solution x∗a .
3.3
Newton’s Method
Newton’s method, sometimes called the Newton-Raphson method, is designed to
find solutions of the equation
f (x) = 0,
where f (x) is some function whose roots are non-trivial to find analytically. Suppose
that we have already iterated a number of times, arriving at our approximation xk
to the true solution. Ideally the next iteration of the method will choose xk+1 so
that f (xk+1 ) = 0. If we approximate f (xk+1 ) using known quantities, then set this
approximation to zero, we can solve for a good value of xk+1 (so that f (xk+1 ) ≈ 0).
But what approximation should we use?
Newton’s method arises from approximating f (xk+1 ) using a Taylor series of
f (x) expanded about xk . Using the standard forward difference operator ∆, so
that ∆xk ≡ xk+1 − xk , this Taylor series is
(∆xk )2 d2 f df +
+ ...
f (xk+1 ) = f (xk + ∆xk ) = f (xk ) + ∆xk
dx xk
2
dx2 xk
Assuming that ∆xk is small, i.e. that we are close to the solution, f (xk+1 ) can be
approximated using the first two terms:
df f (xk+1 ) ≈ f (xk ) + ∆xk
dx xk
16
COMO 201: Numerical Methods and Structural Modelling, 2003
Setting this approximation of f (xk+1 ) to zero gives
df f (xk ) + (xk+1 − xk )
=0
dx xk
and solving for xk+1 yields
xk+1 = xk −
f (xk )
.
f 0 (xk )
This iteration is the basis of Newton’s method.
3.3.1
Graphical Derivation of Newton’s Method
Consider a root-finding method which sets xk+1 to be the intercept of the tangent
line to f (x) at xk with the x-axis.
y
f (xk )
f (x)
xk
x
xk+1
slope =
⇒
f 0 (xk ) =
∆y
∆x
0 − f (xk )
xk+1 − xk
⇒ f 0 (xk )(xk+1 − xk ) = −f (xk )
⇒
xk+1 − xk = −
and so
xk+1 = xk −
f (xk )
.
f 0 (xk )
f (xk )
f 0 (xk )
17
COMO 201: Numerical Methods and Structural Modelling, 2003
3.3.2
Pathological Failure of Newton’s Method
How else could Newton’s method fail?
• Divergence:
y
f (x1 )
f (x0 )
x
x0
x1
f (x)
• Cycles / oscillation:
y
f (x)
f (x1 )
f (x0 )
x
x0 , x2 , x4 , . . .
• Poor convergence (near zero slope).
The issues are often located near turning points.
x 1 , x3 , x5 , . . .
18
COMO 201: Numerical Methods and Structural Modelling, 2003
3.3.3
Convergence Rate
Newton’s method has quadratic convergence1 when close to the root2 - the error at
step k + 1 is of the order of the square of the error at step k. This is much better
than the other methods described (including Aitken’s ∆2 method) which only have
linear convergence.
3.4
Newton’s Method and Fractals
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
The fractal shown is obtained by considering Newton’s method when used on
the complex -valued equation
z 3 − 1 = 0.
1
In the case of repeated roots we have f 0 (x) = 0 at the roots, meaning that Newton’s method
only manages linear convergence.
2
Choosing a starting value close to the root is very important. Newton’s method depends on
a good starting value for convergence.
COMO 201: Numerical Methods and Structural Modelling, 2003
19
Newton’s method gives the iteration
zj+1
zj3 − 1
= zj −
.
3zj2
Looking at the region |z| < 2, we colour a pixel white if Newton’s method converges
to z = 1 and black if it converges to one of the two complex roots. All points
converge, with the exception of the origin and three points along the negative real
axis. Although we may have expected the three regions (one for each root) to be
identically shaped due to symmetry, the fractal behaviour is quite surprising! Points
near local extrema cause Newton’s method to shoot off towards other remote values.
Source File: nfrac.m
function.nfrac
%.Generates.a.fractal.based.on.which.root.Newton’s.
%.method.converges.to.when.considering.
%..........f(z).=.z^ 3.-.1.
%.We.colour.anything.converging.to.z=1.(the.real.root).black.
n=500;.m=zeros(n,n);
for.(it1=1:n)
.....for.(it2=1:n)
..........x.=.((it1-1)/(n-1).-0.5)*4;.
..........y.=.((it2-1)/(n-1).-0.5)*4;
..........if.(x*x+y*y<4)
..................m(it2,it1)=getcolour(.x.+.i.*.y.);
..........else
...............m(it2,it1)=0.5;
..........end
.....end
end
imagesc([-2.2],.[-2.2],m);
colormap(gray);
axis.square;
%-----------------------------------------------------------function.[.thecolour.].=.getcolour(.zz.)
%.This.calculates.20.iterations.of.Newton’s.method.
iterations=20;
thecolour=0;..%.Black.by.default.
for.u=1:iterations
....zz.=.zz.-.(zz^3.-.1)/(3.*.zz^2);
end
if.(abs(zz-1)<0.1).thecolour.=.1;.end.%.White.if.zz.~=.1.
COMO 201: Numerical Methods and Structural Modelling, 2003
3.5
20
Systems of Equations
Newton’s method can easily be applied to systems of nonlinear equations.
3.5.1
Two Dimensions
Consider the simultaneous equations
f (x, y) = 0,
and g(x, y) = 0.
We can think of the equation f (x, y) = 0 as representing a “level curve” at z = 0
of the function z = f (x, y). This level curve is the intersection of the function
z = f (x, y) with the z = 0 plane. Consider for example f (x, y) = x2 + y 2 − 3. Then
f (x, y) = 0 ⇒ x2 + y 2 − 3 = 0 ⇒ x2 + y 2 = 3,
and so the level curve is a circular region in the x, y-plane.
In[12]:= Plot3D@x ^ 2 + y ^ 2 - 3, 8x, -Pi, Pi<, 8y, -Pi, Pi<D;
15
10
5
0
2
0
-2
0
-2
2
In[13]:= ContourPlot@x ^ 2 + y ^ 2 - 3, 8x, -Pi, Pi<, 8y, -Pi, Pi<,
Contours ® 80<, ContourShading ® False, PlotPoints ® 50D;
3
2
1
0
-1
-2
-3
-3
-2
-1
0
1
2
3
COMO 201: Numerical Methods and Structural Modelling, 2003
21
Similarly the equation g(x, y) = 0 represents a level curve of z = g(x, y). The
simultaneous solution of f (x, y) = 0 and g(x, y) = 0 then represents the intersection
points of the two corresponding level curves.
For example, with f (x, y) = x2 + y 2 − 3 and g(x, y) = 2x2 + 41 y 2 − 3, we have:
Level Curve at z=0 for z=2x^2+Hy2L^2-3
3
2
1
0
-1
-2
-3
-3
-2
-1
1
0
2
3
3
2
1
0
-1
-2
-2
-1
0
1
2
3
Generally speaking the two curves representing f = 0 and g = 0 have no relation
to each other at all! Graphically, Newton’s method in two dimensions works by:
1. Starting with an approximate solution (xk , yk ).
2. Calculating the tangent planes to the surfaces z = f (x, y) and z = g(x, y).
3. Calculating the lines of intersection of the planes with the x, y-plane.
4. Calculating the intersection of the two lines. This intersection point is the
new approximate solution (xk+1 , yk+1 ).
COMO 201: Numerical Methods and Structural Modelling, 2003
3.5.2
22
Higher Dimensions
In higher dimensions the situation quickly gets more complicated. In three dimensions we’re looking at intersections of three level surfaces. Graphical interpretation
of the method becomes extremely difficult. However, computationally, Newton’s
method in any dimension can calculated with one algorithm!
3.5.3
Newton-Raphson Method for Nonlinear Systems of
Equations
A typical problem gives N equations to be solved in variables x1 , x2 , . . . , xN :
F1 (x1 , x2 , . . . , xN ) = 0
F2 (x1 , x2 , . . . , xN ) = 0
..
.. ..
.
. .
FN (x1 , x2 , . . . , xN ) = 0
Or, written more compactly,
Fi (x1 , x2 , . . . , xN ) = 0,
i = 1, 2, . . . , N.
The Taylor series for each component of F~ = (F1 , F2 , . . . , FN ) expanded about
~x = (x1 , x2 , . . . , xN ) is given in terms of the column vector δ~x by
Fi (~x + δ~x ) = Fi (~x) + J~i · δ~x + O(|δ~x |2 )
h
∂Fi ∂Fi
∂Fi
where the row vector J~i is given by J~i = ∂x
,
, . . . , ∂x
1 ∂x2
N
terms with total magnitude of the order of |δ~x |2 .
We then have the approximation
Fi (~x + δ~x ) ≈ Fi (~x) + J~i · δ~x .
i
and O(|δ~x |2 ) means
Let J be the matrix with rows J~1 , . . . , J~N , i.e. with elements
∂Fi
Ji,j ≡
,
∂xj
then setting Fi (~x + δ~x ) to zero gives
J δ~x = −F~ .
This is the basis of the multidimensional form of Newton’s method. Suppose, starting from ~x0 , that we have reached iteration k giving approximate solution ~xk . Then
the next step is calculated as follows.
1. Calculate F~ (~xk ).
2. Calculate the Jacobian matrix J at ~x = ~xk .
3. Solve J δ~xk = −F~ (~xk ) for δ~xk (e.g. with an LU decomposition).
4. Set ~xk+1 = ~xk + δ~xk .
We stop iterating when ||~xk+1 − ~xk || is smaller than a specified tolerance.
COMO 201: Numerical Methods and Structural Modelling, 2003
3.5.4
23
Example: Newton’s Method for Two Equations
Consider the example described previously:
1
F2 (x, y) = 2x2 + y 2 − 3,
4
where we wish to find x and y such that F1 (x, y) = 0 and F2 (x, y) = 0. Thus
F1 (x, y) = x2 + y 2 − 3,
F~ ≡
"
F1
F2
#

∂F1
∂x
∂F1
∂y
∂F2
∂x
∂F2
∂y
=
"
x2 + y 2 − 3
2x2 + 41 y 2 − 3
#


The Jacobian matrix is then
J≡




=

2x 2y
1
y
2
4x


The main step of the algorithm , J · δ~xk = −F~ (~xk ) can be solved using the inverse
of the matrix J when it exists:
δ~xk =
and so
⇒
"
⇒
"
⇒
"
δx
δy
#
δx
δy
#
δx
δy
#



−J−1 · F~ (~xk )

= −


= 


= 

xk+1
yk+1



2x 2y
1
y
2
4x
1
14x
2
− 7x
2
7y
4
− 7y
9
14x
6
7y
−
x
2
−
y
2

=

xk
yk
−1 







x2 + y 2 − 3
2x2 + 14 y 2 − 3
x2 + y 2 − 3
2x2 + 14 y 2 − 3











 
+
9
14xk
6
7yk
−
xk
2
−
yk
2
Iterating in Mathematica, starting at (x0 , y0 ) = (−1, 2):



In[19]:= newtstep@8xk_, yk_<D := 8xk + 9  H14 xkL - xk  2, yk + 6  H7 ykL - yk  2<;
In[23]:= NestList@newtstep, 8-1.0, 2.0<, 4D  MatrixForm
-1.
i
j
j
j
-1.14286
j
j
j
j
j
-1.13393
j
j
j
j
j
j
j -1.13389
k -1.13389
Out[23]//MatrixForm=
2.
y
z
z
1.42857 z
z
z
z
z
1.31429 z
z
z
z
z
1.30932 z
z
z
1.30931 {
COMO 201: Numerical Methods and Structural Modelling, 2003
24
Newton’s Method in Mathematica
Mathematica’s FindRoot command uses Newton’s method.
In[1]:=
FindRoot[{2 x^2 + (y/2)^2 - 3 == 0, x^2 + y^2 - 3 == 0},{x, -1}, {y, 2}]
Out[1]= {x -> -1.13389, y -> 1.30931}
3
2
1
0
-1
-2
-2
-1
0
1
2
3
Newton’s Method in MATLAB
MATLAB’s fzero command uses a competing scheme restricted to one dimension.
Multidimensional methods require the Optimization toolbox or custom routines.
Source File: newtdemo.m
function.newtdemo
%.solving.f(x,y).=.x^ 2.+.y^ 2.-.3..=.0,.
%.........g(x,y).=.2.x^ 2.+.(y/2)^ 2..-.3..=.0
%.starting.at.x=-1,.y=2.
vals.=.[-1.;.2];
for.(loopy=1:10)
.....vals.=.newtstep(vals);
end
vals.....%.display.the.final.approximation.
%.ONE.ITERATION.OF.NEWTON’S.METHOD.
function.newvals.=.newtstep(oldvals)
x.=.oldvals(1);.y.=.oldvals(2);
F.=.[.x^2.+.y^2.-.3.;.2*x^2.+.(y/2)^2..-.3.];
J.=.[.2*x.,.2*y.;..4*x..,.y/2...];
deltax.=.-J.\.F;...%.MATLAB’s.left.matrix.divide.
newvals.=.oldvals.+.deltax;
>> format long
>> newtdemo
vals =
-1.13389341902768
1.30930734141595
Chapter 4
Interpolation and Extrapolation
• Used when we know values of a function f (x) at a set of discrete points
x1 , x2 , . . . , xN but we don’t have an analytic expression for f (x) that lets us
calculate its value at an arbitrary point.
• If we want to find f (x) at one or more values of x outside the interval [x1 , xN ]
then the problem is called extrapolation. If x ∈ [x1 , xN ] it’s interpolation.
• The general idea is to fit some smooth function to the data points. Typical
functions used include:
1. Polynomials (by far the most common method).
2. Rational Functions (quotients of polynomials).
3. Trigonometric functions (sines and cosines, giving rise to trigonometric
interpolation and related Fourier methods).
4.1
Lagrange (Polynomial) Interpolation
Suppose that we have N points, (x1 , y1 ), (x2 , y2 ), . . . (xN , yN ) and wish to find the
unique polynomial of degree N − 1 that passes through them. We write this polynomial in terms of the y1 , . . . , yN values as:
P (x) =
N
X
gk (x) yk .
k=1
Since we want p(x) to pass through our N points, we need
P (xi ) = yi =
N
X
gk (xi ) yk .
k=1
This clearly holds if we choose our gk functions so that
gk (xi ) =
(
1
0
25
if i = k
if i =
6 k
COMO 201: Numerical Methods and Structural Modelling, 2003
26
because
P (xi ) =
N
X
gk (xi ) yk = g1 (xi )y1 + g2 (xi )y2 + . . . + gi (xi )yi + . . . + gN (xi )yN
k=1
= 0y1 + 0y2 + . . . + 1yi + . . . + 0yN
= yi
But what should we choose gk (x) to be? It has N − 1 roots, but must have degree
≤ N − 1 because P (x) has degree N − 1. Thus gk (x) has degree N − 1, with its
N − 1 roots at the xi 0 s except for xk . Thus
gk (x) = αk (x − x1 )(x − x2 ) · · · (x − xk−1 )(x − xk+1 ) · · · (x − xN ),
or, more compactly using the product symbol
N
Y
gk (x) = αk
i = 1,
i 6= k
Q
,
(x − xi ).
How do we choose αk ? We need to use the fact that gk (xk ) = 1.
N
Y
1 = gk (xk ) = αk
i = 1,
i 6= k
Rearranging gives:
1
αk =
N
Y
i = 1,
i 6= k
and so
gk (x) =
(xk − xi )
N
Y
i = 1,
i 6= k
Thus
P (x) =

 N
N 
X
 Y


k=1  i = 1,
i 6= k
or, expanded,
P (x) =
(xk − xi )
x − xi
.
xk − x i



x − xi 
 yk  ,

xk − x i  



(x − x1 )(x − x3 ) · · · (x − xN )
(x − x2 )(x − x3 ) · · · (x − xN )
y1 +
y2 +
(x1 − x2 )(x1 − x3 ) · · · (x1 − xN )
(x2 − x1 )(x2 − x3 ) · · · (x2 − xN )
···+
(x − x1 )(x − x2 ) · · · (x − xN −1 )
yN
(xN − x1 )(xN − x2 ) · · · (xN − xN −1 )
COMO 201: Numerical Methods and Structural Modelling, 2003
4.1.1
27
Examples of Lagrange Interpolation
1. Clearly y = x2 goes through the points (−1, 1), (0, 0), and (1, 1). Checking
that Lagrangian interpolation gives x2 :
g1 (x) =
n
Y
x − xi
1
1
(x − 0)(x − 1)
= x2 − x
=
(−1 − 0)(−1 − 1)
2
2
i=2 x1 − xi
(x − (−1))(x − 1)
= 1 − x2
(0 − (−1))(0 − 1)
(x − (−1))(x − 0)
1
1
g3 (x) =
= x2 + x
(1 − (−1))(1 − 0)
2
2
1 2
x − x + x 2 + x = x2
⇒ p(x) = g1 (x) · 1 + g2 (x) · 0 + g3 (x) · 1 =
2
• In Mathematica with a custom Lagrangian interpolation function:
g2 (x) =
In[113]:= Clear@pD;
p@xl_, yl_D := ModuleA8n = Length@xlD<, ExpandA
n-1
y
i
x - Drop@xl, 8k<DPiT
z
j
z
j
ylPkT
€€€€€€€€€€€€€€€€
€€€€€€€€€€€€€€€€€€ z
ä €€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€
ãj
z
j
j
xlPkT - Drop@xl, 8k<DPiT z
i=1
{
k=1 k
n
EE;
In[115]:= p@8-1, 0, 1<, 81, 0, 1<D
Out[115]= x2
• In MATLAB with polyfit:
>> x=[-1 0 1]; y=[1 0 1]; polyfit(x,y,2)
ans =
1.0000
0.0000
0
2. Find the cubic through the points (1, 20), (2, 40), (4, 140), and (6, 200).
• In Mathematica with the above function:
In[94]:= p@8 8 1, 20<, 82, 40<, 84, 140<, 86, 200<<D
Out[94]= 44 - 52 x + 31 x2 - 3 x3
• In Mathematica least squares curve fitting:
Fit@881, 20<, 82, 40<, 84, 140<, 86, 200<<,
81, x, x2 , x3 <, xD
Out[97]= 44. - 52. x + 31. x2 - 3. x3
• In MATLAB with polyfit:
>> x=[1 2 4 6]; y=[20 40 140 200]; polyfit(x,y,3)
ans =
-3.0000
31.0000 -52.0000
44.0000
COMO 201: Numerical Methods and Structural Modelling, 2003
4.2
4.2.1
28
Least Squares Interpolation
Fitting a Straight Line to Data (“Linear Regression”)
Given points (x1 , y1 ), (x2 , y2 ), . . ., (xN , yN ) we wish to find coefficients a and b so
that the straight line y(x) = a + bx best fits the data. We do so by minimising the
sum of squares of the errors:
N
X
i=1
(y(xi ) − yi )2 .
(xi , yi )
yi
y(xi )
xi
The minimum of this sum occurs when the partial derivatives (with respect to a
and b) are zero. Let
D=
N
X
i=1
(a + bxi − yi )2 .
Then the best straight line approximation to the data satisfies
N
X
∂D
= 0 ⇒ 2 (a + bxi − yi ) = 0
∂a
i=1
and
N
X
∂D
=0 ⇒ 2
xi (a + bxi − yi ) = 0.
∂b
i=1
Defining
x̄ ≡
N
1X
xi ,
n i=1
ȳ ≡
and
N
1X
yi
n i=1
the solution to the two above equations can be shown, with some effort, to be
a = ȳ − bx̄,
b=
(xi − x̄)(yi − ȳ)
P
(xi − x̄)2
P
COMO 201: Numerical Methods and Structural Modelling, 2003
4.2.2
29
Least Squares from an Algebraic Perspective
Suppose we have a set of overly constrained (inconsistent) equations written as a
matrix equation:
A~v = ~b
where ~v and ~b are column vectors. The optimal solution, in the least squares sense,
will minimise the sum of the squares of the errors, i.e. minimise
X
i
where ~ = A~v − ~b.
2i = | ~ |2 ,
This is equivalent to finding the minimum of
| ~ | = |A~v − ~b|.
This minimising value of ~v , ~v ∗ say, occurs when the corresponding ~∗ is perpendicular
to the column space 1 of A, as can be seen geometrically:
Column space of A
~∗ = A~x∗ − ~b
A~x∗
~b
∗
Thus ~ is orthogonal to every column A(:, j) =
∀j ∈ {1, 2, . . . , n} ,






[A1,j A2,j . . .
A1,j
A2,j
..
.
Am,j






i.e.,



Am,j ] 


∗1
∗2
..
.
∗m






= 0,
and so
AT ~∗ = ~0.
1
The column space of A is the set of all vectors which can be formed by linear combinations
of the columns of A.
30
COMO 201: Numerical Methods and Structural Modelling, 2003
Thus the optimal value ~v = ~v ∗ satisfies
AT (A~v ∗ − b) = 0
⇒ AT A~v ∗ = AT ~b
⇒ ~v ∗ = (AT A)−1 AT ~b
It can be shown that the inverse (AT A)−1 exists if all columns of A are independent.
Application
Suppose that a linear relation is anticipated between two variables x and y of
which we have a number of measurements (x1 , y1 ) through to (xN , yN ). Let the
relationship be expressed in the form y = a + bx. The “unsolvable” system that we
want a closest solution to is:






y1
1 x1
"
#

1 x2 
 y2
 a
=
.. .. 
 ..

 .
. .  b
yN
1 xN








Then the optimal solution [a∗ b∗ ]T is given by:

"




1 1 ... 1
x1 x2 . . . x N

#




y
1 x1
"
# 1 
 " ∗ #
y2 
1 x2   a
1 1 ... 1 
 . 

=
.. .. 

  b∗
x1 x2 . . . xN  .. 

. . 
yN
1 xN



Or, more compactly,



N
P
xi
P
P
xi
x2i



a∗
b∗




P
=

P
yi
xi y i



We could invert to find the general solution, or alternatively solve the two simultaneous equations for a∗ and b∗ . The latter gives:
N a∗ + N x̄b∗ = N ȳ
N x̄a∗ +
and
X
x2i b∗ =
X
xi y i .
The first equation readily yields a∗ = ȳ − b∗ x̄, as we had before. The second then
gives
⇒
and so
N x̄(ȳ − b∗ x̄) +
(
X
x2i
2
X
∗
x2i b∗ =
− N x̄ )b =
X
P
xi y i
xi yi − N x̄ȳ
xi yi − N x̄ȳ
b = P 2
xi − N x̄2
∗
X
31
COMO 201: Numerical Methods and Structural Modelling, 2003
The numerator can be re-expressed as
X
xi yi − N x̄ȳ =
=
=
=
X
xi y i +
X
(xi − x̄) (yi − ȳ)
X
Similarly the denominator satisfies
familiar expression for b∗ ,
∗
b =
4.2.3
xi yi + N x̄ȳ − N x̄ȳ − N x̄ȳ
X
x̄ȳ −
X
x̄yi −
(xi yi + x̄ȳ − x̄yi − ȳxi )
P
P
X
x2i − N x̄2 =
X
ȳxi
(xi − x̄)2 which gives us the
P
(xi − x̄)(yi − ȳ)
.
P
(xi − x̄)2
General Curve Fitting with Least Squares
Least squares analysis can be used to fit general functions to data. The solution
process for the optimal values of coefficients and parameters becomes more involved
when fitting more complex functions, but computer routines to solve these problems
are widely available (particularly for linear combinations of arbitrary functions).
• In Mathematica:
In[17]:= ydata = 80.1, 0.5, 0.8, 1.2, 1.9, 2.3, 3.5, 5.5, 7.9, 10.0<;
data = Table@8i, ydata@@iDD<, 8i, 10<D
Out[18]= 881, 0.1<, 82, 0.5<, 83, 0.8<, 84, 1.2<,
85, 1.9<, 86, 2.3<, 87, 3.5<, 88, 5.5<, 89, 7.9<, 810, 10.<<
In[19]:= Fit@data, 8E ^ x, Sin@xD, x ^ 2<, xD
Out[19]= 0.000120146 ãx + 0.0766265 x2 + 0.170804 Sin@xD
• MATLAB:
Source File: ls.m
function.ls.......%.demonstrates.least-squares.curve.fitting.
xdata.=.[..1...2...3...4...5...6...7...8...9...10..];
ydata.=.[.0.1.0.5.0.8.1.2.1.9.2.3.3.5.5.5.7.9.10.0.];
coeff0.=.[1.1.1];.%.start.values.for.coefficient.determination.
lsqcurvefit(@f,.[1.1.1],.xdata,.ydata.)
function.y=f(coeff,.x).....%.The.function.to.fit.
y.=.coeff(1).*.exp(x).+.coeff(2).*.sin(x).+.coeff(3).*.x.^2;
>> ls
ans =
0.00012016794938
0.17086338759621
0.07662262204343
Chapter 5
Numerical Differentiation
5.1
Finite Differencing
To approximate the derivative f 0 (x) of a given function f (x) we use the definition
f 0 (x) ≡ lim
h→0
f (x + h) − f (x)
,
h
which suggests the approximation, for small h > 0,
f 0 (x) ≈
f (x + h) − f (x)
.
h
(5.1)
We can get an idea of the error involved in this approximation by considering the
Taylor expansion of f (x + h) about x:
f (x + h) = f (x) + hf 0 (x) +
h2 00
f (ζ),
2
x < ζ < x + h.
Then we have
h
f (x + h) − f (x)
= f 0 (x) + f 00 (ζ).
h
2
Therefore the error in the approximation of Eq. 5.1 is proportional to h. If we halve
h we halve the error in the approximated derivative. When this approximation is
used on computer we need h large enough to avoid underflow in the evaluations,
but small enough to produce accurate derivative approximations.
There are many ways to derive approximations to f 0 (x). One way is to differentiate Lagrange interpolating polynomials. Doing so for the quadratic f (x) = p2 (x)
that passes through the points at x = x1 , x = x0 = x1 − h and x = x2 = x1 + h
gives the central difference approximation:
f 0 (x) ≈
f (x + h) − f (x − h)
.
2h
The central difference approximation approximates f 0 (x) with error proportional to
h2 , so is much better than Eq. 5.1.
32
COMO 201: Numerical Methods and Structural Modelling, 2003
5.1.1
33
Second Derivative
The second derivative may be approximated with three points by
f 00 (x) ≈
f (x − h) − 2f (x) + f (x + h)
h2
Further equations can be derived for higher order derivatives.
5.2
Ordinary Differential Equations
We can turn ordinary differential equations (ODEs) and partial differential equations (PDEs) into finite difference equations (FDEs) using finite difference approximations to the derivatives. They may then be numerically solved on computer.
5.2.1
High Order ODEs
If we have a high order ODE we can reduce it to a system of first order ODEs. For
example the second-order equation
dy
d2 y
+ q(x)
= r(x)
2
dx
dx
can be readily reduced to
dy
= z(x),
dx
dz
= r(x) − q(x)z(x).
dx
where z is a new variable. Thus the generic problem in ordinary differential equations is reduced to the study of a set of N coupled first-order differential equations
for the functions yi , i = 1, 2, . . . , N having the general form
dyi
= fi (x, y1 , y2 , . . . , yN ).
dx
5.2.2
Example: An Initial Value Problem (IVP)
Gomez, the Mexican sea turtle, wants to swim to a small island 50km from the
coast, where he thinks there might be a bright shiny thing. His travel is governed
by the equation
f 00 (t) + 2f 0 (t) = 3et
where t is the time in days since his departure, and f (t) is the distance in km that
he is from the coast. If he starts swimming with a speed of 10km/day, find the time
at which he reaches the island and also his final speed.
COMO 201: Numerical Methods and Structural Modelling, 2003
34
Solution.
Rather than use the F.D. expression for the second derivative, we reduce the ODE
to a system of first order ODEs:
g(t) ≡ f 0 (t),
g 0 (t) = 3et − 2g(t).
Next we substitute the finite difference equations for f 0 (t) and g 0 (t) and rearrange.
g(t) =
f (t + h) − f (t)
,
h
⇒ f (t + h) = f (t) + hg(t),
g(t + h) − g(t)
= 3et − 2g(t)
h
g(t + h) = g(t) + h(3et − 2g(t))
Thus we have the iterative equations:
fk+1 = fk + hgk
gk+1 = gk + h(3etk − 2gk )
tk+1 = tk + h
Source File: turtle.m
f=0;.g=10;.h=0.001;.t=0;
while.(f<50)
.....fo=f;.go=g;.to=t;.
.....
.....f=.fo.+.h.*.go;
.....g=.go.+.h.*.(3*exp(to).-.2*go.);
.....t=.to+h;
end
disp([.’Turtle.makes.it.at.t=’.num2str(t).’.days,.’...
.....’final.velocity=.’.num2str(g).’.km/day’]);
>> turtle
Turtle makes it at t=3.84 days, final velocity= 46.5684 km/day
That’s one very fast turtle!
Concluding Remarks
As you see, it’s fairly easy to come up with a finite difference scheme and iterative
formulas to solve ODEs in IVPs. If higher accuracy is desired then a smaller step
size can be used, or a central difference approximation, or even an adaptive step
size. The same ideas work in more than one dimension (i.e. with more than one
variable), where we calculate values of the functions on a regularly spaced grid or
mesh of points instead of along a line. They also work well for partial derivatives
and for solving PDEs.
Chapter 6
Numerical Integration
Numerical integration is valuable in many situations, for example when integrating
a large number of complicated functions, integrating an unknown function based
on sampled data, and integrating functions which have no closed form expression
for the integral. Examples of the latter include
Z
2
1
ex
dx,
x
and
Z
1
0
√
1 + x5 dx.
Numerical integration is also called numerical quadrature. The basic idea is to
approximate a function (known or unknown) by something that’s easy to integrate.
The integrand f is evaluated or known at n nodes, labeled x1 , x2 , . . ., xn , that are
in the interval defined by the limits of the integral. When the nodes are equally
spaced the resulting integration formulas are known as Newton-Cotes rules. More
advanced methods, such as Gaussian integration, deal with unevenly spaced roots
(in the case of Gaussian quadrature the nodes are chosen as the roots of orthogonal
polynomials in order to minimise error in the method).
6.1
The Trapezoid Rule
The trapezoid rule uses a piecewise linear interpolation to approximate f (x). When
applied to a sequence of points, with uniform spacing h, the trapezoid rule gives:
Z
b
a
1
1
f (x)dx ≈ h f (x1 ) + f (x2 ) + . . . + f (xn−1 ) + f (xn ) .
2
2
This expression has error proportional to h2 .
6.2
Simpson’s Rule
Simpson’s rule approximates f (x) with a series of quadratic interpolating polynomials (a quadratic is fitted to every three points). The number of nodes, n, must
be odd.
35
36
COMO 201: Numerical Methods and Structural Modelling, 2003
For three nodes only one quadratic is fitted. Use of the Lagrange interpolation
method yields
P2 (x) =
1
1
1
(x − x2 )(x − x3 )f1 + 2 (x − x1 )(x − x3 )f2 + 2 (x − x1 )(x − x2 )f3
2
h
h
h
where ∀i ∈ Z + , fi ≡ f (xi ). Integration over [a, b] gives
Z
b
a
f (x)dx ≈ h
1
4
1
f1 + f2 + f3 .
3
3
3
Combining these integrals for more nodes gives the approximation
Z
b
a


n−2
n−1
X
X
h
fi 
fi + 2
f (x)dx ≈ f1 + fn + 4
3
i=3,5,...
i=2,4,...
This expression has error that is at worst proportional to h4 .
In MATLAB the composite formula can be evaluated for a vector f of function
values at n nodes as follows:
h/3 * ( f(1) + f(n) + 4*sum( f(2:2:n-1) ) + 2*sum( f(3:2:n-2) ) )
The inbuilt MATLAB function quad uses an adaptive version of Simpson’s rule to
calculate approximate integrals.
6.3
Gauss-Legendre Integration
Gaussian integration assumes that the function f (x) can be sampled at x values of
our choosing, i.e. that some method of calculating f (x) values for general choices
of x is known. The task then is to approximate the integral
Z
b
a
f (x) dx
as well as possible using a given number of samples (n say). In Gaussian integration
we will carefully choose the position of the samples in such a way that polynomials
are very accurately integrated. Other functions will work to varying degrees - the
closer they are to polynomials the more accurate the results from the Gaussian
integration scheme.
We approximate with the simple expression
Z
b
a
f (x) dx ≈
n
X
wi f (xi ),
i=1
where the “weights” wi and “nodes” (or “abscissas”) xi still need to be chosen.
The derivation of these optimal weights and nodes is beyond the scope of this
course. The interested reader may wish to refer to Numerical Recipes in C. We will
only demonstrate the use of computer packages to determine the xi and wi values.
COMO 201: Numerical Methods and Structural Modelling, 2003
6.3.1
37
Computational Determination of Weights and Nodes
• In Mathematica, considering the integral
Z
1
−1
f (x)dx
to be sampled with five values (x1 ,x2 ,. . . ,x5 , so that n=5):
In[1]:= << NumericalMath‘GaussianQuadrature‘
Using n=5, a=−1,b=1.
In[6]:= Join@88"xi", "wi"<<, GaussianQuadratureWeights@5, -1, 1D D  MatrixForm
xi
i
j
j
j
j
-0.90618
j
j
j
j
-0.538469
j
j
j
j
j
0
j
j
j
j
j
j
j 0.538469
k 0.90618
Out[6]//MatrixForm=
wi
y
z
z
0.236927 z
z
z
z
z
0.478629 z
z
z
z
z
0.568889 z
z
z
z
z
0.478629 z
z
z
0.236927 {
Thus the formula to be used is:
R1
−1
f (x)dx ≈ 0.236927 f (−0.90618) + 0.478629 f (−0.538469) + 0.568889 f (0)
+ 0.478629f (0.538469)
+ 0.236927f (0.90618)
Perfect integration using n = 5 for a 4th order polynomial:
In[143]:= Clear@f, pts, iD;
f@x_D := x ^ 4 - 2 x ^ 3 + 7 x ^ 2 - 12 x + 93;
pts = GaussianQuadratureWeights@5, -1, 1D;
Rationalize@Sum@ ptsPiTP2T * f@ ptsPiTP1T D , 8i, 1, 5<DD
Out[146]=
2866
€€€€€€€€€€€€€
15
In[147]:= Integrate@f@xD, 8x, -1, 1<D
Out[147]=
2866
€€€€€€€€€€€€€
15
Imperfect integration using n = 5 with f (x) = e(x
2 +x3 )
:
f@x_D := E ^ Hx ^ 2 + x ^ 3L ;
In[278]:= Clear@f, pts, iD;
pts = GaussianQuadratureWeights@5, -1, 1D;
Sum@ ptsPiTP2T * f@ ptsPiTP1T D , 8i, 1, 5<D
Out[281]= 3.25312
In[282]:= Integrate@f@xD, 8x, -1, 1<D  N
Out[282]= 3.25661
COMO 201: Numerical Methods and Structural Modelling, 2003
38
• MATLAB does not have inbuilt functions for calculating the weights and
nodes for Gaussian integration. The following m-file does this from scratch:
Source File: gnodewt.m
function.[x,w].=.gnodewt(n,a,b)
%.Computes.nodes.and.weights.for.an.n-point.Gauss.quadrature
%.over.the.interval.[a,b].
[mu,v]=glnodewt(n);
w.=.0.5*(b-a)*v;
x.=.a.+.0.5*(mu+1)*(b-a);
function.[x,w].=.glnodewt(n)
%.Computes.nodes.and.weights.for.an.n-point
%.Gauss-Legendre.quadrature.over.the.interval.[-1,1].
beta.=.(1:n-1)./sqrt(4*(1:n-1).^2-1);
J.=.diag(beta,-1).+.diag(beta,1);
[V,D].=.eig(J);.[x,ix].=.sort(diag(D));.w.=.2*V(1,ix)’.^2;
The subroutine glnodewt(n) uses an advanced method by Golub, Welsch and
Gautschi to calculate weights and nodes for the case:
Z
1
−1
f (µ)dµ ≈
n
X
vi f (µi ).
i=1
We then use a change of variables in gnodewt(n,a,b) to generalize this result.
Let
x−a
µ(x) = 2 ·
− 1.
b−a
Then µ(a) = −1 and µ(b) = 1. Also,
2
dµ
=
,
dx
b−a
and
1
x(µ) = a + (µ + 1)(b − a)
2
Then
Z
b
a
f (x)dx =
Z
1
−1
f (x(µ))
dx
dµ
dµ
b−aZ 1
=
f (x(µ))dµ
2
−1
n
b−aX
≈
vi f (x(µi ))
2 i=1
n X
1
1
=
(b − a)vi f a + (µi + 1)(b − a))
2
2
i=1
COMO 201: Numerical Methods and Structural Modelling, 2003
And thus:
Z
b
a
f (x)dx ≈
where
n
X
39
wi f (xi )
i=1
1
wi = (b − a)vi
2
and
1
xi = a + (µi + 1)(b − a).
2
MATLAB uses the vector equivalents:
1
w
~ = (b − a)~v
2
and
1
~x = ~a + (~µ + ~1)(b − a).
2
where ~a and ~1 are vectors of length n with all components set to a and 1
respectively.
>> [x,w]=gnodewt(5,-1,1);
>> [x w]
ans =
-0.9062
-0.5385
0
0.5385
0.9062
0.2369
0.4786
0.5689
0.4786
0.2369
Source File: gnodedemo.m
function.gnodedemo
[x,w].=.gnodewt(5,-1,1);
I.=.sum(.w.*f(x).);
format.long;.disp(I);
function.y=f(x)
.....y.=.exp(x.^2+x.^3);
>> gnodedemo
3.25311794452507
Chapter 7
Fundamentals of Statics and
Equilibrium
This chapter focuses on background material for the Finite Element Method (FEM)
rather than on Numerical Methods (such as the FEM) themselves.
7.1
Forces and Particles
Force is the action of one body on another. It may be exerted by actual contact
or at a distance. A force is characterized by a vector, having magnitude, sense and
direction. A system of forces can be replaced by a single resultant force.
A particle is an idealization (mathematical model) of an object whereby all of
the object’s mass is focused at a point, usually at its centre of gravity.
7.2
Newton’s Laws
1. A particle remains at rest or continues to move uniformly in the absence of
external forces.
2. The acceleration of a particle is proportional to the resultant force acting on
it and in the direction of this force (F~ = m~a).
3. Forces arising between interacting objects always occur in pairs, equal in magnitude and opposite in direction.
7.3
Static Equilibrium
Static equilibrium is the state when the resultant force on an object is zero, so that
the forces acting on the object are in perfect balance. Thus if the object is initially
stationary it remains so (by either Newton’s first or second laws).
40
COMO 201: Numerical Methods and Structural Modelling, 2003
41
The resultant force here is effectively the vector sum of all of the external forces,
because the internal forces are already in perfect balance (by Newton’s third law,
assuming that the object does not deform).
7.4
Solution of Mechanics Problems
• Approximations are always involved. For example neglection of small angles,
distances or forces.
• Symbolic solutions, when they can be obtained, have advantages over numerical solutions. For example
– dimensional checks are possible with symbolic solutions,
– a symbolic solution highlights the connection between the mathematical
model and the physical system,
– symbolic solutions allow variation of parameters without re-solving the
problem.
7.4.1
SI Units in Mechanics
Standard base and derived SI units are used in mechanics, such as the metre (m)
for length, the newton (N = kg · m · s−2 ) for force and the watt (W = N · m · s−1 )
for power.
7.5
More on Forces
In addition to being a vector, a force acting on a body has a definite point of
application. We speak of F~ as having a line of application, which is a line
through the point of application in the direction of F~ .
F~
-F~
Same vector as F~ ,
but different force (line
of action)!
COMO 201: Numerical Methods and Structural Modelling, 2003
42
The magnitude of F~ is measured in newtons. By Newton’s second law (F~ = m ~a),
a force of 1 newton will give a mass of 1 kg an acceleration of 1 m · s−1 .
~ act on a particle A, then they may be replaced by a single
If two forces P~ and Q
~ where R
~ = P~ + Q.
~
force R
~
Q
A
~
R
⇐⇒
P~
A
~ is called the resultant of P~ and Q.
~
The vector R
7.5.1
Coordinates
F~ = Fx ~i + Fy ~j
Fx = force in the x-direction = |F~ | cos θ = F cos θ
Fy = force in the y-direction = |F~ | sin θ = F sin θ
y-axis, normally vertical
F = |F~ |
Fy
F~
~j
Fx
x-axis, normally horizontal
~i
We say that F~ has been resolved into its x and y components.
43
COMO 201: Numerical Methods and Structural Modelling, 2003
7.5.2
Example I
Find the resultant of the forces shown in the following picture.
4000N
~j
~
Q
~
R
60◦
α
35◦
~i
P~
2500N
~ = P~ + Q,
~ where
R
P~ = 2500 cos 35◦ ~i − 2500 sin 35◦ ~j = 2048 ~i − 1433.9 ~j
~ = 4000 cos 60◦ ~i + 4000 sin 60◦ ~j = 2000 ~i + 3464 ~j
Q
~ = 4048 ~i + 2030 ~j
⇒R
⇒ R = 4530 N (3sf ),
7.5.3
α = tan−1
2030
= 26.6◦ (3sf )
4048
Example II
A man pulls with a force of 300 N on a rope at B. What are the horizontal and
vertical components of the force exerted by the rope at the point A?
A
8m
α
6m
B
~ = 300 N, so R
~ = 300 · 4 ~i − 300 ·
R = |R|
5
3
5
~j
~i
~j. Thus Rx = 240 N and Ry = −180 N.
44
COMO 201: Numerical Methods and Structural Modelling, 2003
7.6
Equilibrium of a Particle
~ of all of the forces acting on a particle is zero, the particle is
When the resultant R
said to be in equilibrium. So for equilibrium we have:
F~1
F~3
~ =
R
F~2
n
X
F~i = ~0 or, using shorthand,
i=1
X
F~ = ~0
F~4
Resolving each force into its components gives equivalent conditions for equilibrium:
X
7.6.1
Fx = 0 and
X
Fy = 0.
Example
Show that the four given forces are in equilibrium when α = β = 30◦ .
y
F4 = 400 N
x
α
F1 = 300 N
F3 = 200 N
β
√
F2 = 100 3 N
1
1
− 400 · = 0
2
√
√ 2
√
3
3
+ 400 ·
=0
= −100 3 − 200 ·
2
2
X
Fx = 300 − 200 ·
X
Fy
Because the sum of the force vectors is required to be zero, the polygon of forces
must be closed:
F~1
F~2
F~4
F~3
COMO 201: Numerical Methods and Structural Modelling, 2003
45
From Newton’s first (or second) law we see that a particle in equilibrium is either
at rest or is moving in a straight line with constant speed.
7.7
Free Body Diagrams
The above analysis, strictly speaking, applies only to a system of forces acting on
a particle. However, a large number of problems involving actual structures may
essentially be reduced to problems concerning the equilibrium of a particle. Then a
separate diagram showing this representative particle together with all of the forces
acting on it may be drawn. Such a diagram is called a free-body diagram.
7.7.1
Example
Determine the magnitude and direction of the smallest force F~ which will maintain
the package shown in equilibrium. Note that the force exerted by the rollers on the
package must be perpendicular to the incline.
Package
weight
is30
kg
α
F~
15◦
We choose the package as a free body, assuming that it may be treated as a particle
(i.e. that it is a rigid body), and draw the corresponding free body diagram.
~,
W
~ | = 30 · 9.81 = 294 N
|W
α
P~
F~
~ is
P~ is the reaction force at the rollers and W
the weight force of the package. The value of
α which minimizes the required force F~ can
easily be determined if we consider the closed
force polygon for the system in equilibrium.
15◦
Clearly |F~ | is minimum when F~ ⊥ P~ . Thus
~ and P~
α = 15◦ . Because the angle between W
is 15◦ in the right angled triangle (force polygon), we have |F~ | = 294 sin 15◦ = 76.1 N.
~
W
F~
P~
α
COMO 201: Numerical Methods and Structural Modelling, 2003
7.8
46
Rigid Bodies: Force Equivalents
So far we have neglected the size of the bodies considered and treated them as
particles. When the size of a body is important analysis is still straightforward if it
is a rigid body, that is one that does not deform. Actual structures or machines are
subject to small deformations which often do not appreciably affect the conditions
of equilibrium or motion of the structure under consideration. To begin with we
shall only be concerned with the external forces applied to the rigid body. The
internal forces hold the parts (or particles) of the rigid body together, and cancel
out by Newton’s third law.
Example
A beam resting on a single pivot, and a free body diagram of the beam showing the
external forces are shown below.
O
G O
~
W
~
d S
~
We assume that the weight of the beam may be represented by a single force W
acting at a point G, called the centre of gravity of the beam. In the case of a uniform
beam G is located at the midpoint. The downward motion of the beam (due to its
~ Clearly the beam will begin to rotate
weight) is opposed by the supporting force S.
unless supported at G. W · d measures the tendency of the beam to rotate about
O in an anticlockwise direction.
7.8.1
Moment
The moment of a force F~ about a point O is defined by
~ O ≡ ~r × F~
M
where ~r is the position vector of the point of application A of F~ relative to O.
F~
~r
O
α
α
A
d
~ O ≡ ~r × F~ = ~n |F~ | |~r| sin α = ~n F d, where d is the
From the diagram we see that M
distance from O to the line of action of F~ , and ~n is a unit vector pointing out of
~ O | = F d, where F = |F~ |. The moment of a force in SI units
the page. Note that |M
is expressed in newton-metres (N · m).
COMO 201: Numerical Methods and Structural Modelling, 2003
47
Example
A force of 800 N acts on a bracket as shown. Determine the moment of the force
about B.
800 N
A
60◦
160 mm
B
~j
~i
200 mm
~ B = ~rA/B × F~ , where ~rA/B is the position of A relative
The moment about B is M
to B, i.e. ~rA/B = −0.2~i + 0.16 ~j (in metres!). Now
F~ = 800 cos 60◦ ~i + 800 sin 60◦ ~j = 400~i + 692.8 ~j N
thus
~ B = (−0.2~i + 0.16 ~j) × (400~i + 692.8 ~j)
M
= −138.56~k − 64~k
= −(203 N · m)~k (3sf).
This is often written as
7.8.2
~ B = 203 N · m
M
Varigon’s Theorem
~ which is the resultant of several concurrent forces (i.e. forces
Consider a force R
~ about any given
applied at the same point) F~1 , F~2 , . . ., F~n . Then the moment of R
point O is equal to the sum of the moments of F~1 through F~n about O.
Proof
Varigon’s theorem is an immediate consequence of the distributive law for the vector
product1 .
~ O = ~r × R
~
M
= ~r × (F~1 + F~2 + . . . + F~n )
= ~r × F~1 + ~r × F~2 + . . . + ~r × F~n
=
n
X
~i
M
O
i=1
~ × (B
~ + C)
~ =
The proof of the distributive law involves expanding both sides of the equation A
~×B
~ +A
~ ×C
~ in coordinate form.
A
1
48
COMO 201: Numerical Methods and Structural Modelling, 2003
7.8.3
Equivalent Forces Theorem
We say that two systems of forces F~1 , F~2 , . . ., F~m and F~1∗ , F~2∗ , . . ., F~n∗ acting on a
rigid body are equivalent if and only if both of the following conditions hold:
1. The resultants of the force systems are equal, i.e.
m
X
F~i =
i=1
n
X
F~j∗
j=1
2. There is a point O such that the sums of the moments of the two systems
about O are equal, i.e.
m
X
i=1
r~i × F~i =
n
X
j=1
r~j∗ × F~j∗
The next theorem tells us that O can be any point.
Theorem
If two force systems are equivalent then the two systems must have the same moment
about every point.
Proof
We’ll prove this in the case of two equivalent force systems where all forces are
P
P
applied at a point A in the rigid body. Then F~ = F~ ∗ and there is a point O
P ~
P ~∗
such that M
MO . We wish to show that for any given point Q we have
O =
P
P ∗
~
~
MQ = MQ .
F~2
F~1∗
A
~
OA
~
QA
O
X
F~1
~
OQ
~O =
M
=
X
X
Q
~ × F~i
OA
~ + QA)
~ × F~i
(OQ
~ ×
~ × F~i
= OQ
F~i +
QA
X
~ ×R
~+
~ × F~i
= OQ
QA
X
X
~ = P F~i = P F~ ∗ since the two systems are equivalent. The same process
where R
j
for the second system gives us
X
~ O∗ = OQ
~ ×R
~+
M
X
~ × F~j∗
QA
49
COMO 201: Numerical Methods and Structural Modelling, 2003
Thus
X
~ × F~i =
QA
X
~ × F~j∗
QA
proving the theorem for the case of concurrent forces. Proof for the general case,
where each force may be applied at a different location, is essentially the same as
this, but requires more notation to account for the various points of application.
7.8.4
Beam Corollary
The weight of a uniform beam of mass M may be represented by a single force
~ = −M g ~j acting at the centre of gravity G, which is at the centre of the uniform
W
beam.
d
mg
d
G
y
x
mg
Mg
Beam regarded as a
large number of
particles, each of
mass m.
Beam regarded as a
single particle of
P
mass M = m.
By symmetry, we see that both force systems have zero moment about the centre
G, and
X
X
~.
−m g ~j = −g ~j
m = −M g ~j = W
Since both force systems have the same sum and moment about G they must be
equivalent; in other words the two force systems must have the same effect on the
beam.