Lecture 1
Introduction to Numerical Computing
Numerical Computing?
J. Chaudhry (Zeb)
Numerical Analysis?
Department of Mathematics and Statistics
University of New Mexico
J. Chaudhry (Zeb) (UNM)
Math/CS 375
1 / 28
Numerical Computation vs. Symbolic Computation
J. Chaudhry (Zeb) (UNM)
Math/CS 375
2 / 28
Analytic Solution vs. Numerical Solution
Analytic Solution (a.k.a. symbolic): The exact numerical or symbolic
representation of the solution
Numerical Computation: involve numbers directly
I
I
manipulate numbers to produce a numerical result
Numerical Solution: The computational representation of the solution
Symbolic Computation: symbols represent numbers
I
I
manipulate symbols according to mathematical rules to produce a symbolic
result
Example (numerical)
may use special characters such as π, e, or tan (83)
entirely numerical
Example (analytic)
1
4
1
3
π
Example (symbolic)
(17.36)2 − 1
= 16.36
17.36 + 1
x2 − 1
=x−1
x+1
Example (numerical)
0.25
0.33333 . . . (?)
3.14159 . . . (?)
0.88472 . . . (?)
tan (83)
J. Chaudhry (Zeb) (UNM)
Math/CS 375
3 / 28
J. Chaudhry (Zeb) (UNM)
Math/CS 375
4 / 28
Numerical Computation and Approximation
Method vs. Algorithm vs. Implementation
Numerical Approximation is needed to carry out the steps in the
numerical calculation. The overall process is a numerical computation.
Method: a general (mathematical) framework describing the solution
process
Example (symbolic computation, numerical solution)
Algorithm: a detailed description of executing the method
Implementation: a particular instantiation of the algorithm
1 1 1
1
+ + −1=
= 0.083333333 . . .
2 3 4
12
Is it a “good” method?
Is it a robust algorithm?
Example (numerical computation, numerical approximation)
Is it a fast implementation?
0.500 + 0.333 + 0.250 − 1.000 = 0.083
J. Chaudhry (Zeb) (UNM)
Math/CS 375
5 / 28
The Big Theme
J. Chaudhry (Zeb) (UNM)
Math/CS 375
6 / 28
History: Numerical Algorithms
date to 1650 BC: The Rhind Papyrus of ancient Egypt contains 85
problems; many use numerical algorithms (T. Chartier, Davidson)
Approximates π with
(8/9)2 ∗ 4 ≈ 3.1605
Accuracy
J. Chaudhry (Zeb) (UNM)
Cost
Math/CS 375
7 / 28
J. Chaudhry (Zeb) (UNM)
Math/CS 375
8 / 28
History: Archimedes
History: Method of Exhaustion
A circle is not a polygon
A circle is a polygon with an infinite number of sides
287-212BC developed the ”Method of
Exhaustion”
Method for determining π
I
I
I
Cn = circumference of an n-sided polygon inscribed in a circle of radius
1/2
limn→∞ = π
find the length of the permieter of a
polygon inscribed inside a circle of
radius 1/2
find the permiter of a polygon
circumscribed outside a circle of
radius 1/2
the value of π is between these two
lengths
Archimedes deterimined
22
223
<π <
71
7
3.1408 <π < 3.1429
two places of accuracy....
http://www.pbs.org/wgbh/nova/archimedes/pi.html
J. Chaudhry (Zeb) (UNM)
Math/CS 375
9 / 28
History: Method of Machin
J. Chaudhry (Zeb) (UNM)
Math/CS 375
10 / 28
Numerical Analysis
Around 1700, John Machin discovered the trig identity
Definition (Trefethen)
1
1
π = 16 arctan ( ) − 4 arctan (
)
5
239
Study of algorithms for the problems of continuous mathematics
Led to calculation of the first 100 digits of π
We’ve been doing this since Calculus (and before!)
Uses the Taylor series of arctan in the algorithm
arctan (x) = x −
Riemann sum for calculating a definite integral
Newton’s Method
x3 x5 x7
+
−
...
3
5
7
Taylor’s Series expansion + truncation
Used until 1973 to find the first Million digits
J. Chaudhry (Zeb) (UNM)
Math/CS 375
11 / 28
J. Chaudhry (Zeb) (UNM)
Math/CS 375
12 / 28
Big Questions
Numerical Analysis
How algorithms work and how they fail
A Numerical Analyst needs
Why algorithms work and why they fail
computational knowledge (e.g. programming skills)
understanding of the application (physical intuition for validation)
Connects mathematics and computer science
Need mathematical theory, computer programming, and scientific inquiry
J. Chaudhry (Zeb) (UNM)
Math/CS 375
13 / 28
Numerical Analysis
mathematical ability to construct a meaningful algorithm
J. Chaudhry (Zeb) (UNM)
Math/CS 375
14 / 28
Numerical Analysis
Why?
Numerical methods improve scientific simulation
Some disasters attributable to bad numerical computing (Douglas Arnold)
Numerical focus:
Approximation An approximate solution is sought. How close is this to the
desired solution?
I
Efficiency How fast and cheap (memory) can we compute a solution?
Stability Is the solution sensitive to small variations in the problem setup?
I
Error What is the role of finite precision of our computers?
I
J. Chaudhry (Zeb) (UNM)
Math/CS 375
15 / 28
The Patriot Missile failure, in Dharan, Saudi Arabia, on February 25, 1991
which resulted in 28 deaths, is ultimately attributable to poor handling of
rounding errors.
The explosion of the Ariane 5 rocket just after lift-off on its maiden voyage off
French Guiana, on June 4, 1996, was ultimately the consequence of a
simple overflow.
The sinking of the Sleipner A offshore platform in Gandsfjorden near
Stavanger, Norway, on August 23, 1991, resulted in a loss of nearly one
billion dollars. It was found to be the result of inaccurate finite element
analysis.
J. Chaudhry (Zeb) (UNM)
Math/CS 375
16 / 28
I thought we were studying ”Numerical Computating”?!
Numerical analysis is the study of numerical computing
In Engineering
Numerical Analysis: understanding general behavior of numerical
computing
Numerical Computing: understanding how to apply certain methods to
solve specific tasks
As computational scientists, we need to understand the concepts of
numerical analysis and implementation aspects of the numerical
computing
We thus focus on
Matlab implementation
I
I
I
I
Applications: Mathematics, Engineering, Computer
Science...
fast learning curve
quick time-to-production: low development times
a major development environment in scientific computing
integrated graphics
Biomolecular
Systems
Rocket
Simulations
Crack
propagation
Errors in computation
Specific methods for solving linear and nonlinear systems, root finding,
integrating, interpolation, etc.
J. Chaudhry (Zeb) (UNM)
Math/CS 375
17 / 28
Applications: Mathematics, Engineering, Computer
Science...
J. Chaudhry (Zeb) (UNM)
Math/CS 375
18 / 28
Course Info
http://math.unm.edu/˜jehanzeb/math375sp17.html
In Computer Science?
AI: transitions, state systems, patterns (eigenvalues, linear
algebra)
Informatics: Google matrix, Amazon recommendations
(eigenvalues, linear algebra, sparse matrices, iterative methods)
Book: Numerical Mathematics and Computing, Seventh
Edition, by Cheney and Kincaid (or 6th)
Graphics: image compression, representation of
curves/surfaces/lighting (interpolation, differentiation, etc)
Security: ssh (random numbers)
Zeb’s contact:
Economics/Finance: modeling/simulation of financial data
(monte carlo)
Office: SMLC 328
Office Hours: Tuesday: 1:30 - 3:00 pm, Wednesday: 9:00 - 10:30 am
Scientific computing: design of algorithms for high
performance/parallel computing.
J. Chaudhry (Zeb) (UNM)
Math/CS 375
email: [email protected]
19 / 28
J. Chaudhry (Zeb) (UNM)
Math/CS 375
20 / 28
Assessment
Finally. . .
http://math.unm.edu/˜jehanzeb/math375sp17.html
Homework
Exam 1
Grades:
Exam 2
Final Exam
Homework:
http://math.unm.edu/˜jehanzeb/math375sp17.html
50%
15%
15%
30%
Schedule and Notes:
on the web
Assigned approximately weekly
everything is tentative including midterm exams
submitted as a report
final exam is fixed
source code included upon request
Grades posted on UNM Learn
no graphics by hand
accepted for 50% credit up to one week later
no dropped scores (or maybe at most one)
collaborate, groups of 2
Questions?
cite collaborators at the top, submit one report
J. Chaudhry (Zeb) (UNM)
Math/CS 375
21 / 28
Preliminaries...
Math/CS 375
22 / 28
Typical Process
Goal
A computational experiment...
Find f 0 (1.0) for function f (x) = ex .
relative vs absolute error
Matlab
Problem
Don’t (want to need to) know f 0 (x) explicitly.
Goals
To think like a numerical analyst
Method
Understanding error in numerical computing
Use definition.
Utilizing Matlab
J. Chaudhry (Zeb) (UNM)
J. Chaudhry (Zeb) (UNM)
f (x + h) − f (x)
h→0
h
f 0 (x) = lim
Math/CS 375
23 / 28
J. Chaudhry (Zeb) (UNM)
Math/CS 375
24 / 28
Typical Process continued
Measuring Error
Here we used the absolute error:
Method
Use definition.
Error(xa ) = |xt − xa |
f (x + h) − f (x)
f (x) = lim
h→0
h
0
Listing 1: Algorithm (psuedocode)
1
2
3
4
5
6
7
8
Listing 2: Implementation (Matlab)
1
h=1
x=1
for i=1 to 20 do
h = h/2
y = (f (x + h) − f (x))/h
err = |(f (x) − y)|
end
J. Chaudhry (Zeb) (UNM)
= |true value − approximate value|
2
3
4
5
6
7
8
f= @(x) (exp(x));
h=1;
x=1;
for i =1:20
h=h/2;
y=(f(x+h) - f(x))/h;
err(i) = abs(f(x)-y);
end
Math/CS 375
More on types of errors in future...
25 / 28
What is MATLAB?
initially developed as an easy interface to LAPACK: Linear Algebra
Package (FORTRAN libraries)
MATrix LABoratory
Written in C. For matrix computations, it calls C/Fortran libraries == Fast
Matlab + “Toolboxes”
I
I
I
I
I
I
Symbolic Math Toolbox: mathematical manipulation of symbols
Partial Differential Equation Toolbox: tools for solving PDEs in 2-D
Statistics Toolbox: statistical data analysis
Image processing toolbox: visualization and image analysis
Bioinformatics toolbox: computational molecular biology
Compiler: application development
many many more.
http://www.mathworks.com
J. Chaudhry (Zeb) (UNM)
J. Chaudhry (Zeb) (UNM)
Math/CS 375
26 / 28
Why Matlab?
both a computing environment and a language
I
This doesn’t tell the whole story. For example, if the values are large, like
billions, then an Error of 100 is small. If the values are smaller, say
around 10, then an Error of 100 is large. We need the relative error:
xt − xa Rel(xa ) = xt true value − approximate value =
true value
Math/CS 375
27 / 28
The Good:
Fast development times
no compiling, easy debugging
accessible syntax and language constructs
in-house graphics capabilities
tons of basic ”libraries” or functions available
many more complicated “toolboxes” can be added
The Bad:
small coding mistakes can result in slow code
loops are computationally intensive
language is limited: no templates etc.
The Ugly:
proprietary (but the language format is open)
expensive (but free for UNM students/faculty?)
the open source substitute, Octave, is not fully compatible
Now we fire up Matlab and start the tutorial...
J. Chaudhry (Zeb) (UNM)
Math/CS 375
28 / 28