MATH 685/CSI 700
Lecture Notes
Lecture 1.
Intro to Scientific Computing
Useful info




Course website:
http://math.gmu.edu/~memelian/teaching/Spring10
MATLAB instructions:
http://math.gmu.edu/introtomatlab.htm
Mathworks, the creator of MATLAB: http://www.mathworks.com
OCTAVE = free MATLAB clone
Available for download at http://octave.sourceforge.net/
Scientific computing

Design and analysis of algorithms for numerically solving
mathematical problems in science and engineering

Deals with continuous quantities vs. discrete (as, say, computer
science)

Considers the effect of approximations and performs error analysis

Is ubiquitous in modern simulations and algorithms modeling natural
phenomena and in engineering applications

Closely related to numerical analysis
Computational problems:
attack strategy

Develop mathematical model (usually requires a combination of
math skills and some a priori knowledge of the system)

Come up with numerical algorithm (numerical analysis skills)

Implement the algorithm (software skills)
Mathematical modeling

Run, debug, test the software

Visualize the results

Interpret and validate the results
Computational problems:
well-posedness

The problem is well-posed, if
(a) solution exists
(b) it is unique
(c) it depends continuously on problem data
The problem can be well-posed, but still sensitive to perturbations.
The algorithm should attempt to simplify the problem, but not make
sensitivity worse than it already is.
Simplification strategies:
Infinite
finite
Nonlinear
linear
High-order
low-order
Sources of numerical errors





Before computation
 modeling approximations
Cannot be controlled
 empirical measurements, human errors
 previous computations
During computation
Can be controlled through
 truncation or discretization
error analysis
 Rounding errors
Accuracy depends on both, but we can only control the second part
Uncertainty in input may be amplified by problem
Perturbations during computation may be amplified by algorithm
Abs_error = approx_value – true_value
Rel_error = abs_error/true_value
Approx_value = (true_value)x(1+rel_error)
Sources of numerical errors

Propagated vs. computational error

x = exact value, y = approx. value
F = exact function, G = its approximation

G(y) – F(x)

Total error

=
=
[G(y) - F(y)]
+
[F(y) - F(x)]
Computational error:
Propagated data error:
+
affected by algorithm
not affected by algorithm
Rounding vs. truncation error


Rounding error: introduced by finite precision calculations in the
computer arithmetic
Truncation error: introduced by algorithm via problem simplification, e.g.
series truncation, iterative process truncation etc.
Computational error = Truncation error + rounding error
Backward vs. forward errors
Backward error analysis

How much must original problem change to give result actually
obtained?

How much data error in input would explain all error in
computed result?

Approximate solution is good if it is the exact solution to a nearby
problem

Backward error is often easier to estimate than forward error
Backward vs. forward errors
Conditioning




Well-conditioned (insensitive) problem: small relative change in
input gives commensurate relative change in the solution
Ill-conditioned (sensitive): relative change in output is much larger
than that in the input data
Condition number = measure of sensitivity
Condition number = |rel. forward error| / |rel. backward error|
= amplification factor
Conditioning
Stability

Algorithm is stable if result produced is relatively insensitive to
perturbations during computation

Stability of algorithms is analogous to conditioning of
problems

From point of view of backward error analysis, algorithm is stable
if result produced is exact solution to nearby problem

For stable algorithm, effect of computational error is no worse
than effect of small data error in input
Accuracy

Accuracy : closeness of computed solution to true solution
of problem

Stability alone does not guarantee accurate results

Accuracy depends on conditioning of problem as well as
stability of algorithm

Inaccuracy can result from applying stable algorithm to
ill-conditioned problem or unstable algorithm to well-conditioned
problem

Applying stable algorithm to well-conditioned problem
yields accurate solution
Floating point representation
Floating point systems
Normalized representation
Not all numbers can be represented this way, those that can are called
machine numbers
Rounding rules

If real number x is not exactly representable, then it is approximated by
“nearby” floating-point number fl(x)

This process is called rounding, and error introduced is called rounding
error

Two commonly used rounding rules



chop: truncate base- expansion of x after (p − 1)st digit;
also called round toward zero
round to nearest : fl(x) is nearest floating-point number to x, using floatingpoint number whose last stored digit is even in case of tie; also called round
to even
Round to nearest is most accurate, and is default rounding rule in IEEE
systems
Floating point arithmetic
Machine precision
Floating point operations
Summing series in floatingpoint arithmetic
Loss of significance
Loss of significance
Loss of significance
Loss of significance