Floating-Point
and
High-Level Languages
Programming Languages
Fall 2003
Floating-Point, the Basics
Floating-point numbers are
approximations of real numbers, but they
are not real numbers.
 Typical format in a machine is
sign exponent mantissa
 Exponent determines range available
 Mantissa determines precision
 Base is usually 2 (rarely 16, never 10)

The Notion of Precision
Precision is relative. Large numbers have
less absolute precision than small numbers
 For example if we have a 24 bit mantissa,
then relative precision is 1 in 2**24
 For 1.0, this is an absolute precision of
1.0*2**(-24).
 For 100, this is an absolute precision of
100*2**(-24).

Representing Numbers
Some numbers can typically be
represented exactly, e,g. 1.0, 2**(-13),
2**(+20) [assume 24 bit mantissa].
 But other numbers are represented only
approximately or not at all

Problems in Representation
2**(-9999999)
Too small, underflows to 0.0
 2**(+9999999)
Too large, error or infinity
 0.1
Cannot be represented exactly in
binary (repeating fracion in binary)
 145678325785.25
Representable in binary, but 24-bit mantissa
too small to represent exactly

Floating-Point Operations
Result may be representable exactly
r = 81.0;
s = 3.0;
x = r / s;
 Machines typically have a floating-point
division instruction 
 But it may not give the exact result 

Floating-Point Operations
Result may not be representable exactly
r = 1.0;
s = 10.0;
t = r / s;
 Result cannot be precisely corrrect
 Will it be rounded to nearest bit, or
perhaps truncated towards zero, or
perhaps even more inaccurate, all are
possible.

Unexpected Results
Let’s look at this code
a = 1.0;
b = 10.0;
c = a / b;
if (c == 0.1) printf (“hello1”);
if (c == 1.0/10.0) printf (“goodbye”);
if (c == a/b) printf (“what the %$!”);
 We may get nothing printed!

Why was Nothing Printed?
if (c == 0.1) …
 In this case, we have stored the result of
the run-time computation of 0.1, but it’s
not quite precise, in c.
 The other operand has been converted to
a constant by the compiler.
 Both are good approximations of 0.1
 But neither are accurate
 And perhaps they are a little bit different

Why Was Nothing Printed?
if (c == 1.0 / 10.0) …
 The compiler may compute 1.0/10.0 at
compile time and treat it as though it had
seen 0.1, and get a different result
 Really ends up being the same as last
case

Why Was Nothing Printed?
if (c == a/b)
 Now surely we should get the same
computation.
 Maybe not, compiler may be clever
enough to know that a/b is 0.1 in one
case and not in the other.

Now Let’s Get Something Printed!
Read in value of a at run time
 Read in value of b at run time
 Compiler knows nothing
 Now we will get some output or else!
c = a / b;
if (c == a/b) printf (“This will print!”);

Still Nothing Printed!!!
How can that be
 First a bit of background
 Typically we have two or more different
precisions of floating-point values, with
different length mantissas
 In registers we use only the higher
precision form, expanding on a load,
rounding on a store.

What Happened?
c = a / b;
if (c == a/b) printf (“This will print!”);
 First compute a/b in high precision
 Now round to fit in low precision c, loosing
significant bits
 Compute a/b in high precision into a
register, load c into a register, expanding
 Comparison does not say equal

Surprises in Precision
Let’s compute x**4
 Two methods:

 Result
= x*x*x*x;
 Result = (x*x)**2
Second has only two multiplications,
instead of 3, must be more accurate.
 Nope, first is more accurate!

Subtleties of Rounding
Suppose we insist on floating-point
operations being properly rounded.
 What does properly rounded mean for 0.5
 Typical rule, round up always if half way
 Introduces Bias
 Some computations sensitive to this bias
 Computation of orbit of pluto significantly
off because of this problem

Moral of this Story
Floating-point is full of surprises
 If you base your expectations on real
arithmetic, you will be surprised
 On any given machine, floating-point
operations are well defined
 But may be more or less peculiar
 But the semantics will differ from machine
to machine

What to do in High Level Languages
We can punt. We just say that floating-point
numbers are some approximation of real
numbers, and that the results of floating-point
operations are some approximation of the real
results.
 Nice and simple from a language definition point
of view
 Fortran and C historically did this
 Not so simple for a poor application programmer

Doing a Bit Better
Parametrize the machine model of
floating-point. What exponent range does
it have, what precision of the mantissa.
 Define fpt model in terms of these
parameters.
 Insist on results being accurate where
possible, or one of two end points it not.
 This is the approach of Ada

Doing Quite a Bit Better
What if all machines had exactly the same
floating-point model?
 IEEE floating-point heads in that direction
 Precisely defines two floating-point
formats (32-bit and 64-bit) and precisely
defines operations on them.

More on IEEE
We could define our language to require IEEE
semantics for floating-point.
 But what if the machine does not efficiently
implement IEEE
 For example, x86 implements the two formats,
but all registers have an 80-bit format, so you
get extra precision
 Which sounds good, but is as we have seem a
possible reason for suprising behavior.

IEEE and High Level Languages
Java and Python both expect/require IEEE
semantics for arithmetic.
 Java wants high efficiency, which causes a
clash if the machine does not support
IEEE in the “right” way.
 Java is potentially inefficient on x86
machines. Solution: cheat 
 Python requires IEEE too, but does not
care so much about efficiency.
