263-2200
Types and Programming Languages
1 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
2 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
3 / 68
(Review) A Context-Free Grammar for NB
The set NB of terms is defined by the following abstract grammar:
t ::=
true
false
if t then t else t
0
succ t
pred t
iszero t
terms
constant true
constant false
conditional
constant zero
successor
predecessor
zero test
4 / 68
(Review) Inference Rules Defining Membership in NB
The set (of terms) NB can also be defined as the smallest set
closed under the following rules:
true ∈ NB
t1 ∈ NB
succ t1 ∈ NB
false ∈ NB
t1 ∈ NB
pred t1 ∈ NB
0 ∈ NB
t1 ∈ NB
iszero t1 ∈ NB
t1 ∈ NB
t2 ∈ NB
t3 ∈ NB
if t1 then t2 else t3 ∈ NB
5 / 68
Generating Functions
Definition
A generating function is a function that takes a set as an input and
produces a set as its output.
F :S →S
6 / 68
Example
R
F (S)
F (R)
=
{1, 2, 3}
def
=
{i + 1 | i ∈ S}
=
{2, 3, 4}
7 / 68
Example
R
F (S)
F (R)
=
{1, 2, 3}
def
=
{i + 1 | i ∈ S ∧ i < 3}
=
{2, 3}
8 / 68
Example
R1
R2
F (S)
F (R1 )
F (R2 )
=
=
{a, f (a), f (f (a))}
{a, b, f (a)}
def
=
{f (i) | i ∈ S}
=
=
{f (a), f (f (a)), f (f (f (a)))}
{f (a), f (b), f (f (a))}
9 / 68
Closure
Definition
Given a set U and a generating function F , we say that U is closed
under F (or F -closed) if:
F (U) ⊆ U
10 / 68
Example
N
=
{0, 1, 2, . . . }
def
=
{i + 1 | i ∈ S}
F (N )
=
{1, 2, 3, . . . }
F (N )
⊆
N
F (S)
11 / 68
Example
N1
=
{b, succ(b), succ(succ(b)), . . . }
def
=
{succ(i) | i ∈ S} ∪ {b}
F (N1 )
=
{b, succ(b), succ(succ(b)), succ(succ(succ(b))), . . . }
F (N1 )
⊆
N1
F (S)
12 / 68
Example
N2
=
{a, b, succ(b), succ(succ(b)), . . . }
def
=
{succ(i) | i ∈ S}
F (N2 )
=
{succ(a), succ(b), succ(succ(b)), . . . }
F (N2 )
6⊆
N2
F (S)
13 / 68
Smallest Closure
Definition
For a given F , a set S is the smallest F -closed set if it satisfies the
following property:
∀R : F (R) ⊆ R → S ⊆ R
14 / 68
Example
Na
Nb
Nab
=
=
=
{a, succ(a), succ(succ(a)), . . . }
{b, succ(b), succ(succ(b)), . . . }
Na ∪ N b
def
=
{succ(i) | i ∈ S} ∪ {a}
F (Na )
F (Nb )
F (Nab )
=
=
=
{a, succ(a), succ(succ(a)), succ(succ(succ(a))), . . . }
{a, succ(b), succ(succ(b)), succ(succ(succ(b))), . . . }
F (Na ) ∪ F (Nb )
F (Na )
F (Nb )
F (Nab )
Na
⊆
6
⊆
⊆
⊆
Na
Nb
Nab
Nab
F (S)
15 / 68
Term Algebras and Generating Functions
Consider the term-algebra rules shown below defining the set NB.
Each of these rules can be thought of as a generating function.
Saying that NB is closed under these rules means that NB cannot
be made any bigger using these generating functions — it already
contains everything “justified by its members.”
true ∈ NB
t1 ∈ NB
succ t1 ∈ NB
false ∈ NB
t1 ∈ NB
pred t1 ∈ NB
0 ∈ NB
t1 ∈ NB
iszero t1 ∈ NB
t1 ∈ NB
t2 ∈ NB
t3 ∈ NB
if t1 then t2 else t3 ∈ NB
16 / 68
Let’s translate each of the rules in the term algebra into a
corresponding generating function.
F1 (U)
F2 (U)
F3 (U)
F4 (U)
F5 (U)
F6 (U)
F7 (U)
=
=
=
=
=
=
=
{true}
{false}
{0}
{succ t1 | t1 ∈ U}
{pred t1 | t1 ∈ U}
{iszero t1 | t1 ∈ U}
{if t1 then t2 else t3 | t1 , t2 , t3 ∈ U}
Each one takes a set of terms U as input and produces a set of
“terms justified by U” as output.
17 / 68
If we now define a generating function for the whole set of
inference rules (by combining the generating functions for the
individual rules),
F (U) = F1 (U) ∪ F2 (U) ∪ F3 (U) ∪ F4 (U) ∪ F5 (U) ∪ F6 (U) ∪ F7 (U)
then we can restate the previous definition of the set of terms NB
like this:
Definition
Let NB denote the smallest set closed under F .
18 / 68
Our alternate definition of the set of terms NB can also be stated
using the generating function F :
NB0
= ∅
NBi+1 = F (NBi )
NB
=
[
NBi
i
Compare this definition of NB with the one we saw last time:
S0
Si+1
=
=
∅
∪
∪
{true, false, 0}
{succ t1 , pred t1 , iszero t1 | t1 ∈ Si }
{if t1 then t2 else t3 | t1 , t2 , t3 ∈ Si }
[
NB =
NBi
i
What we have done is “pull out” the term-generating functionality
and given it the name F .
19 / 68
Note that our two definitions of terms characterize the same set
from different directions:
I
“from above,” as the intersection of all F -closed sets;
I
“from below,” as the limit (union) of a series of sets that start
from ∅ and get “closer and closer to being F -closed.”
Proposition 3.2.6 in the book shows that these two definitions
actually define the same set.
20 / 68
Structural Induction
The principle of structural induction on terms can also be re-stated
using generating functions:
Suppose NB is the smallest F -closed set.
If, for each set U, we can show that
I
I
from the assumption “P(u) holds for every u ∈ U”
we can show “P(v ) holds for any v ∈ F (U),”
then P(t) holds for all t ∈ NB.
21 / 68
Structural Induction
The principle of structural induction on terms can also be re-stated
using generating functions:
Suppose NB is the smallest F -closed set.
If, for each set U, we can show that
I
I
from the assumption “P(u) holds for every u ∈ U”
we can show “P(v ) holds for any v ∈ F (U),”
then P(t) holds for all t ∈ NB.
Why?
22 / 68
Structural Induction
Why? Because:
I
We assumed that NB was the smallest F -closed set, i.e., that
NB ⊆ O for any other F -closed set O.
I
Therefore, if it follows that
for each set U, we can show
I
I
given P(u) for all u ∈ U
we can show P(v ) for all v ∈ F (U)
this amounts to showing that “the set of all terms satisfying
P” (call it O) is itself an F -closed set.
I
Since NB ⊆ O, every element of NB satisfies P.
23 / 68
Structural Induction
Compare this with the structural induction principle for terms from
last lecture:
If, for each term s,
given P(r ) for all immediate subterms r of s
we can show P(s), then P(t) holds for all t.
24 / 68
Recall, from the definition of NB, it is clear that, if a term t is in
NBi , then all of its immediate subterms must be in NBi−1 , i.e.,
they must have strictly smaller depths. Therefore:
If, for each term s,
given P(r ) for all immediate subterms r of s
we can show P(s),
then P(t) holds for all t.
Slightly more explicit proof:
I
Assume that for each term s, given P(r ) for all immediate
subterms of s, we can show P(s).
I
Then show, by induction on i, that P(t) holds for all terms t
with depth i.
I
Therefore, P(t) holds for all t.
25 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
26 / 68
(Review) Abstract Machines
An abstract machine consists of:
I
a set of states
I
a transition relation on states, written −→
We read “t −→ t 0 ” as “t evaluates to t 0 in one step”.
For the simple languages we are considering at the moment, the
term being evaluated is the whole state of the abstract machine.
27 / 68
Syntax for Booleans
Syntax of terms and values
t
v
::=
true
false
if t then t else t
terms
constant true
constant false
conditional
true
false
values
true value
false value
::=
28 / 68
Evaluation relation for Booleans
The evaluation relation t −→ t 0 is the smallest relation closed
under the following rules:
if true then t2 else t3 −→ t2
(E-IfTrue)
if false then t2 else t3 −→ t3
(E-IfFalse)
t1 −→ t10
if t1 then t2 else t3 −→ if t10 then t2 else t3
(E-If)
29 / 68
Digression
Suppose we wanted to change our evaluation strategy so that the
then and else branches of an if get evaluated (in that order)
before the guard. How would we need to change the rules?
30 / 68
Digression
Suppose we wanted to change our evaluation strategy so that the
then and else branches of an if get evaluated (in that order)
before the guard. How would we need to change the rules?
Suppose, moreover, that if the evaluation of the then and else
branches leads to the same value, we want to immediately produce
that value (“short-circuiting” the evaluation of the guard). How
would we need to change the rules?
31 / 68
Digression
Suppose we wanted to change our evaluation strategy so that the
then and else branches of an if get evaluated (in that order)
before the guard. How would we need to change the rules?
Suppose, moreover, that if the evaluation of the then and else
branches leads to the same value, we want to immediately produce
that value (“short-circuiting” the evaluation of the guard). How
would we need to change the rules?
Of the rules we just invented, which are computation rules and
which are congruence rules?
32 / 68
Evaluation, more explicitly
−→ is the smallest two-place relation closed under the following
rules:
((if true then t2 else t3 ), t2 )
∈ −→
((if false then t2 else t3 ), t3 ) ∈ −→
(t1 , t10 ) ∈ −→
((if t1 then t2 else t3 ), (if t10 then t2 else t3 )) ∈ −→
33 / 68
Even more explicitly...
What is the generating function corresponding to these rules?
(exercise)
34 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
35 / 68
Derivations
We can record the “justification” for a particular pair of terms that
are in the evaluation relation in the form of a tree.
Terminology:
I
These trees are called derivation trees (or just derivations).
I
The final statement in a derivation is its conclusion.
I
We say that the derivation is a witness for its conclusion (or a
proof of its conclusion) — it records all the reasoning steps
that justify the conclusion.
36 / 68
Example: Derivation
if true then false else true −→ false
(E-IfTrue)
if (if true then false else true) then true else false −→ if false then true else false
(E-If)
Remark
You can use the trace mode in Prolog to “step through” such
derivations.
37 / 68
Observation
Lemma: Suppose we are given a derivation tree D witnessing the
pair (t, t 0 ) in the evaluation relation. Then either
1. the final rule used in D is E-IfTrue and we have
t = if true then t2 else t3 and t 0 = t2 , for some t2 and t3 , or
2. the final rule used in D is E-IfFalse and we have
t = if false then t2 else t3 and t 0 = t3 , for some t2 and t3 , or
3. the final rule used in D is E-If and we have
t = if t1 then t2 else t3 and
t 0 = if t10 then t2 else t3 , for some t1 , t10 , t2 , and t3 ;
moreover, the immediate subderivation of D witnessess
(t1 , t10 ) ∈ −→
38 / 68
Induction on Derivations
We can now write proofs about evaluation “by induction on
derivation trees.”
Given an arbitrary derivation D with conclusion t −→ t 0 , we
assume the desired result for its immediate sub-derivation (if any)
and proceed by a case analysis (using the previous lemma) of the
final evaluation rule used in constructing the derivation tree.
E.g....
39 / 68
Induction on Derivations — Example
Theorem: If t −→ t 0 then size(t) > size(t 0 ).
Proof: By induction on a derivation D of t −→ t 0 .
1. Suppose the final rule used in D is E-IfTrue, with
t = if true then t2 else t3 and t 0 = t2 . Then the result is
immediate from the definition of size.
2. Suppose the final rule used in D is E-IfFalse, with
t = if false then t2 else t3 and t 0 = t3 . Then the result is
again immediate from the definition of size.
3. Suppose the final rule used in D is E-If, with
t = if t1 then t2 else t3 and
t 0 = if t10 then t2 else t3 , where t1 −→ t10 is witnessed by a
derivation D1 . By the induction hypothesis,
size(t1 ) > size(t10 ). But then, by the definition of size, we
have size(t) > size(t 0 ).
40 / 68
Normal forms
Definition
A normal form is a term that cannot be evaluated any further —
i.e., a term t is a normal form (or “is in normal form”) if there is
no t 0 such that t −→ t 0 .
A normal form is a state where the abstract machine is halted —
i.e., it can be regarded as a “result” of evaluation.
41 / 68
Normal forms
Definition
A normal form is a term that cannot be evaluated any further —
i.e., a term t is a normal form (or “is in normal form”) if there is
no t 0 such that t −→ t 0 .
A normal form is a state where the abstract machine is halted —
i.e., it can be regarded as a “result” of evaluation.
Recall that we intended the set of values (the boolean constants
true and false) to be exactly the possible “results of evaluation.”
Did we get this definition right?
42 / 68
Values = normal forms
Theorem: A term t is a value iff it is in normal form.
Proof:
The ⇒ direction is immediate from the definition of the evaluation
relation.
43 / 68
Values = normal forms
Theorem: A term t is a value iff it is in normal form.
Proof:
The ⇒ direction is immediate from the definition of the evaluation
relation.
For the ⇐ direction,
44 / 68
Values = normal forms
Theorem: A term t is a value iff it is in normal form.
Proof:
The ⇒ direction is immediate from the definition of the evaluation
relation.
For the ⇐ direction, it is convenient to prove the contrapositive: If
t is not a value, then it is not a normal form.
45 / 68
Values = normal forms
Theorem: A term t is a value iff it is in normal form.
Proof:
The ⇒ direction is immediate from the definition of the evaluation
relation.
For the ⇐ direction, it is convenient to prove the contrapositive: If
t is not a value, then it is not a normal form. The argument goes
by induction on t.
Note, first, that t must have the form if t1 then t2 else t3
(otherwise it would be a value). If t1 is true or false, then rule
E-IfTrue or E-IfFalse applies to t, and we are done. Otherwise, t1
is not a value and so, by the induction hypothesis, there is some t10
such that t1 −→ t10 . But then rule E-If yields
if t1 then t2 else t3 −→ if t10 then t2 else t3
i.e., t is not in normal form.
46 / 68
Adding Numbers
New syntactic forms
t
::=
...
0
succ t
pred t
iszero t
terms
constant zero
successor
predecessor
zero test
v
::=
...
nv
values
numeric value
nv
::=
0
succ nv
numeric values
zero value
successor value
47 / 68
New evaluation rules
t1 −→ t10
succ t1 −→ succ t10
(E-Succ)
pred 0 −→ 0
(E-PredZero)
pred (succ nv1 ) −→ nv1
(E-PredSucc)
t1 −→ t10
pred t1 −→ pred t10
(E-Pred)
iszero 0 −→ true
(E-IsZeroZero)
iszero (succ nv1 ) −→ false
(E-IsZeroSucc)
t1 −→ t10
iszero t1 −→ iszero t10
(E-isZero)
48 / 68
Values are normal forms, but we have stuck terms
Our observation a few slides ago that all values are in normal form
still holds for the extended language.
Is the converse true? I.e., is every normal form a value?
49 / 68
Values are normal forms, but we have stuck terms
Our observation a few slides ago that all values are in normal form
still holds for the extended language.
Is the converse true? I.e., is every normal form a value?
No: some terms are stuck.
Formally, a stuck term is one that is a normal form but not a value.
What are some examples?
Stuck terms model run-time errors.
50 / 68
Multi-step Evaluation
The multi-step evaluation relation, −→∗ , is the reflexive, transitive
closure of single-step evaluation.
I.e., it is the smallest relation closed under the following rules:
t −→ t 0
t −→∗ t 0
t −→∗ t
t −→∗ t 0 t 0 −→∗ t 00
t −→∗ t 00
51 / 68
Termination of evaluation
Theorem: For every t there is some normal form t 0 such that
t −→∗ t 0 .
Proof:
52 / 68
Termination of evaluation
Theorem: For every t there is some normal form t 0 such that
t −→∗ t 0 .
Proof:
I First, recall that single-step evaluation strictly reduces the size
of the term:
if t −→ t 0 , then size(t) > size(t 0 )
I
Now, assume (for a contradiction) that
t0 , t1 , t2 , t3 , t4 , . . .
is an infinite-length sequence such that
t0 −→ t1 −→ t2 −→ t3 −→ t4 −→ · · ·
I
Then
size(t0 ) > size(t1 ) > size(t2 ) > size(t3 ) >
size(t4 ) > . . .
I
But such a sequence cannot exist — contradiction!
53 / 68
Termination Proofs
Most termination proofs have the same basic form:
Theorem: The relation R ⊆ X × X is terminating —
i.e., there are no infinite sequences x0 , x1 , x2 , etc. such
that (xi , xi+1 ) ∈ R for each i.
Proof:
1. Choose
I
I
a well-founded set (W , <) — i.e., a set W with a
partial order < such that there are no infinite
descending chains w0 > w1 > w2 > . . . in W
a function f from X to W
2. Show f (x) > f (y ) for all (x, y ) ∈ R
3. Conclude that there are no infinite sequences x0 , x1 ,
x2 , etc. such that (xi , xi+1 ) ∈ R for each i, since, if
there were, we could construct an infinite descending
chain in W .
54 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
55 / 68
Adding Numbers
New syntactic forms
t
::=
...
0
succ t
pred t
iszero t
terms
constant zero
successor
predecessor
zero test
v
::=
...
nv
values
numeric value
nv
::=
0
succ nv
numeric values
zero value
successor value
56 / 68
New evaluation rules
t1 −→ t10
succ t1 −→ succ t10
(E-Succ)
pred 0 −→ 0
(E-PredZero)
pred (succ nv1 ) −→ nv1
(E-PredSucc)
t1 −→ t10
pred t1 −→ pred t10
(E-Pred)
iszero 0 −→ true
(E-IsZeroZero)
iszero (succ nv1 ) −→ false
(E-IsZeroSucc)
t1 −→ t10
iszero t1 −→ iszero t10
(E-isZero)
57 / 68
Step 1: Design a Mapping to a Term Algebra
Abstract Syntax
true
false
if t1 then t2 else t3
−→
map
−→
map
−→
0
succ t
pred t
iszero t
−→
map
−→
map
−→
map
−→
map
map
Term
true
false
cond(t1 , t2 , t3 )
0
succ(t)
pred(t)
iszero(t)
58 / 68
Concrete Examples
true
false
if true then true else false
if false then true else false
0
succ0
pred0
iszero0
iszero(succ0)
map
−→
map
−→
map
−→
map
−→
map
−→
map
−→
map
−→
map
−→
map
−→
true
false
cond(true, true, false)
cond(false, true, false)
0
succ(0)
pred(0)
iszero(0)
iszero(succ(0))
59 / 68
Step 2: Implement Computation and Congruence Rules
Add these clauses to those given for the previous implementation
reduce( succ(T1), succ(T2) )
:-
not(isValue(T1)),
reduce(T1, T2).
reduce( pred(0), 0 ).
reduce( pred(succ(NV)), NV )
:-
isNatValue(NV) .
reduce( pred(T1), pred(T2) )
:-
not(isValue(T1)),
reduce(T1, T2).
reduce( iszero( 0 ), true ).
reduce( iszero( succ(NV) ), false )
:-
isNatValue(NV) .
reduce( iszero( T1 ), iszero( T2 ) )
:-
not(isValue(T1)),
reduce(T1, T2).
60 / 68
Step 3: Implement Syntactic Category Checks
isValue( true ).
isValue( false ).
isValue( NV )
:-
isNatValue( NV ).
isNatValue(0).
isNatValue( succ( NV ) )
:-
isNatValue( NV ).
61 / 68
Step 4: Implement a Computation Engine
% T 0 −→ T 1 −→ T 2 . . . −→ Tnormal form
% General Case:
eval(T0,T normal) :- reduce(T0, T1),
eval(T1, T normal).
% Base Case 1:
eval(V,V)
:-
isValue(V) .
% Base Case 2: this engine assumes “stuck” terms are possible
eval(T,stuck(T))
:- not(isValue(T)),
not(reduce(T, )).
62 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
63 / 68
Big-Step evaluation rules
v ↓v
t1 ↓ true
t2 ↓ v2
if t1 then t2 else t3 ↓ v2
t1 ↓ false
t3 ↓ v3
if t1 then t2 else t3 ↓ v3
t1 ↓ nv1
succ t1 ↓ succ nv1
(B-Value)
(B-IfTrue)
(B-IfFalse)
(B-Succ)
64 / 68
continued...
t1 ↓ 0
pred t1 ↓ 0
t1 ↓ succ nv1
pred t1 ↓ nv1
t1 ↓ 0
iszero t1 ↓ true
t1 ↓ succ nv1
iszero t1 ↓ false
(B-PredZero)
(B-PredSucc)
(B-IsZeroZero)
(B-IsZeroSucc)
65 / 68
Outline
The Language NB
Small-Step Operational Semantics
Reasoning about Evaluation
A Prolog Implementation
Big-Step Semantics
Additional Proof Example
66 / 68
Theorem: If t → t 0 ∧ t → t 00 then t 0 = t 00
Proof: (by induction on the derivation t → t 0 ). We perform a
case analysis on the last step of the derivation.
E −IfTrue
Case 1: t −−−−−→ t 0
1. t = if true then t2 else t3
E −IfTrue
2. t −−−−−→ t 00
3. t 0 = t 00
Given.
By inspection only E-IfTrue is possible.
From 1 and 2.
E −IfFalse
Case 2: t −−−−−−→ t 0
1. t = if false then t2 else t3
E −IfFalse
2. t −−−−−−→
3. t 0 = t 00
t 00
Given.
By inspection only E-IfFalse is possible.
From 1 and 2.
67 / 68
Proof continued...
E −If
Case 3: t −−−→ t 0
1. t = if t1 then t2 else t3 .
2. t1 → t10
E −If
Given.
Semantics of E-If. The previous step
in the derivation.
3. t −−−→ t 00
By inspection of rules.
4. t1 → t100
Required by derivation of t 00 .
5. t10 = t100
By inductive hypothesis.
6. t 0 = t 00
68 / 68