The environment-based
operational semantics
Chapter 4.3
1
What’s Wrong with Substitution?
• On every application we have to:
– Rename and substitute: heavy operations
– Analyze the body (ask what kind of expression it is
etc)
• mixed value/expression. Evaluator value
distinction required
2
The Environment Based Operational Semantics
• Replace local variable management
Substitution with A hierarchical environment
structure.
• The env-eval algorithm evaluates an
expression with respect to an environment.
• Advantage:
– Directly access local variables (procedures are
pure code, no need for evaluator value distinction.
– Later: Body of procedure may be analyzed once.
3
The Environments Model
Substitution model: a single global environment
Environment Table
Name
score
Value
23
Environments model: many environments.
4
Binding: a pairing of a name and a value
Frame: a table of bindings
Example:
x is bound to 15 in frame A
y is bound to (1 2) in frame A
the value of the variable x in frame A is 15
A
x: 15
y:
2
1
5
Environment: a sequence of frames
• Environment E1 consists of frames A and B
• Environment E2 consists of frame B only
(A frame may be shared by multiple environments)
B
E2
z: 10
this arrow is called
the enclosing
environment pointer
A
E1
x: 15
y:
1
6
2
Evaluation in the environment model
• All evaluations occur with respect to an environment
• The current environment changes when the
interpreter applies a procedure
• The top environment is called the global environment
(GE)
• Only the GE has no enclosing environment
7
The environment data structure
frame is a list of bindings, a variable-value mapping: Variable –> Scheme-type.
environment is a finite sequence of frames E=<f1, f2,..., fn>.
environment diagram
8
Operations on environments and frames:
9
Operations on environments and frames:
10
Operations on environments and frames:
11
The closure data structure
• The closure is the value of a lambda expression.
<Closure parameters, body, environment>.
• The components of a closure cl are denoted
clparameters , clbody , clenvironment .
• The closure carries the environment it was constructed in.
• this enables the evaluator algorithm to have a lexical scoping
policy.
12
The closure data structure
• A closure carries an environment - the one containing all local
variables defined when it was created.
>(define f
B
(let ((x 3)) 1
(lambda(y) (+ x y)))
>(f 1)
B2
The interpreter need to know that:
• x=3 when evaluating B1.
• This local variable binding needs to be saved for future use of the
closure corresponding to (lambda(y) (+ x y)).
• Procedure application involves an extension of that environment.
13
Double bubble: how to draw a
procedure
(lambda (x) (* x x))
#[proc-...]
Environment
pointer
A compound proc
that squares its
argument
Code pointer
parameters: x
body: (* x x)
14
The Environment Model Evaluation Algorithm
15
The Environment Model Evaluation Algorithm continued
16
The Environment Model Evaluation Algorithm continued
17
The Environment Model Evaluation Algorithm continued
18
Notes
• The recursive algorithm passes an 'env' parameter
• env-eval consults or modify the environment structure in the following
steps:
(a) Creation of a compound procedure (closure): Evaluation of a 'lambda'
expression (and 'let').
(b) Application of a compound procedure (closure) –
the only way to add a frame (also 'let').
(c) Evaluation of 'define' expression – adds a binding to the global
environment.
(d) Evaluation of a variable.
• De-allocation of frames: garbage collection...
• An environment corresponds to a lexical scope
19
Example 4.7.
>(define member
(lambda (x list)
(cond ((null list) (list))
((eq? x (car list)) list)
(else (member x (cdr list)))))
>(define a (list ’a ’b ’c))
>(member ’b a)
Drawing environment diagrams is a way to represent the computation of the
env-eval algorithm.
20
Example 4.8.
Try a "curried" version of member:
>(define c_member
(lambda (list)
(lambda (el)
(cond ((null list) (list))
((eq? el (car list)) list)
(else ((c_member (cdr list)) el))))))
>(define a (list ’a ’b ’c))
>(define search-a (c_member a))
>(search-a 'b)
21
Example 4.5.
22
Example 4.6.
23
Static (Lexical) vs. Dynamic Scoping Policies
(Section 4.3.3)
• Policies for interpreting variables (variable scoping) in a program.
• applicative-eval, normal-eval and env-eval algorithms
have a Static (lexical) scoping policy.
The nesting of lexical blocks determines the variable binding at runtime
• In dynamic scoping, a variable occurrences is bound by the most
recent declaration of that variable.
• In dynamic scoping: the access link is defined by the control link and
closures do not carry an environment.
• Do not confuse static scoping with static type-inference algorithms!
=> languages with static scoping policies allow for static type inference
24
dynamic-env-eval
25
Example 4.6.
• Not all evaluation algorithms are equivalent! dynamic-eval != env-eval
(compute the same function, have the same domain)
26
Example 4.10.
>(define f
(lambda (x) (a x x)))
>(define g
(lambda (a x) (f x)))
>(define a +)
>(g * 3)
env-eval[(g*3),GE] ==> 6
dynamic-env-eval[(g*3),GE] ==> 9
27
Example 4.12.
>(define init 0)
>(define 1+
(lambda(x)(+ x 1)))
>(define f
(lambda (f1)
(let ((f2 (lambda () (f1 init))))
(let ((f1 1+)
(init 1))
(f2) ))))
>(f (lambda (x) (* x x)))
env-eval[(f (lambda (x) (* x x)))] ==> 0
dynamic-env-eval[(f (lambda (x) (* x x)))] ==> 2
28
4.4 The env-eval Evaluator Implementation
1. Abstract Syntax Parser (same as "applicative-eval" implementation)
2. Data structures - environment hierarchy, closures.
3. Core ("env-eval" algorithm implementation)
; Type: [<Scheme-exp> -> Scheme-type]
(define derive-eval
(lambda (exp)
(env-eval (derive exp) the-global-environment)))
29
evaluator structure
evaluator
Scheme
expression
(Global)
Environment
Value
Chapter 4 - Evaluators for Functional
Programming
30
Files
Racket-Evaluators\env-functional-interpreter-compiler>dir
analyzer-core.rkt
analyzer-tests.rkt
env-ds.rkt
interpreter-core.rkt
interpreter-tests.rkt
31
4.4.2 Data Structures Package
4.4.2.1 Procedure ADTs and their implementation
• Primitive procedure: same as in applicative-eval.
32
4.4.2 Data Structures Package
4.4.2.1 Procedure ADTs and their implementation
• A Closure (procedure value) - contains an environment in which is was
created
33
4.4.2 Data Structures Package
4.4.2.1 Procedure ADTs and their implementation
• A Closure (procedure value) - contains an environment in which is was
created
• Identify procedures in general
34
4.4.2.2 Environment related ADTs and their
implementations:
• The interpreter holds a "DrRacket" variable
the-global-environment
* Bindings, Frames, Environments.
35
4.4.2.2 Environment related ADTs and their implementations:
The Binding ADT and its implementation:
binding
var
val
Alternative definition:
(define
(define
(define
What is
make-binding cons)
binding-variable car)
binding-value cdr)
the difference?
36
4.4.2.2 Environment related ADTs and their implementations:
The Frame ADT and its implementation:
(define make-frame
(lambda (variables values)
(make-sub variables values)))
(define make-sub
(lambda (variables values)
(let ((sub (list variables values)))
(if (sub? sub)
sub
(error …)))))
37
4.4.2.2 Environment implementation
• An environment is implemented as a list of boxed (mutable) frames.
Racket box operations
box(x)
unbox(b)
set-box!(b, y)
environment
box
frame
substitution
frame
substitution
* in implementation language
frame
substitution
variable->value lookup function
38
4.4.2.2 Global environment construction
39
40
(define lookup-variable-value
(lambda (env var)
(letrec ((defined-in-env
(lambda (var env)
(if (empty-env? env)
env
(let ((b (get-value-of-variable
(first-frame env)
var)))
(if (eq? b '_not-found)
(defined-in-env var (enclosing-env env))
b))))))
(let ((b (defined-in-env var env)))
(if (empty? b)
(error 'lookup "variable not found: ~s\n env = ~s" var env)
b)))))
41
(define get-value-of-variable
(lambda (sub var)
(letrec ((lookup
(lambda (vars vals)
(cond ((or
(empty-sub? sub)
(not (member var vars)))
'_not-found)
((eq? var (car vars)) (car vals))
(else (lookup (cdr vars) (cdr vals)))))))
(lookup (get-variables sub) (get-values sub)))))
42
; Global environment mutator: ADT type is [Binding -> Unit]
; Type: [PAIR(Symbol,T) -> Unit]
; Note: Mutation is enabled only for the global environment
(define add-binding!
(lambda (binding)
(let ((frame (first-frame the-global-environment)))
(set-box! (first-boxed-frame the-global-environment)
(extend-frame binding frame)))))
43
4.4.1.1 Main evaluator loop:
44
45
46
4.4.1.2 Evaluation of atomic expressions
47
48
49
50
4.4.1.4 Evaluation of applications
; Type: [Evaluator-procedure*LIST -> Scheme-type]
(define apply-procedure
(lambda (procedure arguments)
(cond ((primitive-procedure? procedure)
(apply-primitive-procedure procedure arguments))
((compound-procedure? procedure)
(let* ((parameters (procedure-parameters procedure))
(body (procedure-body procedure))
(env (procedure-environment procedure))
(new-env
(extend-env
(make-frame parameters arguments)
env)))
(eval-sequence body new-env)))
(else
(error 'apply "unknown procedure type: ~s" procedure))))
Primitive procedure application
52
Testing
(define (app-lambda-tests)
(test (derive-eval '((lambda (x) x) 12)) => 12)
(test
(derive-eval '((lambda (x y z) (+ x y z)) 12 13 14))
=> 39)
...
)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Invoking tests
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(app-lambda-tests)
53
The Analyzer
So far, no distinction between syntax analysis
and evaluation
For example: the kind of an expression (special
form, application etc) can be decided
statically.
54
Files
Racket-Evaluators\env-functional-interpreter-compiler>dir
analyzer-core.rkt
analyzer-tests.rkt
env-ds.rkt
interpreter-core.rkt
interpreter-tests.rkt
55
4.5.1 The Analyzer
• Compile time (static time):
Things performed first, once if possible.
– static syntax analysis
• Run time (dynamic time):
Things performed later, as needed
– data-structure (environments and closures) related
• Compile time is less expensive than run time.
• Analyzing a procedure body once, independently from
its application, means compiling its code into something
more efficient/optimal, which is ready for evaluation.
56
analyzer structure
Scheme
expression
code in the
implementation
language.
syntax-analysis
(compilation)
Program in
target
language
(Global)
Environment
run-time
evaluation
Value
Chapter 4 - Evaluators for Functional
Programming
57
4.5 An Environment-based FP Meta-Circular Compiler
The analyzer
• avoid repetition of syntax-analysis in every procedure
application.
• Idea: separate static syntax analysis (syntax parsing)
from run-time evaluation (closure/environment datastructure manipulation).
• means: curried function style.
58
An Environment-based FP Meta-Circular Compiler
The analyzer
• Use currying
Interpreter:
Compiler:
(lambda (exp env)
(lambda (exp)
syntax analysis+
run-time evaluation
syntax analysis
(performed once)
returns
(lambda(env)
[Expression*Env->T]
run-time evaluation
[Expression->[Env->T]]
• Compiler output is code in the implementation language.
59
Environment-based compiler
(define derive-analyze-eval
(lambda(exp)
((analyze (derive exp)) the-global-environment))
(define analyze
(lambda (exp)
(cond ((atomic? exp) (analyze-atomic exp))
((special-form? exp) (analyze-special-form exp))
((application? exp) (analyze-application exp))
(else
(error "Unknown expression type -- EVAL" exp)
))))
60
Environment-based compiler
• Interpreter:
(define eval-atomic
(lambda (exp env)
(if (or (number? exp) (boolean? exp) (null? exp))
exp
(lookup-variable-value exp env))))
• Compiler:
(define analyze-atomic
(lambda (exp)
(if (or (number? exp) (boolean? exp) (null? exp))
(lambda (env) exp)
(lambda (env) (lookup-variable-value exp env)))))
61
Environment-based compiler
• Interpreter:
(define eval-if
(lambda (exp env)
(if (true? (env-eval (if-predicate exp) env))
(env-eval (if-consequent exp) env)
(env-eval (if-alternative exp) env))))
• Compiler:
(define analyze-if
(lambda (exp)
(let ( (pred (analyze (if-predicate exp)))
(consequent (analyze (if-consequent exp)))
(alternative (analyze (if-alternative exp))))
(lambda (env)
(if
(true? (pred env))
(consequent env)
(alternative env))))))
62
Environment-based compiler
• Interpreter
(define eval-lambda
(lambda (exp env)
(make-procedure (lambda-parameters exp)
(lambda-body exp)
env)))
• Compiler - body is analyzed once!
(define analyze-lambda
(lambda (exp)
(let ((parameters (lambda-parameters exp))
(body (analyze-sequence (lambda-body exp))))
(lambda (env)
(make-procedure parameters body env))))
63
Environment-based compiler
(define analyze-sequence
(lambda (exps)
(let ((procs (map analyze exps))
(last-in-list (lambda (lst) (car (reverse lst)))))
(if (null? procs)
(error "Empty sequence -- ANALYZE"))
(lambda (env)
(let ((vals (map (lambda (proc)(proc env))
procs)))
(last-in-list vals)))))
• relies on the order of map in the underlining
(implementation) Scheme.
64
Environment-based compiler
• Interpreter
(define eval-special-form
(lambda (exp env)
(cond ...
((definition? exp)
(if (not (eq? env the-global-environment))
(error "Non global definition" exp)
(eval-definition exp)))
...
(define eval-definition
(lambda (exp)
(add-binding!
(make-binding (definition-variable exp)
(env-eval (definition-value exp)
the-global-environment)))
’ok))
65
Environment-based compiler
• Compiler:
(define (analyze-definition
(lambda (exp)
(let ((var (definition-variable exp))
(val (analyze (definition-value exp))))
(lambda (env)
(if (not (eq? env the-global-environment))
(error "Non global definition" exp)
(begin (add-binding!
(make-binding
var
(val the-global-environment)))
’ok))))))
66
• Interpreter:
(define env-eval
(lambda (exp env)
(cond ...
((application? exp)
(apply-procedure (env-eval (operator exp) env)
(list-of-values (operands exp) env)))
...
Compiler:
(define analyze
(lambda (exp)
(cond ... ((application? exp) (analyze-application exp))...
(define analyze-application
(lambda (exp)
(let
((application-operator (analyze (operator exp)))
(application-operands (map analyze (operands exp))))
(lambda (env)
(apply-procedure
(application-operator env)
(map (lambda (operand) (operand env))
application-operands))))))
67
• Interpreter:
(define apply-procedure
(lambda (procedure args)
(cond ((primitive-procedure? procedure)
(apply-primitive-procedure procedure args))
((compound-procedure? procedure)
(let
((proc-params (procedure-parameters procedure))
(proc-body (procedure-body procedure))
(proc-env (procedure-environment procedure)))
(eval-sequence
proc-body
(extend-env
(make-frame proc-params args)
proc-env))
(else (error ... )))))
68
• Compiler - evaluation of analyzed operator on extended environment
(define apply-procedure
(lambda (procedure args)
(cond ((primitive-procedure? procedure)
(apply-primitive-procedure procedure args))
((compound-procedure? procedure)
(let ((proc-params
(procedure-parameters procedure)))
(proc-abody
(procedure-body procedure))
(proc-env
(procedure-environment procedure)))
(proc-abody
(extend-env
(make-frame proc-params args)
proc-env))
(else (error ... )))))
69
No repeated analysis - Tracing example
> (require racket/trace)
> (trace analyze)
> (derive-analyze-eval ’(define (factorial n) (if (= n 1) 1 (* (factorial (- n 1)) n)))
| (analyze (define factorial (lambda(n) (if (= n 1) 1 (* (factorial (- n 1)) n))))
| |(analyze (lambda (n) (if (= n 1) 1 (* (factorial (- n 1)) n))))
| | (analyze (if (= n 1) 1 (* (factorial (- n 1)) n)))
| | |(analyze (= n 1))
| | | (analyze =)
| | | #<procedure>
| | | (analyze n)
| | | #<procedure>
| | | (analyze 1)
| | | #<procedure>
| | |#<procedure>
// returned from analyze '(= n 1)
70
| | |(analyze 1)
| | |#<procedure>
| | |(analyze (* (factorial (- n 1)) n))
| | | (analyze *)
| | | #<procedure>
| | | (analyze (factorial (- n 1)))
| | | |(analyze factorial)
| | | |#<procedure>
| | | |(analyze (- n 1))
| | | | (analyze -)
| | | | #<procedure>
| | | | (analyze n)
| | | | #<procedure>
| | | | (analyze 1)
| | | | #<procedure>
| | | |#<procedure> // returned from analyze '(- n 1)
| | | #<procedure> // returned from analyze '(factorial (- n 1))
| | | (analyze n)
71
No repeated analysis - example
| | | #<procedure>
| | |#<procedure>
| | #<procedure>
| |#<procedure>
| #<procedure>
|ok
// returned from analyze '( * (factorial...
// returned from analyze '(if ...
// returned from analyze '(lambda ...
// returned from analyze '(define ...)
// returned from application on the-global-environment
> (derive-analyze-eval ’(factorial 4))
|( (analyze (factorial 4)) the-global-environment)
| (analyze (factorial 4))
| |(analyze factorial)
| |#<procedure>
// returned from analyze 'factorial
| |(analyze 4)
| |#<procedure>
// returned from analyze 4
| #<procedure>
// returned from analyze '(factorial 4)
|24
// returned from application on the-global-environment
// no recursive analysis when recursive analyzed procedure is applied!!
72
Chapter 4 Summary
• Interpreter algorithms that have a static scoping policy:
applicative-eval, env-eval and normal-eval
are functionally equivalent (on the domain conjunction).
• An interpreter algorithm that has a dynamic scoping policy.
dynamic-env-eval
• Implementations for applicative-eval, env-eval :
use ASP (incl. handling expression derivation), Data structures
(environments, procedure and other values), have test modules.
• Analyzer optimization for the environment based interpreter.
73