Lab 07
Interpreters 3 – Closures, Set! and Letrec
Due Wednesday April 22 at 11:59 PM
In this
lab we will finish the interpreter project.
No
language would be complete without the ability to create new procedures. Our new version of MiniScheme will implement the lambda expression. A lambda
expression should evaluate to a package containing the formal parameters, the
body, and the environment that was current when the procedure was created (i.e.
when the lambda expression was evaluated. This package is known as a closure.
You should start by making a datatype
for closures that holds 3 parts: parameters, body, and environment.
We
parse a lambda expression such as (lambda (x y) (+ x y) )
into a lambda-exp tree.
This is a new kind of tree node with 2 parts: the parameter list (x y) and the
tree that results from parsing the body. The parse function doesn't track the
environment, so it can't build a full closure. Parsing a lambda expression just
gives a tree; it is when we evaluate that tree that we get a closure. If exp is
the tree we get from parsing such a lambda expression, (eval-exp exp
env) builds a closure with exp's parameter list and body combined with
the environment env.
We
are ready for MiniSchemeF. The syntax is extended
once more, this time to include lambda expressions. Here is the grammar for MiniSchemeF.
EXP ::=
number parse
into lit-exp
| symbol parse
into var-ref
| (if EXP EXP EXP) parse into if-exp
| (let (LET-BINDINGS) EXP) parse into let-exp
| (lambda (PARAMS) EXP) parse into
lambda-exp
| (EXP EXP*) parse into
app-exp
LET-BINDINGS ::= LET-BINDING*
LET-BINDING ::= (symbol EXP)
PARAMS ::= symbol*
We
parse a lambda expression into a lambda-exp node that stores the parameter list
and the parsed body. In eval-exp, we evaluate a lambda-exp node as a closure
that has the lambda-exp's parameter list and parsed body and
also the current enviornment.
In Part
3 of Lab06 (Minischeme C) we defined apply-proc like this:
(define apply-proc (lambda (p arg-values)
(cond
[ (prim-proc? p)
(apply-primitive-op p arg-values)]
[
else (error 'apply-proc "Bad procedure: ~s" p)])))
We now extend this with a
case for p being a closure. To evaluate the application of a closure to some
argument values we start with the environment from the closure, extend that
environment with bindings of the parameters to the arg-values,
and call eval-exp on the closure's body with this extended environment. We have
already written procedures to handle each of these steps; it is just a matter
of calling them. After implementing this
you should be able to do the following:
> (read-eval-print)
MS> ((lambda (x) x) 1)
1
MS> ((lambda (x y) (* x y)) 2 4)
8
MS> (let ((sqr (lambda (x) (* x x)))) (sqr 64))
4096
MS> (let ((sqr (lambda (x) (* x x)))) (let ((cube
(lambda (x) (* x (sqr x))))) (cube 3)))
27
Our
next feature will be variable assignment, with set!.
Unfortunately, our implementation of environments does not provide a way to
change the value bound to a variable. We will modify our old implementation so
that variable names are bound to a mutable datatype called a box, which is
provided by Dr. Racket.
Take
a moment to familiarize yourself with boxes in Scheme:
> (define abox
(box 17))
> (box? abox)
#t
> (unbox abox)
17
> (set-box! abox 32)
> (unbox abox)
32
> ...
When
variables are created (when we extend an environment) we will bind them to
boxes. When they are referenced we will unbox their
bindings. We will take these tasks sequentially.
First,
when eval-exp currently evaluates a varref-exp, it
gets the symbol from the expression and looks it up in the environment. When
our variables all refer to boxes, eval-exp needs to do an extra step -- it gets
the symbol from the expression, looks it up in the environment, and unboxes the
result.
Secondly,
whenever the environment is extended, the new bindings will be boxes that
contain values. This occurs in two places. One is when we evaluate a
let-expression in eval-exp, the other is when we apply a closure in apply-proc.
For the latter our code used to be a recursive call to eval-exp on the body
from the closure, using the environment (extended-env params arg-values env). After we introduce boxes we will still do this with a recursive call to eval-exp on
the body only now we need to box the arg values as we extend the environment.
There are two ways to implement this -- you can either change the calls to extended-env to map box onto the values, or change the code for extended-env itself to always box values when it puts them in a new environment. Take your pick; one approach is as easy as the other.
At
this point your interpreter should be running exactly as it did for MiniSchemeF -- let expressions, lambda expressions and
applications should all work correctly. Make sure this is the case before you
proceed. We will now take advantage of our boxed bindings to implement set!
MiniSchemeG will
implement variable assignment in the form of set! expressions. Note that
we will not be implementing set! as a primitive function, but as an expression
-- in (set! x 5) we don't want to evaluate variable x to its previous value, as
a call would, but rather to store value 5 in its box. .
The grammar for MiniSchemeG will be:
EXP ::= number parse
into lit-exp
| symbol parse
into var-ref
| (if EXP EXP EXP) parse into if-exp
| (let (LET-BINDINGS) EXP) parse into let-exp
| (lambda (PARAMS) EXP) parse into lambda-exp
| (set! symbol EXP) parse into assign-exp
| (EXP EXP*) parse into app-exp
LET-BINDINGS ::= LET-BINDING*
LET-BINDING ::= (symbol EXP)
PARAMS ::= symbol*
We
need to extend eval-exp to handle assign-exp tree nodes. This is just a matter
of putting all of the pieces together: we lookup the
symbol from the expression (the variable being assigned to) in the current
environment; this should give us a box. We call set-box! on this box with the
value we get from recursively calling eval-exp on the expression part of the
assign-exp.
Here
is what we can do when this is implemented:
> (read-eval-print)
MS> (set! + -)
#
MS> (+ 2 2)
0
MS> (set! + (lambda (x y) (- x (minus y))))
#
MS> (+ 2 2)
4
MS> (+ 2 5)
7
MS> exit
returning to Scheme proper
>
Now that we have introduced side effects, it
seems a natural next step to implement sequencing of expressions. We add a
begin expression to the grammar:
EXP
::= number parse into
lit-exp
| symbol parse into var-ref
| (if EXP EXP EXP) parse
into if-exp
| (let (LET-BINDINGS)
EXP) parse into let-exp
| (lambda (PARAMS) EXP) parse into lambda-exp
| (set! symbol EXP) parse
into assign-exp
| (begin EXP*) parse
into begin-exp
| (EXP EXP*) parse into app-exp
LET-BINDINGS ::= LET-BINDING*
LET-BINDING ::= (symbol EXP)
PARAMS ::= symbol*
Evaluating (begin e1 e2 ... en) results in the evaluation of e1, e2, .. en in
that order. The returned result is the last expression, en.
A
begin-exp holds a list of parsed expression. You will need to think about how to add begin-exp to your eval-exp procedures. You need to iterate through the list of expressions in such
a way that
(let ([x 1] [y 2]) (begin (set! x 23) (+
x y)))
returns
25; the whole point of begin is
that the subexpressions might have side effects that alter the environment.
Perhaps this will encourage you to be more appreciative of functional
programming ....
It
looks like we're about done, but let's take a closer look. What happens if we
try to define a recursive procedure in MiniSchemeG?
Let's try the ever-familiar factorial function:
(read-eval-print)
MS> (let ( [fac (lambda (n)
(if
(equals? n 0) 1
(* n (fac (- n 1)))))])
(fac
4))
This
gets an error message saying there is no binding for fac. But we
bound fac using let. Why is MiniScheme
reporting that fac is unbound? The problem is in the recursive call
to fac in (* n (fac (- n 1))). When we evaluated the lambda
expression to create the closure, we did so in an environment in
which fac was not bound. Because procedures use static environments
when they are executed, the recursive call failed. The same thing would happen
in Scheme itself; this is why we have letrec.
Try
this
MS> (let ([fac (lambda (x) (+ x 150))])
(let ([fac (lambda (n)
(if (equals? n
0) 1 (* n (fac (- n 1)))))])
(fac 4)))
This
time the program returns 612, which is 153*4; it is the first binding of
function fac that is seen in the call to (fac (- n 1)
Recall
what happens when a function is created. A closure is created that contains the
environment at the time the function was created, along with the body of the
function and the formal parameters. MiniScheme had no
problems with this, and shouldn't have.
When
a function is called the free variables in the
body are looked up in the environment that was present at the time of the
creation of the function. This is where MiniScheme
ran into problems. In the first example the variable fac in the line
(* n (fac (- n 1))))))
was
not bound to anything at the time the function was created, and so we got an
error. In the second example fac was bound to the earlier procedure
produced by evaluating
(lambda (x) (+ x 150)), which fortunately wasn't recursive. But neither case is
what we want.
There
is a clever way to get around this problem. Try running the following code:
MS> (let ([fac 0])
(let ([f (lambda (n) (if (equals? n 0) 1 (*
n (fac (- n 1)))))])
(begin
(set! fac f)
(fac
4))))
This
works correctly. You can use this pattern for all recursive functions.
So
then, it appears that recursive procedures are really "syntactic sugar;"
we will rewrite letrec-expressions as let-expressions inside
let-expressions with set!s to tie everything
together. Here is the grammar for our final language, MiniSchemeH:
EXP ::=
number parse
into lit-exp
| symbol parse into var-ref
| (if EXP EXP EXP) parse
into if-exp
| (let (LET-BINDINGS) EXP) parse into let-exp
| (lambda (PARAMS) EXP) parse into lambda-exp
| (set! symbol EXP) parse into assign-exp
| (begin EXP*) parse into begin-exp
| (letrec
(LET-BINDINGS) EXP) translate to
equivalent let expression and parse that
| (EXP EXP*) parse into app-exp
LET-BINDINGS ::= LET-BINDING*
LET-BINDING ::= (symbol EXP)
PARAMS ::= symbol*
The
way we are handling letrecs is what is known as
a syntactic transformation. When the parser sees a letrec expression it can either product an equivalent let
expression and parse that, or it can directly create the appropriate let-exp
tree. The latter is what I do, but either approach works.
To implement
MiniSchemeH you should only have to modify the parser.
This means that InterpH is the same as InterpG. Use a helper function (make-letrec ids vals body) to
do the work so that you don't clutter your parser.
You
will need some fresh variables to play the role of placeholders. The
procedure (gensym) always returns a fresh,
unused variable.
> (gensym)
g62
When
you have this completed the following examples
should work.
(read-eval-print)
MS>
(letrec ([fac (lambda (x) (if (equals? x 0) 1 (* x
(fac (sub1 x)))))]) (fac 4))
24
MS>
(letrec ([fac (lambda (x) (if (equals? x 0) 1 (* x
(fac (sub1 x)))))]) (fac 10))
3628800
MS>
(letrec ( [even? (lambda (n) (if (equals? 0 n) True (odd? (sub1 n))))]
[odd? (lambda (n)
(if (equals? 0 n) False (even? (sub1 n))))] )
(even? 5))
False
You are done! Make the final
versions of your modules be env.rkt, parse.rkt, and interp.rkt so the
grader doesn't have to look through all of your code
to figure out where you stopped. Hand in REP.rkt as
well as minischeme.rkt along with your modules.