WRITING
A SCANNER
Module 05.5
COP4020 –
Programming Language
Concepts
Dr. Manuel E. Bermudez
TOPICS

Writing a scanner using DFA in
table form.

Writing a hard-coded scanner.

Writing a scanner for real,
i.e. for a programming language.
WRITING A SCANNER
Table-driven
Scanner:
ST
a
b
1
2 3
2
2 4
3
2 3
4
2 5
5
2 3
a
a
b
2
1
Token scan() {
a
Token t=empty();
b
S=1;
3
while not (S in [5] && c = null) {
b
if ST[S,c] = “_”
{ Error(c); c=getchar(); continue; }
else
{S=ST[S,c]; t=t+c; c=getchar(); }
}
return t;
} // assume c is previously read;
4
a
b
b
a
5
WRITING A SCANNER
a
a
1
b
2
4
a
a
b
b
3
b
a
5
b
Hard-coded Scanner:
Token scan() {
Token t=empty();
S=1;
while not (S in [5] && c = null) {
case S of
1: case c of a: S=2; b: S=3; end;
2: case c of b: S=4; end;
3: case c of a: S=2; end;
4: case c of a: S=2; b: S=5; end;
5: case c of a: S=2; b: S=3; end;
end;
t=t+c; c=getchar();
}
return t;
} // assume c is previously read;
WRITING A SCANNER (FOR REAL, I.E. PL’S)
L
L+D+_
Id
S
D
ℇ
<id>
ℇ
<int>
D
Int
No DFA.
Attempt alternatives “in order”.
Each symbol from S goes into
a sub-FSA. Must hard-code.
any-{*}+eol
〳
*
〳
*
ℇ
any
〳
=
ℇ
eol
<Punc:/=>
<Punc:/>
<comm>
<comm>
SUMMARY



Writing a scanner using DFA in
table form.
Writing a hard-coded scanner.
Writing a scanner for real, i.e. for
a programming language

Attempt alternatives in order.