A Short History of Two-Level
Morphology
Lauri Karttunen, Xerox PARC
Kenneth R. Beesley, XRCE
Lauri Karttunen / 24 Aug 2001 / page 2
Overview
• Introduction
What is morphology?
Two strains of finite-state morphology
State of the art circa 1980.
• Two-Level Morphology
Origins, basic idea
Implementations, compilers
• Recent Developments
Lauri Karttunen / 24 Aug 2001 / page 3
What is Morphology?
• Morphosyntax
Words are composed of smalled units of meaning called
morphemes that must be combined in a certain order.
piti-less-ness vs. *piti-ness-less
• Morphological Alternations
The shape of a morpheme depends on its environment.
piti-less vs *pity-less
Lauri Karttunen / 24 Aug 2001 / page 4
Sequential Model
Lexical form
fst 1
Intermediate form
fst 2
...
fst n
Surface form
Ordered sequence
of rewrite rules
(Chomsky & Halle ‘68)
can be modeled
by a cascade of
finite-state transducers
Johnson ‘72
Kaplan & Kay ‘81
Lauri Karttunen / 24 Aug 2001 / page 5
Parallel Model
Lexical form
fst 1
fst 2
...
Surface form
Set of parallel
of two-level rules
compiled into finite-state automata
interpreted as transducers
Koskenniemi ‘83
fst n
Lauri Karttunen / 24 Aug 2001 / page 6
Sequential vs. Parallel
Lexical form
Lexical form
fst 1
fst 1
fst 2
...
fst n
Intermediate form
fst 2
Surface form
...
fst n
Surface form
FST
Perhaps too large
to be practical.
Lauri Karttunen / 24 Aug 2001 / page 7
State of the Art circa 1980
• Cut-and-paste analysis
leaves --> leave --> leav --> leaf
ad-hoc programs, not reversible for generation
• Paradigm tables
comprendre 45
not reversible for analysis, impractical for
morphologically complex languages
• Chomsky-Halle rewrite rules
x -> y / z _ w
computationally complex, no implementation,
reversible?
Lauri Karttunen / 24 Aug 2001 / page 8
Discovery and Rediscovery
• C. Douglas Johnson (1972) showed that
– phonological rewrite rules are interpreted in a way
that makes them less powerful than they appear
– rewrite rules can be modeled by finite transducers
– for any two finite transducers applied in a sequence
there exists an equivalent single transducer
(Schützenberger 1961).
• Johnson’s result was ignored and forgotten,
rediscovered by Ronald M. Kaplan and Martin
Kay at Xerox around 1980.
Lauri Karttunen / 24 Aug 2001 / page 9
Sequential Application
k a N p a n
N -> m / _ p
k a m p a n
p -> m / m _
k a m m a n
Lauri Karttunen / 24 Aug 2001 / page 10
Sequential Application in Detail
k a N p a n
N:m
p
m
0 0 0 2 0 0 0
?
N:m
m
0
?
p
k a m p a n
1
N
0 0 0 1 0 0 0
N
m
p
k a m m a n
2
?
0
?
p:m
1
m
Lauri Karttunen / 24 Aug 2001 / page 11
Composition
N:m
3
p:m
k a N p a n
0 0 0 3 0 0 0
k a m m a n
N:m
m
?
0
2
?
p:m
p
N
?
m
N:m
N
m
1
N
Lauri Karttunen / 24 Aug 2001 / page 12
Building a Compiler
• Requires a finite-state calculus
concatenation, union, intersection, complementation...
• Constraints are regular languages
“if p occurs then q follows”
. . . p. . . . q. . . .
?* p
?* q ?*
~[ ?* p ~[ ?* q ?* ]]
• The idea of double negation was Kaplan and
Kay’s first insight. Many details remained to be
worked out.
Lauri Karttunen / 24 Aug 2001 / page 13
The Problem of “Overanalysis”
k a N p a n
k a m p a n
k a m p a n
k a m m a n
k a m m a n
Lauri Karttunen / 24 Aug 2001 / page 14
The Birth of Two-Level Morpholgy
• In the spring of 1981 Kimmo Koskenniemi came
to UT at Austin in search of a dissertation topic.
• Karttunen demoed his TEXFIN
analyzer/generator for Finnish.
• Kaplan and Kay briefed him about their
discoveries. Koskenniemi visited PARC.
• After a gestation period of about a year, twolevel morphology was born.
Lauri Karttunen / 24 Aug 2001 / page 15
The Three Ideas of Two-Level
Morphology
• Rules are symbol-to-symbol constraints that are
applied in parallel, not sequentially like rewrite
rules.
• The constraints can refer to the lexical context,
to the surface context or to both contexts at the
same time.
• Lexical lookup and morphological analysis are
performed in tandem.
Lauri Karttunen / 24 Aug 2001 / page 16
Two-Level Constraints 1
k a N p a n
k a N p a n
k a m m a n
k a m m a n
N:m correspondence requires a
following p on the lexical
side.
p:m correspondence requires a
preceding m on the surface
side.
In this context, all other
possible realization of a
lexical p are prohibited.
In this context, all other
possible realization of a
lexical N are prohibited.
Lauri Karttunen / 24 Aug 2001 / page 17
Two-Level Constraints 2
s p y 0 + s
s p y 0 + s
s p i e 0 s
s p i e 0 s
y:i <=> _ 0:e
0:e <=> Cons: y: _ +: s:
Lauri Karttunen / 24 Aug 2001 / page 18
Parallel Application
k
a
N
p
N:m
Rule
k
a
m
p:m
Rule
m
a
n
Lauri Karttunen / 24 Aug 2001 / page 19
Lookup and Analysis in Tandem
k
a
p
N
N:m
Rule
k
a
m
p:m
Rule
m
a
n
Lauri Karttunen / 24 Aug 2001 / page 20
Two-Level Implementations
•
•
•
•
•
•
•
•
1982 Koskenniemi (Pascal)
1983 Karttunen et al. at UTexas (Lisp)
1986- Antworth et al. at SIL (C)
1987 Black et al. Alvey Project (Lisp)
1989 Beesley Alpnet (Lisp)
1991 Pulman et al. ALEP (Prolog)
1995 Carter SRI CLE (Prolog)
1995 Petitpierre et al. MULTEXT (C)
Lauri Karttunen / 24 Aug 2001 / page 21
Two-Level Rule Compilers
• 1985 Kaplan and Koskenniemi: the basic compilation
algorithm developed during Koskenniemi’s visit at CSLI at
Stanford on a Dandelion (Xerox Lisp machine). It was
based on the techniques Kaplan and Kay had developed
for compiling rewrite-rules.
• 1985-87 Koskenniemi and Karttunen: the first compiler
• 1992 Current C version (twolc) by Karttunen and Beesley.
• 1996 Grimley-Evans, Kiraz, Pulman: compiler for a
“partition-based” two-level formalism
Lauri Karttunen / 24 Aug 2001 / page 22
Seeds of Dissatisfaction
• Two-level morphological analyzers became a
standard component in natural language
processing systems.
• But there was no publicly available compiler
until recently.
• Morphotactics was “improved” by adding
feature unification.Two-level analyzers acquired
a reputation for being slow.
• Two-level rules are notoriously difficult to write,
even with a compiler.
Lauri Karttunen / 24 Aug 2001 / page 23
Rule Conflicts
k:0
General rule
k:v
Exception
Vowel _ Vowel
makun
ma un
pukun
puvun
u _ u
Resolution by underspecification:
k:0 | k:v <=> Vowel _ Vowel
k:v
<=>
u _ u
Lauri Karttunen / 24 Aug 2001 / page 24
Recent Developments
• The pioneers of finite-state morphology new
that a cascade of transducers or a set of
parallel rules could be combined into a single
transducer.
• But the resulting single transducer is typically
huge compared to the size of the original rule
networks. Impractical in most cases.
• The obvious solution, not seen for a long time,
was to compose the rules with the lexicon.
Lauri Karttunen / 24 Aug 2001 / page 25
Lexical Transducer
Source
Lexicon
composition
Lexical
Transducer
o
R1 & R2
& ...
Rn
Karttunen, Kaplan, Zaenen 1992
intersection
canonical form
s
p
y
0
s
e
p
i
inflected form
inflection codes
+Noun +PL
0
s
Lauri Karttunen / 24 Aug 2001 / page 26
Cascade of Compositions
Ci
composition
o
Source
Lexicon
o
replace
rule
constraint
R1
o
Cj
o
...
Rn
Lexical
Transducer
Lauri Karttunen / 24 Aug 2001 / page 27
Linguistic Issues
• The idea of rules as parallel constraints was not
picked up by mainstream linguists in the 80’s.
• Many arguments had been advanced to show
that phonological alternations could not be
described or explained without sequential
rewrite rules.
• The two-level model was perceived as a
computational “hack”, not worthy of academic
interest.
Lauri Karttunen / 24 Aug 2001 / page 28
Rise of Optimality Theory
• Optimality Theory, the dominant paradigm in
phonology since 1993 is a two-level model with
parallel constraints.
• Most optimality constraints can be encoded
trivially as two-level rules.
• The main difference is that OT constraints are
ranked and violable.
Lauri Karttunen / 24 Aug 2001 / page 29
Back to the Big Picture
Lexical form
Lexical form
fst 1
fst 1
fst 2
...
fst n
Intermediate form
fst 2
...
fst n
Surface form
Surface form
While the sequential model was
popular among mainstream linguists,
computational linguists preferred the
parallel model. Now it is almost
the other way round, although for
computational linguists there is no
substantive difference.