Introduction
The DMV+CCM Model
Our work on DMV+CCM
Ten (Or More) Minutes on
Unsupervised Parsing
Franco M. Luque
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
Outline
Introduction
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
The DMV+CCM Model
CCM
DMV
DMV+CCM
Our work on DMV+CCM
Using Punctuation
Possible Future Work
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
The Problem of Parsing
Obtain the syntactic structure of a sentence. The two most
common linguistic structures are:
I
Constituent trees:
I
Dependency structures:
I
These usually include word categories, called POS tags.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
The Problem of Supervised Parsing
Supervised Parsing is the problem of building a parser using a
treebank.
I
Treebanks are corpuses of parsed sentences.
I
A part of the treebank is used to train the parser.
I
Another part is used to evaluate the parser.
It can be seen as learning a function f : X → Y (the parser) by
knowing some points (x1 , f (x1 )), . . . , (xn , f (xn )) (the treebank).
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
The Problem of Unsupervised Parsing
What can we do by only knowing a set of points {x1 , . . . , xn } of
the domain of f ?
I
The parser is trained with sentences.
I
The evaluation is still done against a treebank.
I
The set of syntactic categories is unknown.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Parsing
Supervised Parsing
Unsupervised Parsing
Parser Evaluation
Parser Evaluation
Gold Tree:
Proposed Tree:
I
Precision: proportion of constituents that are right (4/5).
I
Recall: proportion of right constituents that are found (4/5).
I
F1: Harmonic mean between Precision and Recall.
There are several variants of this measures.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
The DMV+CCM Model
I
Developed by Dan Klein and Chris Manning in 2004.
I
Parses dependency trees that can be converted to binary
bracketings.
I
Learn and parse from POS tags instead of words. Must be
combined with a POS tagger to obtain a real parser.
I
Evaluated on English, German and Chinese treebanks, only
with sentences of length ≤ 10.
I
Punctuation is not considered in order to emulate spoken
language.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
Constituents and Contexts
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
The Constituent Context Model
I
Parses binary bracketings (unlabeled binary trees).
I
It is a generative model (defines P(s, B) in terms of P(s|B)):
P(s, B) = Pbin (B)P(s|B)
I
Each span i sj and context si−1 , sj is generated with probability
conditioned on the boolean Bij :
Y
P(s|B) =
Pspan (i sj |Bij )Pctx (si−1 , sj |Bij )
i,j:i≤j
I
Obviously, this is mass deficient.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
CCM Training
I
The parameters of the model are the values of Pspan and Pctx .
I
The model is trained with a corpus S of sentences (of POS
tags).
I
Iterative Expectation
Maximization is used to maximize the
Q
likelihood s∈S P(s).
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
The Dependency Model with Valence
I
Parses dependency trees that can
be converted to binary bracketings.
I
The dependency structure is
generated from the sentence in a
head-outwards procedure.
I
A word takes dependents until it
decides to stop, doing this to each
side independently.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
DMV Training
I
Each decision in the procedure has
a probability. These are the
parameters of the model.
I
The probability P(s, D) is the
product of the probabilities of the
decisions taken to build D.
I
Iterative Expectation Maximization
is
Qused to maximize the likelihood
s∈S P(s).
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
CCM
DMV
DMV+CCM
The CCM and DMV Models Combined
I
Parses the same structures as
DMV.
I
The generation procedure includes
the CCM factor.
Results:
I
The strengths of both models are
complementary.
I
Combined with a tagger,
performance drops, but not below
CCM’s.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Using Punctuation
Possible Future Work
Why Use Punctuation?
I
Punctuation gives syntactic information that must be used.
I
This information is more important as sentences are longer.
I
These claims have been verified in other models for
Unsupervised Parsing (Seginer 2007).
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Using Punctuation
Possible Future Work
How Can We Use Punctuation?
I
Opening/closing punctuation determines brackets with prob.
1 (e.g. parenthesis and quotes).
I
From individual punctuation there are no language
independent rules. It is modeled into DMV parameters and
it’s behaviour is learnt.
I
This way we could improve DMV+CCM performance from
77.6 to 78.4.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing
Introduction
The DMV+CCM Model
Our work on DMV+CCM
Using Punctuation
Possible Future Work
Possible Future Work
I
Combine unsupervised POS tagging and unsupervised parsing.
I
Develop a dependency parser with a faster training algorithm.
I
Parse longer sentences.
I
Study syntactic category induction.
Franco M. Luque
Ten (Or More) Minutes on Unsupervised Parsing