Prepositional Phrase Attachment in
Shallow Parsing
Vincent Van Asch & Walter Daelemans
CLiPS - Computational Linguistics
Contents
•
•
•
•
•
•
•
•
Shallow parsing
Motivation
Task description
Representation of PPs
Corpus extraction
System architecture
Results in-domain and out-domain
Lexical vs. Non-lexical
1
Motivation
- Attempt to extend the MBSP with a
competitive memory based PP attacher
- Machine learning based PP attacher: more
possibilities to adapt to different domains
- MB more flexible: removing lexical features,
adding semantic features
- One step prediction
2
Shallow parser
The rest went to investors from France and Hong Kong.
3
Shallow parser
The rest went to investors from France and Hong Kong.
DT
NN
VBD
TO
NNS
IN
NNP
CC
NNP
NNP
4
Shallow parser
The rest went to investors from France and Hong Kong.
DT
NN
VBD
TO
NNS
IN
NNP
CC
NNP
NNP
5
Task description
The rest went to investors from France and Hong Kong.
6
Task description
The rest went to investors from France and Hong Kong.
Ratnaparkhi (1994):
(went, investors, from, France)
7
Task description
The rest went to investors from France and Hong Kong.
Ratnaparkhi (1994):
(went, investors, from, France)
class: N
8
Task description
The rest went to investors from France and Hong Kong.
Ratnaparkhi (1994):
(went, investors, from, France)
class: N
How do we extend to more than two candidate anchors?
How do we extend to more complicated phrase structures?
9
Simplified representation
Extraction of gold standard attachments from
the WSJ Penn and GENIA treebank
10
Simplified representation
Extraction of gold standard attachments from
the WSJ Penn and GENIA treebank
The rest went to investors from France
11
Simplified representation
Extraction of gold standard attachments from
the WSJ Penn and GENIA treebank
The rest went to investors from France
1
2
3 4
5
6
7
12
Simplified representation
Extraction of gold standard attachments from
the WSJ Penn and GENIA treebank
The rest went to investors from France
1
2
3 4
5
6
7
went-to
investors-from
(3, 4)
(5, 6)
13
Extraction of the corpus
Based on WSJ Penn Treebank and GENIA
14
Description of the corpus
TRAINING:
WSJ Section 2-21: 96000 PP’s
TEST:
WSJ Section 0-1: 8900 PP’s (in-domain)
GENIA:
5600 PP’s (out-domain)
WSJ anchor type distribution
NOUN
50.5%
VERB
45.8%
Other
3.7%
15
System architecture
16
Instances
The rest went to investors from France
17
Instances
The rest went to investors from France
rest-from
18
Instances
The rest went to investors from France
rest-from
went-from
19
Instances
The rest went to investors from France
rest-from
went-from
investors-from
20
Instances
The rest went to investors from France
rest-from
went-from
investors-from
No
No
Yes
21
Class distribution
Skewed data
WSJ training instances
NOUN
6.0%
VERB
5.9%
NONE
88.1%
22
?
Features
The rest went to investors from France and Hong Kong.
Distance:
- anchor-pp order (anchor first)
- anchor-pp distance (1)
- number of commas in between (0)
- number of other punctuation marks (0)
- number of noun phrases in between (0)
- number of prepositional phrases in between (0)
Lexical:
- lemma
- lemma
- lemma
- lemma
- lemma
of
of
of
of
of
Part-of-speech:
- POS tag of anchor (NNS)
- POS tag of noun in prepositional phrase (NNP)
- POS tag of token in front of preposition (NNS)
preposition of anchor (to)
anchor (investor)
preposition (from)
noun in prepositional phrase (France)
token in front of preposition (investor)
23
Baseline
• Rule based system
• Appends to candidate anchor in front of
preposition
• No lexical information used
• Uses POS and chunk information
24
Collins
• Dan Bikel’s implementation
• Output: syntactic trees
• Extraction from anchor-preposition pairs with
the same script used to construct the corpus
25
Experiments
- 4 systems: baseline, Timbl, Maxent, Collins
- 1 training set: WSJ
- 2 test sets: WSJ and GENIA
- Is the ML PP attacher competitive?
- Comparison of ML with full parser?
- How robust are PP attacher systems to domain
shifts?
26
Accuracies
WSJ testset
Baseline
69.85
TiMBL
82.65
Maxent
81.40
GENIA testset
GENIA/WSJ
27
Accuracies
WSJ testset
GENIA testset
Baseline
69.85
69.20
TiMBL
82.65
77.70
Maxent
81.40
77.15
GENIA/WSJ
28
Accuracies
WSJ testset
GENIA testset
GENIA/WSJ
Baseline
69.85
69.20
0.991
TiMBL
82.65
77.70
0.940
Maxent
81.40
77.15
0.948
29
Accuracies
WSJ testset
GENIA testset
GENIA/WSJ
Baseline
69.85
69.20
0.991
TiMBL
82.65
77.70
0.940
Maxent
81.40
77.15
0.948
Collins
83.85
77.80
0.940
30
Lexical vs. Non-lexical ML
WSJ
GENIA
GENIA/WSJ
Baseline
69.85
69.20
0.991
TiMBL with
lemmas
82.65
77.70
0.940
TiMBL w/o
lemmas
78.65
78.20
0.994
31
Conclusions
- All ML systems do equally well and better than
baseline
- The ML PP attachers are competitive when
compared to state-of-the-art full parser
- All systems, except baseline, do worse when
applied to out of domain test data
- Baseline is more robust: independence of
lexical information
32
Future research
- Can we show that a ML PP attacher leads to
more robust results in domain adaptation
experiments?
- Are lexical differences the only influential
differences between domains?
33
TiMBL features
-a 0 :
-m J :
-L 2 :
-w 0 :
-d ID :
-k 8 :
IB1
Jeffrey Divergence
MVDM threshold at level 2
No Weighting
Inverse Distance
8 nearest neighbors
34
MBSP
-
PoS tagging
Chunk finding
Syntactic relation finding
lemmatisation
35