CSCE 771
Natural Language Processing
Lecture 21
Computational Lexical Semantics
Topics



Features in NLTK III
Computational Lexical Semantics
Semantic Web USC
Readings:
 NLTK book Chapter 10
 Text Chapter 20
April 3, 2013
Overview
Last Time (Programming)

Features in NLTK




NL queries  SQL
NLTK support for Interpretations and Models
Propositional and predicate logic support
Prover9
Today




Last Lectures slides 25-29
Features in NLTK
Computational Lexical Semantics
Semantic Web USC
Readings:


Text 19,20
NLTK Book: Chapter 10
Next Time: Computational Lexical Semantics II
–2–
CSCE 771 Spring 2011
Model Building in NLTK - Chapter 10continued
Mace model builder
lp = nltk.LogicParser()
# install Mace4
config_mace4('c:\Python26\Lib\site-packages\prover9')
a3 = lp.parse('exists x.(man(x) & walks(x))')
c1 = lp.parse('mortal(socrates)')
c2 = lp.parse('-mortal(socrates)')
mb = nltk.Mace(5)
print mb.build_model(None, [a3, c1])
True
print mb.build_model(None, [a3, c2])
True
print mb.build_model(None, [c1, c2])
–False
3–
CSCE 771 Spring 2011
>>> a4 = lp.parse('exists y. (woman(y) & all x. (man(x) ->
love(x,y)))')
>>> a5 = lp.parse('man(adam)')
>>> a6 = lp.parse('woman(eve)')
>>> g = lp.parse('love(adam,eve)')
>>> mc = nltk.MaceCommand(g, assumptions=[a4, a5, a6])
>>> mc.build_model()
True
–4–
CSCE 771 Spring 2011
10.4 The Semantics of English
Sentences
Principle of compositionality --
–5–
CSCE 771 Spring 2011
Representing the λ-Calculus in NLTK
(33)
a.
(walk(x) ∧ chew_gum(x))
b.
λx.(walk(x) ∧ chew_gum(x))
c.
\x.(walk(x) & chew_gum(x)) -- the NLTK way!
–6–
CSCE 771 Spring 2011
Lambda0.py
import nltk
from nltk import load_parser
lp = nltk.LogicParser()
e = lp.parse(r'\x.(walk(x) & chew_gum(x))')
print e
\x.(walk(x) & chew_gum(x))
e.free()
print lp.parse(r'\x.(walk(x) & chew_gum(y))')
\x.(walk(x) & chew_gum(y))
–7–
CSCE 771 Spring 2011
Simple β-reductions
>>> e = lp.parse(r'\x.(walk(x) & chew_gum(x))(gerald)')
>>> print e
\x.(walk(x) & chew_gum(x))(gerald)
>>> print e.simplify() [1]
(walk(gerald) & chew_gum(gerald))
–8–
CSCE 771 Spring 2011
Predicate reductions
>>> e3 = lp.parse('\P.exists x.P(x)(\y.see(y, x))')
>>> print e3
(\P.exists x.P(x))(\y.see(y,x))
>>> print e3.simplify()
exists z1.see(z1,x)
–9–
CSCE 771 Spring 2011
Figure 19.7 Inheritance of Properties
Exists e,x,y Eating(e) ^ Agent(e, x) ^ Theme(e, y)
“hamburger edible?” from wordnet
Speech and Language Processing, Second Edition
Daniel Jurafsky and James H. Martin
– 10 –
Copyright ©2009 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
CSCE 771 Spring 2011
Shank’s Conceptual Dependencies
Oversimplified but expandable NLU system
Four Primitive conceptual categories
• ACTs: Actions
• PPs: Objects (picture producers)
• AAs: Modifiers of Actions (action aiders)
• PAs: Modifiers of PPs
– 11 –
CSCE 771 Spring 2011
Figure 19.8 Shank’s Conceptual
Dependency: Primitive Acts
Speech and Language Processing, Second Edition
Daniel Jurafsky and James H. Martin
– 12 –
Copyright ©2009 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
CSCE 771 Spring 2011
Speech and Language Processing, Second Edition
Daniel Jurafsky and James H. Martin
– 13 –
Copyright ©2009 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.
CSCE 771 Spring 2011
Example"Bob told John that his
wedding ring was at the jeweler's. "
(MTRANS
(ACTOR BOB)
(MOBJECT (AT-LOC
(OBJECT WEDDING_RING)
(VALUE JEWELER)))
(FROM BOB)
(TO JOHN)
(TIME PAST))
http://www.cc.gatech.edu/classes/cs3361_96_spring/Fall95/Notes/cd.html
– 14 –
CSCE 771 Spring 2011
– 15 –
CSCE 771 Spring 2011
Example"Bob told John that his
wedding ring was at the jeweler's. "
(MTRANS
(ACTOR BOB)
(MOBJECT (AT-LOC
(OBJECT WEDDING_RING)
(VALUE JEWELER)))
(FROM BOB)
(TO JOHN)
(TIME PAST))
– 16 –
CSCE 771 Spring 2011
The product is moved from shop floor to finished
goods (FG) inventory. Employee receives raw
material from vendor.
.
Artificial Intelligence in EngineeringVolume 15, Issue 2, April 2001, Pages 207-218
– 17 –
CSCE 771 Spring 2011
Classification of arc types
Artificial Intelligence in EngineeringVolume 15, Issue 2, April 2001, Pages 207-218
– 18 –
CSCE 771 Spring 2011
Figure 20.1 Possible sense tags for
bass
Chapter 20 – Word Sense disambiguation (WSD)
Machine translation
Supervised vs unsupervised learning
Semantic concordance – corpus with words tagged
with sense tags
– 19 –
CSCE 771 Spring 2011
Feature Extraction for WSD
Feature vectors
Collocation
[wi-2, POSi-2, wi-1, POSi-1, wi, POSi, wi+1, POSi+1, wi+2, POSi+2]
Bag-of-words – unordered set of neighboring words
Represent sets of most frequent content words with
membership vector
[0,0,1,0,0,0,1] – set of 3rd and 7th most freq. content word
– 20 –
CSCE 771 Spring 2011
Naïve Bayes Classifier
w – word vector
s – sense tag vector
f – feature vector [wi, POSi ] for i=1, …n
^
s  arg max P ( s | f )
sS
Approximate by frequency counts
But how practical?
– 21 –
CSCE 771 Spring 2011
Looking for Practical formula
.
^
s  arg max P( s | f )
sS
P( f | s ) P( s )
 arg max
P( f )
sS
Still not practical
– 22 –
CSCE 771 Spring 2011
Naïve == Assume Independence
n
P( f | s )   P( f j | s )
j 1
Now practical, but realistic?
n
^
s  arg max  P( f j | s)
sS
– 23 –
j 1
CSCE 771 Spring 2011
Training = count frequencies
.
P( si ) 
count ( si , w j )
count ( w j )
Maximum likelihood estimator
P( f j | si ) 
– 24 –
count ( f j , s )
count ( s )
CSCE 771 Spring 2011
Decision List Classifiers
Naïve Bayes hard for humans to examine decisions
and understand
Decision list classifiers - like “case” statement
sequence of (test, returned-sense-tag) pairs
– 25 –
CSCE 771 Spring 2011
Figure 20.2 Decision List Classifier
Rules
– 26 –
CSCE 771 Spring 2011
WSD Evaluation, baselines, ceilings
Extrinsic evaluation - evaluating embedded NLP in endto-end applications (in vivo)
Intrinsic evaluation – WSD evaluating by itself (in vitro)
Sense accuracy
Corpora – SemCor, SENSEVAL, SEMEVAL
Baseline - Most frequent sense
Ceiling – Gold standard – human experts with
discussion and agreement
– 27 –
CSCE 771 Spring 2011
Figure 20.3 Simplified Lesk Algorithm
– 28 –
CSCE 771 Spring 2011
Simplified Lesk example
The bank can guarantee deposits will eventually cover
future tuition costs because it invests in adjustable
rate mortgage securities.
– 29 –
CSCE 771 Spring 2011
– 30 –
CSCE 771 Spring 2011
Figure 20.4
– 31 –
CSCE 771 Spring 2011
Figure 20.5
– 32 –
CSCE 771 Spring 2011
Figure 20.6
– 33 –
CSCE 771 Spring 2011
Figure 20.7
– 34 –
CSCE 771 Spring 2011
Figure 20.8
– 35 –
CSCE 771 Spring 2011
Figure 20.9
– 36 –
CSCE 771 Spring 2011
Figure 20.10
– 37 –
CSCE 771 Spring 2011
Figure 20.11
– 38 –
CSCE 771 Spring 2011
Figure 20.12
– 39 –
CSCE 771 Spring 2011
Figure 20.13
– 40 –
CSCE 771 Spring 2011
Figure 20.14
– 41 –
CSCE 771 Spring 2011
Figure 20.15
– 42 –
CSCE 771 Spring 2011
Figure 20.16
– 43 –
CSCE 771 Spring 2011