Maximilian Köper &
Sabine Schulte im Walde
Improving Verb Metaphor Detection
by Propagating Abstractness
to Words, Phrases and Individual Senses
Overview
1
In this talk ...
Abstractness and Concreteness
Metaphor detection using abstractness
Learning abstractness norms for:
Single words (comparison of approaches)
Phrases
Individual senses
M. Köper: Improving Verb Metaphor Detection by ...
2
In this talk ...
Abstractness and Concreteness
Metaphor detection using abstractness
Learning abstractness norms for:
Single words (comparison of approaches)
Phrases
Individual senses
M. Köper: Improving Verb Metaphor Detection by ...
2
Abstractness & Concreteness
Concrete words refer to things, events, and
properties that we can perceive directly with our
senses
Abstract: anxiety, happiness, democracy,
opportunity
Concrete: flower, stone, cinnamon, wall ...
M. Köper: Improving Verb Metaphor Detection by ...
3
Obtaining Ratings
be
lie
lo f
gi
c
sc
he
m
fit e
ne
ss
gr
ee
n
sp
ag
he
tt
i
Dataset, created by manual annotation
Concrete
Abstract
word
spaghetti
green
fitness
scheme
logic
belief
Rating
5.0
4.0
3.1
2.4
1.7
1.1
Ignacio Iacobacci
al (ACL
2016).
Brysbaert
et etal.
(2014)
: 40 000 words
MRC Psycholinguistic Database: ≈8 200 words
Extension of manually collected ratings with
supervised learning techniques.
Ignacio Iacobacci
al (ACL 2016).
Turney
et al.et (2011)
: 114 501 words
M. Köper: Improving Verb Metaphor Detection by ...
4
Metaphor Detection
Example 1:
The detective digs up enough evidence about the crime.
Example 2:
The dog digs up a small bone in our garden.
Features
Rating of the verbs subject
Rating of the verbs object
Average rating of all nouns
Average rating of all proper names
Average rating of all verbs, excluding the target verb
Average rating of all adjectives
Average rating of all adverbs
M. Köper: Improving Verb Metaphor Detection by ...
5
Metaphor Detection
Example 1:
The detective digs up enough evidence about the crime.
Example 2:
The dog digs up a small bone in our garden.
Features
Rating of the verbs subject
Rating of the verbs object
Average rating of all nouns
Average rating of all proper names
Average rating of all verbs, excluding the target verb
Average rating of all adjectives
Average rating of all adverbs
M. Köper: Improving Verb Metaphor Detection by ...
5
Metaphor Detection
Example 1 = metaphorical
(C)
4.1
target
(A)
1.2
(A)
2.3
The detective digs up enough evidence about the crime.
Example 2 = literal
(C) target
4.8
(C)
4.7
(C)
4.1
The dog digs up a small bone in our garden.
M. Köper: Improving Verb Metaphor Detection by ...
5
Learning Abstractness Overview
Propagate abstractness
w~
1 :
w~
2 :
w~
3 :
w~
4 :
wnew
~
:
4.2
0.5
8.1
0.0
5.1
5.5
1.0
2.3
6.3
8.4
9.1
3.1
0.2
1.3
2.2
3.3
4.7
5.1
2.3
1.2
...
...
...
...
...
learn
predict
Rating
5
2
3
8
?
Propagate abstractness to all words for which we
have vector space representation
Compare supervised methods:
Ignacio Iacobacci
al (ACL 2016). (2003) (T&L 03).
Algorithm from Turney
&etLittman
Ignacio Iacobacci et
Linear regression (L-Reg), used by Tsvetkov
etal (ACL
al.2016).
(2014)
Regression forest (Reg-F)
Feed forward neural network (NN)
Experiment: comparison of methods
M. Köper: Improving Verb Metaphor Detection by ...
6
Learning Abstractness Results
Glove50
Glove100
Glove200
Glove300
W2V300
T&L 03
L-Reg.
Reg-F.
NN
.76
.80
.78
.76
.83
.76
.79
.78
.78
.84
.78
.79
.76
.74
.79
.79
.85
.84
.85
.90
Split Brysbaert ratings into 20% test, 80% train
(small part of train used for tuning)
Results show Spearman’s ρ against human ratings
M. Köper: Improving Verb Metaphor Detection by ...
7
Learning Abstractness Results
Glove50
Glove100
Glove200
Glove300
W2V300
T&L 03
L-Reg.
Reg-F.
NN
.76
.80
.78
.76
.83
.76
.79
.78
.78
.84
.78
.79
.76
.74
.79
.79
.85
.84
.85
.90
Split Brysbaert ratings into 20% test, 80% train
(small part of train used for tuning)
Results show Spearman’s ρ against human ratings
Applied best setup to create a resource of
automatically created ratings for 3.000.000 English
words and multi-words (available!)
M. Köper: Improving Verb Metaphor Detection by ...
7
Abstractness
for Individual
Senses
2
Abstractness & Metaphor Detection
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
9
Abstractness & Metaphor Detection
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
9
Abstractness & Metaphor Detection
(A)
1.9
(C/A)
4.5
(C)
7.0
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
9
Abstractness & Metaphor Detection
laziness
(A)
(A)
1.9
(C)
(C)
7.0
“Today we’re tackling sloth”, said the student.
Words are ambiguous
fan – supporter/ventilator
outlet – socket/escape
latex – rubber latex/LATEX
Assumption: Sense specific ratings are more accurate
→ improve token-based verb metaphor classification
M. Köper: Improving Verb Metaphor Detection by ...
9
Learning Multi-Sense Embeddings
Many approaches for learning sense embeddings
Ignacio Iacobacci
et al(2016)
(ACL 2016).
Pelevina
et al.
“Making Sense of Word Embeddings”
Learn single sense embeddings (word2vec)
Cluster based on WordSimilarity Graph
Pooling of Word Vectors = multi sense
embeddings
M. Köper: Improving Verb Metaphor Detection by ...
10
Learning Multi-Sense Embeddings
Many approaches for learning sense embeddings
Ignacio Iacobacci
et al(2016)
(ACL 2016).
Pelevina
et al.
“Making Sense of Word Embeddings”
Learn single sense embeddings (word2vec)
Cluster based on WordSimilarity Graph
Pooling of Word Vectors = multi sense
embeddings
Apply previous setting:
W2V300 vectors→ Multi-Sense + Brysbaert ratings
+ NN method
Sense embeddings “live” in the same space as single
sense embeddings
M. Köper: Improving Verb Metaphor Detection by ...
10
Multi-Sense Embeddings & Sense Specific Abstractness Ratings
Compare each sense embedding with each context
word and pick the one with the highest cosine
similarity
Example
laziness
(A)
(C)
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
11
Multi-Sense Embeddings & Sense Specific Abstractness Ratings
Compare each sense embedding with each context
word and pick the one with the highest cosine
similarity
Example sloth sense-1, rating: 6.7 (C)
laziness
(A)
(C)
cosine similarity
sense-1: context
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
11
Multi-Sense Embeddings & Sense Specific Abstractness Ratings
Compare each sense embedding with each context
word and pick the one with the highest cosine
similarity
Example sloth sense-2, rating: 0.4 (A)
laziness
(A)
(C)
cosine similarity
sense-2: context
“Today we’re tackling sloth”, said the student.
M. Köper: Improving Verb Metaphor Detection by ...
11
Multi-Sense Embeddings & Sense Specific Abstractness Ratings
Compare each sense embedding with each context
word and pick the one with the highest cosine
similarity
VU Amsterdam Metaphor Corpus (VUA) 2 174 verbs
and 23 113 sentences
TroFi metaphor dataset 50 verbs 3 737 labeled
sentences
10-fold Cross Validation
M. Köper: Improving Verb Metaphor Detection by ...
11
Results
Feat.
TroFi(10F)
VUA(10F)
1S
MS
.72
.74
.42
.44*
F1 Metaphor Class
Classifier: (balanced) Logistic Regression
Ratings obtain state of the art results
Ignacio Iacobacci
2016).
(VUAtest ) Klebanov
et etal.al (ACL
(2016)
Effect vanishes when adding target verb itself as feature
M. Köper: Improving Verb Metaphor Detection by ...
12
Maximilian Köper
Institut für Maschinelle Sprachverarbeitung (IMS), Universität Stu
eMail
Phone
[email protected]
+49 711 685-81355