Using of part-of-speech patterns to automatically construct adjectival scales
Emiel van Miltenburg – www.emielvanmiltenburg.nl
UiL-OTS Universiteit Utrecht
Adjectival scales
Precision
Main Question
Ordered set of adjectives along a certain dimension (Cf. Horn 1972).
hdecent, good, excellenti
I hlukewarm, warm, hot, scaldingi
I hlocal, regional, national, international, globali
Can we use patterns like X, if not Y (warm, if not hot; cold, if not
freezing, etc.) to find pairs of adjectival scalemates?
I If so, can we use those pairs to construct full scales, consisting of
several different adjectives?
I
I
Use:
Improving thesaurus information.
I Information retrieval.
I Dialogue systems, e.g. gauging Indirect yes/no Question-Answer pairs
(de Marneffe et al. 2010): Q: “Was it good?”
A: “It was great!”
I
Searching for scales: previous work
I
Setup
I
hbad,terriblei
hbig,enormousi
hbig,gigantici
hcareful,meticulousi
hcheap,freei
...
Probabilistic modeling (de Marneffe et al. 2010)
I Neural network modeling (Kim & de Marneffe 2013)
Inspiration for this study
→ Outperformed by (2) and (3)
→ Highest recall, lower precision than (3)
→ Highest precision, lower recall than (2)
Lobanova’s promising suggestion (based on Sang & Hofmann 2009):
“Using a larger data set [. . . ] might yield higher precision” (p. 217)
Hearst’s (1992) algorithm
1. Determine the lexical relation R to be investigated.
2. Collect pairs of lexical items that are in this relation to each other.
3. Find and collect sentences/environments in which these pairs occur.
4. Construct patterns based on commonalities between those sentences.
5. Use the new patterns to gather more pairs of lexical items that are in
relation R.
This means that we need a very sensitive method to capture them.
I Are pattern-based methods sensitive enough?
I
Conclusion
More than 3 billion words, over six times the size of the TwNC
I Diverse
I Freely available
Perform study analogous to Lobanova (2012: Ch. 5)
I Except: only select patterns that match two or more seed pairs (so
that we get more general patterns).
I
Score patterns for their likelihood to contain one of the seed pairs.
I Use scores to estimate the probability that found pairs are in R.
I Patterns with a score over a threshold of 0.9 are evaluated.
I
It is possible to automatically find pairs of scalemates in a corpus
using POS-based methods.
I However, such methods have a low precision, even with a huge corpus.
I But: patterns do seem ideal for determining the order of the scale.
I
Results (work in progress)
References
Number of pairs: 587643
Above threshold: 1582 → Manually checked
General impression
I
Results are very noisy: 316 found pairs, 59 doubtful, 1207
non-scalemates
Opposites
Synonyms
Other*
94
103
1069
number of pairs (from doubtful + other)
* Pairs of strongly related adjectives that aren’t opposites or synonyms,
e.g {contemporary, recent}, {irrelevant, redundant}.
Regarding order
Order in which pairs of adjectives occur seems to be a good heuristic
I We can use this order to construct scales from found pairs
I
weak before strong
Lobanova’s contribution: automate step (4) and (5):
maximal contrast
UMBC WebBase corpus (Han et al. 2013) >3 billion words
Lobanova (2012) compares three pattern-based methods to find
antonyms:
1. Textual patterns
2. Part-of-speech patterns
3. Dependency patterns
Antonyms
Scalemates
little/no contrast
I
I
The level of contrast between pairs of scalemates is right in between
that of antonym and synonym pairs.
Synonyms
Manually collected 60 seed pairs (van Tiel et al. 2013).
hadequate,goodi
hallowed,obligatoryi
hangry,furiousi
I
Currently, the precision is only about 20%.
I We can bring it up to 21% by filtering the results, blocking all
identical words, all antonym pairs found in Wordnet (Fellbaum 2010),
and all antonym pairs that can be generated by adding a prefix
(il,in,un,. . . ) to one of the members of a pair.
Problem
I
90
80
70
60
50
40
30
20
10
0
–100%
– 90%
– 80%
– 70%
– 60%
– 50%
– 40%
– 30%
– 20%
– 10%
1122
119
123
179
Fellbaum, Christiane. 2010. Wordnet: An electronic lexical database. 1998. Available from http:
//www.cogsci.princeton.edu/wn.
Han, Lushan, Abhay L. Kashyap, Tim Finin, James Mayfield & Johnathan Weese. 2013. UMBC
EBIQUITY-CORE: Semantic textual similarity systems. In Proceedings of the second joint conference on lexical and computational semantics, Association for Computational Linguistics.
Hearst, Marti A. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of
the 14th conference on computational linguistics-volume 2, 539–545. Association for Computational
Linguistics.
Horn, Laurence R. 1972. On the semantic properties of the logical operators in english: University of
California at Los Angeles dissertation.
Kim, Joo-Kyung & Marie-Catherine de Marneffe. 2013. Deriving adjectival scales from continuous space
word representations. In Proceedings of the 2013 conference on empirical methods in natural language
processing, 1625–1630. Association for Computational Linguistics.
Lobanova, Ganna Volodymyrivna. 2012. The anatomy of antonymy: a corpus-driven approach: Rijksuniversiteit Groningen (RUG) dissertation.
de Marneffe, Marie-Catherine, Christopher D Manning & Christopher Potts. 2010. Was it good? it was
provocative. learning the meaning of scalar adjectives. In Proceedings of the 48th annual meeting of
the association for computational linguistics, 167–176. Association for Computational Linguistics.
Sang, Erik Tjong Kim & Katja Hofmann. 2009. Lexical patterns or dependency patterns: which is
better for hypernym extraction? In Proceedings of the thirteenth conference on computational natural
language learning, 174–182. Association for Computational Linguistics.
van Tiel, Bob, Emiel van Miltenburg, Natalia Zevakhina & Bart Geurts. 2013. Scalar diversity. Paper
presented at XPRAG 2013, Utrecht.
Acknowledgements
319
50
85
49
19
Many thanks to Antal van den Bosch for pointing me to Lobanova’s work, and for
providing initial support, along with Bart Geurts. Alexis Dimitriadis has been very
helpful in providing feedback and suggesting ways to optimize my code.
222
number of patterns