Rhetorical Figure Detection:
the Case of Chiasmus
Marie Dubremetz & Joakim Nivre
Rhetoric workshop - February 2016
Today
Stylometric tools: example
Match string, words, shallow pattern
More? Rhetorical devices? Chiasmus?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
2/24
Today
Stylometric tools: example
Match string, words, shallow pattern
More? Rhetorical devices? Chiasmus?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
2/24
Introduction
Chiasmus/Antimetabole: Traditional Definition
A rhetorical figure in which two words are repeated in reverse order.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
3/24
Introduction
Chiasmus/Antimetabole: Traditional Definition
A rhetorical figure in which two words are repeated in reverse order.
Example
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
3/24
Why?
Help corpus linguistics and rhetorician projects.
Improve our general knowledge of the figure?
If we can make it for chiasmus, you can hope to make it for
more devices.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
4/24
Why?
Help corpus linguistics and rhetorician projects.
Improve our general knowledge of the figure?
If we can make it for chiasmus, you can hope to make it for
more devices.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
4/24
Why?
Help corpus linguistics and rhetorician projects.
Improve our general knowledge of the figure?
If we can make it for chiasmus, you can hope to make it for
more devices.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
4/24
Why?
Help corpus linguistics and rhetorician projects.
Improve our general knowledge of the figure?
If we can make it for chiasmus, you can hope to make it for
more devices.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
4/24
Question
Why does the chiasmus detector not exist yet?
Lack of interest?
Impossible technically?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
5/24
Question
Why does the chiasmus detector not exist yet?
Lack of interest?
Impossible technically?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
5/24
Question
Why does the chiasmus detector not exist yet?
Lack of interest?
Impossible technically?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
5/24
Problem
Traditional definition
"Chiasmus: reversal of two pairs of words"
This is a definition of necessary but not sufficient conditions
In other words
Chiasmus is not a trivial linguistic device. Can a computer detect
it?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
6/24
Problem
Traditional definition
"Chiasmus: reversal of two pairs of words"
This is a definition of necessary but not sufficient conditions
In other words
Chiasmus is not a trivial linguistic device. Can a computer detect
it?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
6/24
State of the Art
The research on chiasmus
Gawryjolek [2009]: Extract every double pair of words with
reverse order without exception
Chuck Norris does not fear death, death fears Chuck Norris
100% recall
Very low precision (< 1%)
Hromada [2011]: Identify not two but three pairs of reverted
words
Love makes time pass, time makes love pass.
Very high precision
But low recall
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
7/24
State of the Art
Problem
There are criss-cross patterns that are not chiasmi such as:
‘I like beer from time to time but I prefer wine’
They are frequent but chiasmi are rare.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
8/24
State of the Art
Problem
There are criss-cross patterns that are not chiasmi such as:
‘I like beer from time to time but I prefer wine’
They are frequent but chiasmi are rare.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
8/24
State of the Art
Problem
There are criss-cross patterns that are not chiasmi such as:
‘I like beer from time to time but I prefer wine’
They are frequent but chiasmi are rare.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
8/24
State of the Art
Problem
There are criss-cross patterns that are not chiasmi such as:
‘I like beer from time to time but I prefer wine’
They are frequent but chiasmi are rare.
This is the problem of the needle in the haystack!
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
8/24
(Re-)Defining the Task
Both pair [2009] and triplet [2011] solutions are extreme...
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
9/24
(Re-)Defining the Task
Both pair [2009] and triplet [2011] solutions are extreme...
...Why not outputting chiasmi in a sorted manner?
Thus the user can choose to read only a couple of pages of results
(instead of thousands!)
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
9/24
How Do We Sort?
Any ranking is based on a score.
How does the computer score a chiasmus?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
10/24
How Do We Rank?
We split the judgement into micro evaluations...
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
11/24
How Do We Rank?
We split the judgement into micro evaluations...
Thus the task becomes computer friendly.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
12/24
How Do We score? An Example of Features
How our algorithm sorts criss-cross patterns:
5 representative examples of our 20 features
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
13/24
How Do We Score? An Example of Features
How our algorithm sorts criss-cross patterns:
5 representative examples of our 20 features
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
14/24
How Do We Score? An Example of Features
How our algorithm sorts criss-cross patterns:
5 representative examples of our 20 features
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
15/24
Our Model
A standard linear model
We encode 4 different types of features:
Basic (stopwords and punctuation)
Size
Ngram
Lexical Features
We give a score to every criss-cross pattern relative to those
features.
This score allows us to rank chiasmi.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
16/24
Evaluation
To evaluate the system we asked a human user to annotate
800 criss-cross patterns
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
17/24
Evaluation
To evaluate the system we asked a human user to annotate
800 criss-cross patterns
- True: Food should be our medicine and medicine our food.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
17/24
Evaluation
To evaluate the system we asked a human user to annotate
800 criss-cross patterns
- True: Food should be our medicine and medicine our food.
- Borderline: If all this is not helped by a fund , the fund is no
help at all.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
17/24
Evaluation
To evaluate the system we asked a human user to annotate
800 criss-cross patterns
- True: Food should be our medicine and medicine our food.
- Borderline: If all this is not helped by a fund , the fund is no
help at all.
- False: A ban is already in place in several Member States,
as several speakers have already mentioned.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
17/24
Evaluation
No ranking system in previous work.
9 candidates only in [2011]
Thus table limited to precision at 9.
Comparison table
Precision
Precision (%)
Our
system
[2015]
7/9
78
3
pairs
[2011]
6/9
67
2
pairs
[2009]
0/9
0
=> Precision comparable to [2011]
=> And improved recall (17/(+-)22)
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
18/24
Evaluation
No ranking system in previous work.
9 candidates only in [2011]
Thus table limited to precision at 9.
Comparison table
Precision
Precision (%)
Our
system
[2015]
7/9
78
3
pairs
[2011]
6/9
67
2
pairs
[2009]
0/9
0
=> Precision comparable to [2011]
=> And improved recall (17/(+-)22)
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
18/24
Evaluation
No ranking system in previous work.
9 candidates only in [2011]
Thus table limited to precision at 9.
Comparison table
Precision
Precision (%)
Our
system
[2015]
7/9
78
3
pairs
[2011]
6/9
67
2
pairs
[2009]
0/9
0
=> Precision comparable to [2011]
=> And improved recall (17/(+-)22)
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
18/24
Future Work
What about syntactic features?
More annotation with more annotators
Apply our method to other devices? Anaphora? Anadiplosis?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
19/24
Contributions
1
A new approach to a rhetorical figure: chiasmus as a graded
phenomenon
2
Pushed the theoretical limits of computers
3
More work for rhetoricians and linguists! Your judgement is
required
and
A system that works on any text file! Demonstration?
Bible, 800 000 words
Sherlock Holmes, 650 000 words
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
20/24
Contributions
1
A new approach to a rhetorical figure: chiasmus as a graded
phenomenon
2
Pushed the theoretical limits of computers
3
More work for rhetoricians and linguists! Your judgement is
required
and
A system that works on any text file! Demonstration?
Bible, 800 000 words
Sherlock Holmes, 650 000 words
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
20/24
Contributions
1
A new approach to a rhetorical figure: chiasmus as a graded
phenomenon
2
Pushed the theoretical limits of computers
3
More work for rhetoricians and linguists! Your judgement is
required
and
A system that works on any text file! Demonstration?
Bible, 800 000 words
Sherlock Holmes, 650 000 words
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
20/24
Contributions
1
A new approach to a rhetorical figure: chiasmus as a graded
phenomenon
2
Pushed the theoretical limits of computers
3
More work for rhetoricians and linguists! Your judgement is
required
and
A system that works on any text file! Demonstration?
Bible, 800 000 words
Sherlock Holmes, 650 000 words
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
20/24
Thank You!
Questions?
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
21/24
Bonus: a Short Chiasmus Viewed by a Computer
All for one, one for all. (In binary.)
01000001 01101100 01101100 00100000
01110010 00100000 01101111 01101110
00100000 01101111 01101110 01100101
01101111 01110010 00100000 01100001
00101110
Marie Dubremetz & Joakim Nivre
01100110
01100101
00100000
01101100
01101111
00101100
01100110
01101100
Automating Chiasmus Detection.
22/24
References
Gawryjolek, J. J. (2009). Automated Annotation and Visualization
of Rhetorical Figures. Master thesis, Universty of Waterloo.
Hromada, D. D. (2011). Initial Experiments with Multilingual
Extraction of Rhetoric Figures by means of PERL-compatible
Regular Expressions. In Proceedings of the Second Student
Research Workshop associated with RANLP 2011, (pp. 85–90).,
Hissar, Bulgaria.
System, corpus, annotation available at:
http://stp.lingfil.uu.se/~marie/chiasme.htm.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
23/24
References
System, corpus, annotation available at:
http://stp.lingfil.uu.se/~marie/chiasme.htm.
Marie Dubremetz & Joakim Nivre
Automating Chiasmus Detection.
24/24