Answering
Questions
Grzegorz Chrupala
Language in the Digital Age
Grzegorz CHRUPALA (TiU)
QA
1 / 54
What is the capital of Syria?
Grzegorz CHRUPALA (TiU)
QA
2 / 54
What is the capital of Syria?
Grzegorz CHRUPALA (TiU)
QA
2 / 54
Who discovered penicillin?
Grzegorz CHRUPALA (TiU)
QA
3 / 54
Who discovered penicillin?
Grzegorz CHRUPALA (TiU)
QA
3 / 54
Who is Angela Merkel married to?
Grzegorz CHRUPALA (TiU)
QA
4 / 54
Who is Angela Merkel married to?
Grzegorz CHRUPALA (TiU)
QA
4 / 54
What is the meaning of life?
Grzegorz CHRUPALA (TiU)
QA
5 / 54
What is the meaning of life?
42
Grzegorz CHRUPALA (TiU)
QA
5 / 54
Who discovered penicillin?
Grzegorz CHRUPALA (TiU)
QA
6 / 54
Who discovered penicillin?
What if we just look for passages
which mention penicillin.
Grzegorz CHRUPALA (TiU)
QA
6 / 54
Penicillin (sometimes abbreviated PCN or
pen) is a group of antibiotics derived from
Penicillium fungi.
Grzegorz CHRUPALA (TiU)
QA
7 / 54
Penicillin (sometimes abbreviated PCN or
pen) is a group of antibiotics derived from
Penicillium fungi.
Penicillin is one of the first and still one of
the most widely used antibiotic agents derived from the Penicillium mold.
Grzegorz CHRUPALA (TiU)
QA
7 / 54
Penicillin (sometimes abbreviated PCN or
pen) is a group of antibiotics derived from
Penicillium fungi.
Penicillin is one of the first and still one of
the most widely used antibiotic agents derived from the Penicillium mold.
Penicillin is used to treat infections caused
by bacteria, such as ear infections.
Grzegorz CHRUPALA (TiU)
QA
7 / 54
Penicillin (sometimes abbreviated PCN or
pen) is a group of antibiotics derived from
Penicillium fungi.
Penicillin is one of the first and still one of
the most widely used antibiotic agents derived from the Penicillium mold.
Penicillin is used to treat infections caused
by bacteria, such as ear infections.
Penicillins are a certain collection of
antibiotics that eliminate infection causing
bacteria.
Grzegorz CHRUPALA (TiU)
QA
7 / 54
Not relevant
Grzegorz CHRUPALA (TiU)
QA
8 / 54
What if we look for
passages which
mention penicillin
and discover.
Grzegorz CHRUPALA (TiU)
QA
9 / 54
Fleming was the first to discover the
properties of the active substance, giving
him the privilege of naming it: penicillin
Grzegorz CHRUPALA (TiU)
QA
10 / 54
Fleming was the first to discover the
properties of the active substance, giving
him the privilege of naming it: penicillin
Howard Florey did not discover penicillin.
Grzegorz CHRUPALA (TiU)
QA
10 / 54
Fleming was the first to discover the
properties of the active substance, giving
him the privilege of naming it: penicillin
Howard Florey did not discover penicillin.
It could be argued here that Fleming did
accidentally discover penicillin since he
wasn’t looking for it at that moment in
time.
Grzegorz CHRUPALA (TiU)
QA
10 / 54
Fleming was the first to discover the
properties of the active substance, giving
him the privilege of naming it: penicillin
Howard Florey did not discover penicillin.
It could be argued here that Fleming did
accidentally discover penicillin since he
wasn’t looking for it at that moment in
time.
Did Ian Fleming discover penicillin?
Grzegorz CHRUPALA (TiU)
QA
10 / 54
We’re getting
somewhere.
Grzegorz CHRUPALA (TiU)
QA
11 / 54
We’re getting
somewhere.
Can we already answer the
question?
Grzegorz CHRUPALA (TiU)
QA
11 / 54
KISS approach
Recognize mentions of PEOPLE.
Grzegorz CHRUPALA (TiU)
QA
12 / 54
KISS approach
Recognize mentions of PEOPLE.
Count how many times each mention appears.
Grzegorz CHRUPALA (TiU)
QA
12 / 54
KISS approach
Recognize mentions of PEOPLE.
Count how many times each mention appears.
Return most frequent.
Grzegorz CHRUPALA (TiU)
QA
12 / 54
Or we could try to understand
the question even better.
Grzegorz CHRUPALA (TiU)
QA
13 / 54
Or we could try to understand
the question even better.
How?
Grzegorz CHRUPALA (TiU)
QA
13 / 54
Who discovered penicillin?
Grzegorz CHRUPALA (TiU)
QA
14 / 54
Expected answer type
Grzegorz CHRUPALA (TiU)
QA
15 / 54
Expected answer type
Who
discovered penicillin?
PERSON
Grzegorz CHRUPALA (TiU)
QA
15 / 54
Who/what the question is about
Who discovered
Grzegorz CHRUPALA (TiU)
QA
penicillin?
SUBSTANCE
16 / 54
Relation expressed in the question
Who discovered
penicillin?
X-DISCOVER-Y
Grzegorz CHRUPALA (TiU)
QA
17 / 54
Who
discovered
penicillin?
PERSON X-DISCOVER-Y SUBSTANCE
Grzegorz CHRUPALA (TiU)
QA
18 / 54
Who discovered penicillin?
Grzegorz CHRUPALA (TiU)
QA
19 / 54
Who discovered penicillin?
Retrieve passages which
Grzegorz CHRUPALA (TiU)
QA
19 / 54
Who discovered penicillin?
Retrieve passages which
Talk about the SUBSTANCE penicillin
Grzegorz CHRUPALA (TiU)
QA
19 / 54
Who discovered penicillin?
Retrieve passages which
Talk about the SUBSTANCE penicillin
Express relation X-DISCOVER-Y
Grzegorz CHRUPALA (TiU)
QA
19 / 54
Who discovered penicillin?
Retrieve passages which
Talk about the SUBSTANCE penicillin
Express relation X-DISCOVER-Y
Mention a PERSON
Grzegorz CHRUPALA (TiU)
QA
19 / 54
The laboratory in which Fleming discovered
and tested penicillin is preserved as the
Alexander Fleming Laboratory Museum in
St. Mary’s Hospital, Paddington.
Grzegorz CHRUPALA (TiU)
QA
20 / 54
How about these.
In contrast, Fleming isolated the substance
penicillin from the mould Penicillium
notatum.
The discovery of penicillin is attributed to
Scottish scientist and Nobel laureate
Alexander Fleming in 1928.
Duchesne cured typhoid, but the penicillin
isolated by Fleming does not cure typhoid.
Grzegorz CHRUPALA (TiU)
QA
21 / 54
Variations
In contrast, Fleming isolated the substance
penicillin from the mould Penicillium
notatum.
The discovery of penicillin is attributed to
Scottish scientist and Nobel laureate
Alexander Fleming in 1928.
Duchesne cured typhoid, but the penicillin
isolated by Fleming does not cure typhoid.
Grzegorz CHRUPALA (TiU)
QA
22 / 54
Lexical variation
Grzegorz CHRUPALA (TiU)
QA
23 / 54
Morphology
discover discovering discovered
discoverer discovery rediscover
penicillin penicillins
Grzegorz CHRUPALA (TiU)
QA
24 / 54
Morphology
discover discovering discovered
discoverer discovery rediscover
penicillin penicillins
Grzegorz CHRUPALA (TiU)
QA
24 / 54
Synonymy
discover find
isolate invent develop originate
penicillin PCN pen
Penicilium mold agent
Grzegorz CHRUPALA (TiU)
QA
25 / 54
Syntactic variation
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Y, which X discovered, ...
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Y, which X discovered, ...
Y, which was discovered by X, ...
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Y, which X discovered, ...
Y, which was discovered by X, ...
X was the discovered of Y
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Y, which X discovered, ...
Y, which was discovered by X, ...
X was the discovered of Y
X, the discoverer of Y, ...
Grzegorz CHRUPALA (TiU)
QA
26 / 54
Syntactic variation
X discovered Y
Y was discovered by X
discovery of Y by X
X, who discovered Y, ...
Y, which X discovered, ...
Y, which was discovered by X, ...
X was the discovered of Y
X, the discoverer of Y, ...
Y discoverer X
Grzegorz CHRUPALA (TiU)
QA
26 / 54
X isolated Y
Y was found by X
isolation of Y by X
X, who invented Y, ...
Y, which X found, ...
Y, which was originated by X, ...
X was the isolated of Y
X, the inventor of Y, ...
Y inventor X
Grzegorz CHRUPALA (TiU)
QA
27 / 54
We need generalization
Grzegorz CHRUPALA (TiU)
QA
28 / 54
How can we generalize over
lexical variation?
Grzegorz CHRUPALA (TiU)
QA
29 / 54
Discover words with similar
meanings
Grzegorz CHRUPALA (TiU)
QA
30 / 54
Discover words with similar
meanings
Words with similar meanings
occur in similar contexts
Grzegorz CHRUPALA (TiU)
QA
30 / 54
Distributional
semantics
Grzegorz CHRUPALA (TiU)
QA
31 / 54
Penicillin
antibiotics
causes
symptoms
fungi
allergies
wonder
fungal
Fleming
Grzegorz CHRUPALA (TiU)
QA
32 / 54
Penicillin
antibiotics
causes
symptoms
fungi
allergies
wonder
fungal
Fleming
Grzegorz CHRUPALA (TiU)
PCN
antibiotics
fungi
allergies
dosage
lactamase
drugs
bacteria
medication
QA
32 / 54
Penicillin PCN
antibiotics
10
15
fungi
8
3
causes
7
1
allergies
7
7
symptoms
5
1
fungal
3
1
bacteria
3
5
...
...
...
astronaut
0.01
0.0
Grzegorz CHRUPALA (TiU)
QA
33 / 54
Word classes
Grzegorz CHRUPALA (TiU)
QA
34 / 54
Word classes
Clustering
Grzegorz CHRUPALA (TiU)
QA
34 / 54
Flat clustering
Grzegorz CHRUPALA (TiU)
QA
35 / 54
Flat clustering
Berlin Bangkok Tokyo Warsaw
Grzegorz CHRUPALA (TiU)
QA
35 / 54
Flat clustering
Berlin Bangkok Tokyo Warsaw
Sarkozy Merkel Obama Berlusconi
Grzegorz CHRUPALA (TiU)
QA
35 / 54
Flat clustering
Berlin Bangkok Tokyo Warsaw
Sarkozy Merkel Obama Berlusconi
Mr Ms President Dr
Grzegorz CHRUPALA (TiU)
QA
35 / 54
Agglomerative clustering algorithm
Grzegorz CHRUPALA (TiU)
QA
36 / 54
Agglomerative clustering algorithm
Given a way to measure similarity between
clusters of words:
Grzegorz CHRUPALA (TiU)
QA
36 / 54
Agglomerative clustering algorithm
Given a way to measure similarity between
clusters of words:
1 Put each word into a separate cluster.
Grzegorz CHRUPALA (TiU)
QA
36 / 54
Agglomerative clustering algorithm
Given a way to measure similarity between
clusters of words:
1 Put each word into a separate cluster.
2 Look for two most similar clusters and
merge them.
Grzegorz CHRUPALA (TiU)
QA
36 / 54
Agglomerative clustering algorithm
Given a way to measure similarity between
clusters of words:
1 Put each word into a separate cluster.
2 Look for two most similar clusters and
merge them.
3 If there is only one cluster, stop.
Grzegorz CHRUPALA (TiU)
QA
36 / 54
Agglomerative clustering algorithm
Given a way to measure similarity between
clusters of words:
1 Put each word into a separate cluster.
2 Look for two most similar clusters and
merge them.
3 If there is only one cluster, stop.
Othewise, go to step 2.
Grzegorz CHRUPALA (TiU)
QA
36 / 54
those these that it he she Obama Merkel Sarkozy
Grzegorz CHRUPALA (TiU)
QA
37 / 54
A possible result
Grzegorz CHRUPALA (TiU)
QA
38 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Grzegorz CHRUPALA (TiU)
QA
39 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Retrieve passages containing
Grzegorz CHRUPALA (TiU)
QA
39 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Retrieve passages containing
{discover, find, invent, isolate, ...}
Grzegorz CHRUPALA (TiU)
QA
39 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Retrieve passages containing
{discover, find, invent, isolate, ...}
and
Grzegorz CHRUPALA (TiU)
QA
39 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Retrieve passages containing
{discover, find, invent, isolate, ...}
and
{PCN, penicillin, Penicilium mold agent,
Beta-lactamase inhibitor, ...}
Grzegorz CHRUPALA (TiU)
QA
39 / 54
After clustering, penicillin and PCN will
hopefully end up in the same cluster.
Retrieve passages containing
{discover, find, invent, isolate, ...}
and
{PCN, penicillin, Penicilium mold agent,
Beta-lactamase inhibitor, ...}
and mentioning a PERSON.
Grzegorz CHRUPALA (TiU)
QA
39 / 54
Fleming discovered penicillin but Duchesne
cured typhoid.
Do we want Fleming or Duchesne?
Grzegorz CHRUPALA (TiU)
QA
40 / 54
Answer extraction
Grzegorz CHRUPALA (TiU)
QA
41 / 54
Syntactic analysis
Grzegorz CHRUPALA (TiU)
QA
42 / 54
Syntactic analysis
aka Parsing
Grzegorz CHRUPALA (TiU)
QA
42 / 54
Look for the PERSON who is a subject of the
verb discover whose object is penicillin.
Grzegorz CHRUPALA (TiU)
QA
43 / 54
dobj
subj
|
Who
Grzegorz CHRUPALA (TiU)
discovered
QA
#
penicillin
44 / 54
dobj
subj
|
Who
#
discovered
dobj
subj
{
Fleming
Grzegorz CHRUPALA (TiU)
penicillin
discovered
QA
#
penicillin
44 / 54
dobj
subj
|
Who
#
discovered
dobj
subj
{
Fleming
penicillin
#
discovered
subj
|
Duchesne
Grzegorz CHRUPALA (TiU)
penicillin
dobj
cured
QA
!
typhoid
44 / 54
Do we need to worry about
syntactic variability?
Grzegorz CHRUPALA (TiU)
QA
45 / 54
Do we need to worry about
syntactic variability?
To some degree
Grzegorz CHRUPALA (TiU)
QA
45 / 54
Duchesne cured typhoid, but the penicillin
discovered by Fleming does not cure typhoid.
Grzegorz CHRUPALA (TiU)
QA
46 / 54
Duchesne cured typhoid, but the penicillin
discovered by Fleming does not cure typhoid.
How do we choose between Duchesne and
Fleming?
Grzegorz CHRUPALA (TiU)
QA
46 / 54
dobj
subj
|
Who
Grzegorz CHRUPALA (TiU)
discovered
QA
#
penicillin
47 / 54
dobj
subj
|
Who
#
discovered
dobj
subj
|
Duchesne
Grzegorz CHRUPALA (TiU)
penicillin
cured
QA
!
typhoid
47 / 54
dobj
subj
|
Who
#
discovered
penicillin
dobj
subj
|
Duchesne
!
cured
subj
~
penicillin
Grzegorz CHRUPALA (TiU)
typhoid
pp
discovered
QA
pobj
by
Fleming
47 / 54
Solution?
We need generalization over
syntactic contructions.
Grzegorz CHRUPALA (TiU)
QA
48 / 54
We could apply clustering techniques
similar to those for clustering words.
Grzegorz CHRUPALA (TiU)
QA
49 / 54
X discovered Y Y discovered by X
X=scientist
15
14
X=Columbus
10
11
Y=America
9
10
Y=antibiotic
8
8
X=Tombaugh
4
5
Y=Pluto
3
2
Y=radium
1
1
X=Curie
1
1
...
...
...
Grzegorz CHRUPALA (TiU)
QA
50 / 54
dobj
subj
|
Who
#
discovered
penicillin
dobj
subj
|
Duchesne
!
cured
pp
subj
~
penicillin
Grzegorz CHRUPALA (TiU)
typhoid
discovered
QA
pobj
by
Fleming
51 / 54
Figure out Expected Answer Type.
Grzegorz CHRUPALA (TiU)
QA
52 / 54
Figure out Expected Answer Type.
Identify entities of this type in answers.
Grzegorz CHRUPALA (TiU)
QA
52 / 54
Figure out Expected Answer Type.
Identify entities of this type in answers.
Identify relations and slots while abstracting
from lexical and syntactic variability.
Grzegorz CHRUPALA (TiU)
QA
52 / 54
Figure out Expected Answer Type.
Identify entities of this type in answers.
Identify relations and slots while abstracting
from lexical and syntactic variability.
Extract answer by matching question
structure to passage.
Grzegorz CHRUPALA (TiU)
QA
52 / 54
Playful Jepardy questions
Grzegorz CHRUPALA (TiU)
QA
53 / 54
Playful Jepardy questions
“A long winded argument delivered by a frothy
pie topping.”
Grzegorz CHRUPALA (TiU)
QA
53 / 54
Playful Jepardy questions
“A long winded argument delivered by a frothy
pie topping.”
“Meringue Harangue.”
Grzegorz CHRUPALA (TiU)
QA
53 / 54
Finding opinions
Grzegorz CHRUPALA (TiU)
QA
54 / 54
Finding opinions
What do people think about iPhone.
Grzegorz CHRUPALA (TiU)
QA
54 / 54
Finding opinions
What do people think about iPhone.
iPhone is getting faster year after year!
Bugger all the iPhone-wielding Apple
fanbois...
i soooo want the new gold iphone!!
Grzegorz CHRUPALA (TiU)
QA
54 / 54