Language and Cognition
Center of Excellence
Cognitive Interaction Technology
Universität Bielefeld
Distinguishing Cognitive Models of Spatial Language Understanding
1
Kluth ,
Thomas
Michele
1
Burigo ,
Holger
2
Schultheis ,
Pia
3
Knoeferle
1: CITEC (Cognitive Interaction Technology Excellence Cluster), Bielefeld University, Bielefeld, Germany
2: Cognitive Systems Group, Department of Computer Science, University of Bremen, Bremen, Germany
3: Department of German Language and Linguistics, Humboldt-University Berlin, Berlin, Germany.
Research Question
Introduction
• two models computing acceptability ratings of spatial prepositions:
– Attentional Vector Sum model (AVS, Regier & Carlson, 2001):
attention shifts from the RO to the LO (Logan & Sadler, 1996)
– reversed AVS model (rAVS, Kluth, Burigo, & Knoeferle, 2016):
attention shifts from the LO to the RO (Burigo & Knoeferle, 2015;
Roth & Franconeri, 2012)
• equal performance on the data from Regier and Carlson (2001, see
Kluth et al., 2016)
The circle is above the rectangle.
located object
(LO)
reference object
(RO)
AVS model
(Regier & Carlson, 2001)
reversed AVS (rAVS) model
How to distinguish the two models?
• different predictions • a more flexible model
for specific stimuli
is harder to falsify
→ empirical study to → investigate flexibiltest these predictions
ity of models
(Kluth, Burigo, & Knoeferle, 2016)
LO 2
LO
tor
vec
• deviation δ from canonical downwards of one vector pointing
from the LO to point between proximal point P and centerof-mass C of the RO.
• the greater the relative distance, the more the vector points
|LO,P |y
|LO,P |x
towards C, relative distance = ROwidth + ROheight
δ
sum
LO 1
δ2
δ1
RO
P
height
D1
(plus “height component” not shown here)
* no vector sum (because of simplified LO)
* attentional distribution is computationally irrelevant
D2 C
Stimuli & Predictions
width
h
◦2 · h
×
◦4 · h
×
Empirical Results
Parameter Space Partitioning
◦6 · h
×
rel. volume of parameter space (%)
◦
×
mean difference
relative distance effect, 'über' ratings
rAVS prediction: higher ratings for LOs above taller rectangles
0.05
0.00
-0.05
-0.10
-0.15
-0.20
square
tall rect.
center-of-mass e ect, 'über' ratings
d d
mean di erence
unclear prediction
d d
thick rect.
thin rect. minus ...
AVS prediction: ?
d d
100
0.10
d d
0.0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
-0.7
-0.8
-0.9
C
mC
L
mL
all
rAVS prediction: 0
no difference
AVS prediction: +
higher rating for LOs above mass
× = center-of-mass; ◦ = center-of-object
up to 28 LOs above each RO
References
Burigo, M., & Knoeferle, P. (2015). Visual attention during spatial language comprehension.
PLoS ONE , 10 (1), e0115758. doi: 10.1371/journal.pone.0115758
Kluth, T., Burigo, M., & Knoeferle, P. (2016). Shifts of attention during spatial language
comprehension: A computational investigation. In Proceedings of the 8th International Conference on Agents and Artificial Intelligence – Volume 2 (pp. 213–222).
SCITEPRESS. doi: 10.5220/0005851202130222
Kluth, T., Burigo, M., Schultheis, H., & Knoeferle, P. (2016). The role of the center-of-mass
in evaluating spatial language. In Proceedings of the 13th Biannual Conference of
the German Society for Cognitive Science.
Logan, G. D., & Sadler, D. D. (1996). A computational analysis of the apprehension of
spatial relations. In P. Bloom, M. A. Peterson, L. Nadel, & M. F. Garrett (Eds.),
Language and Space (pp. 493–530). The MIT Press.
Navarro, D. J., Pitt, M. A., & Myung, I. J. (2004). Assessing the distinguishability of
models and the informativeness of data. Cognitive Psychology , 49 (1), 47–84. doi:
10.1016/j.cogpsych.2003.11.001
Pitt, M. A., Kim, W., Navarro, D. J., & Myung, J. I. (2006). Global model analysis by
parameter space partitioning. Psychological Review , 113 (1), 57–83. doi: 10.1037/
0033-295X.113.1.57
Regier, T., & Carlson, L. A. (2001). Grounding spatial language in perception: An empirical
and computational investigation. Journal of Experimental Psychology: General,
130 (2), 273–298. doi: 10.1037//0096-3445.130.2.273
Roth, J. C., & Franconeri, S. L. (2012). Asymmetric coding of categorical spatial relations
in both language and vision. Frontiers in Psychology , 3 (464). doi: 10.3389/
fpsyg.2012.00464
Veksler, V. D., Myers, C. W., & Gluck, K. A. (2015). Model flexibility analysis. Psychological
Review , 122 (4), 755–769. doi: 10.1037/a0039657
Landscaping
(Navarro, Pitt, & Myung, 2004)
AVS generated data
0.22
0.18
0.14
0.10
0.06
60
20
0
• Are the models able to generate “intuitive” predictions?
rAVS: -0; AVS: ?+
• equality of ratings = 0.5
40
rAVS
model
AVS
Model Flexibility Analysis
rAVS generated data
rAVS-fit (nRMSE)
◦×
rAVS-fit (nRMSE)
×◦
×
◦
80
→ PSP confirms intuitive predictions for rAVS but not for AVS
RO
×◦
-0-0
00
(Pitt et al., 2006)
(Veksler, Myers, & Gluck, 2015)
stimuli used in
0.22
0.18
0.14
0.10
0.06
0.0 0.1 0.1 0.1 0.2
6 0 4 8 2
0.0 0.1 0.1 0.1 0.2
6 0 4 8 2
AVS-fit (nRMSE)
AVS-fit (nRMSE)
× = fits to 1000 artificial data sets;
= fits to empirical data
AVS model
rAVS model
present study
Regier and Carlson (2001)
φ = 0.000899
φ = 0.000544
φ = 0.000420
φ = 0.000292
The lower the φ value, the less flexible the model.
Discussion
• PSP: mechanisms of AVS harder to translate into testable predictions
• relative distance:
– prediction of rAVS model empirically confirmed
• asymmetrical ROs:
– people seem to rely on the center-of-object instead of on the center-of-mass of the RO
→ disconfirms both models
– work in progress to implement this suggestion into both models (Kluth, Burigo, Schultheis, & Knoeferle, 2016)
• contrasting results for model flexibility:
– PSP & MFA: AVS is more flexible than rAVS
– landscaping: none of the models mimics the other model
ICCM 2016, August 4-6, State College, Pennsylvania, USA