Mind Reading - Can Imaging Tell What You Are Thinking?
Andreas Kleinschmidt
INSERM U992
CEA NeuroSpin
Saclay, France
Cognitive
Neuroscience
In Humans –
Many Modalities
One Imaging Modality –
Many Signals
Trait vs. State - Structural and Functional Abnormalities
in Adolescents with Conduct Disorder
0,2
grey matter concentration
Voxel-based morphometry
in 12 adolescents
with conduct disorder
compared with 12 controls
r = -5.4
0,15
p = 0.006
0,1
0,05
0
-0,05
-0,1
-0,15
(Sterzer et al., NeuroImage 2007)
-0,2
40
60
80
100
aggression score
A
y = 36
size of effect
Different responses
to affective stimuli
x=9
B
3
2
1
0
-1
-2
-3
-4
-5
neutral
negative
controls
(Sterzer et al., Biol. Psychiatry 2005, Stadler et al., J Psych Res 2006)
C
patients
Reversing the Direction of Inference –
From Brain Mapping to Mind Reading
(Dehaene et al., Nature Neuroscience 1998)
From Simple Actions to
Cognitive Categories
From Single Voxels or
Regions of Interest to
Multi-Voxel Patterns
Classification
(Decoding)
(Norman et al., 2006)
From categories to exemplars and invariance
=
from Op de Beeck et al., 2008
Training and Testing
the Classifier (SVM)
(Eger et al. J Neurophysiol, 2008)
≠
Decoding Images From Visual Cortex Activity
(Miyawiki et al., Neuron 2008)
Decoding: From Individual Natural Images
to Individual Art Painters
(Yamamura et al., Neuroreport 2009)
From Actions and Images to Thoughts
Thoughts are forms created in the mind,
rather than the forms perceived through
the five senses. Thought and thinking are the
processes by which these imaginary sense
perceptions arise and are manipulated.
Cognitive psychology:
Mental processes which mediate between
stimulus and response, including the
psychology of reasoning, and how people
make decisions and choices, solve problems,
as well as engage in creative discovery and
imaginative thought.
Developmental psychology (Piaget):
Thought evolves from being based on perceptions and actions at the
sensorimotor stage to internal representations. Subsequently, representations
are gradually organized into logical structures which first operate on the
concrete properties of the reality, in the stage of concrete operations, and then
operate on abstract principles that organize concrete properties, in the stage of
formal operations.
Decoding Untrained Items
(Mitchell et al., Science 2008)
Measuring signals from individual numerosities
Delayed number comparison task:
Sample stimulus 200 ms
+
Delay period 4-7 s
high-res fMRI
1.5 mm voxels
at 3T
Match stimulus 200 ms
-> response: smaller or larger number?
(50 % difference)
Sample numerosities: 4 8 16 32
either item size, or overall luminance matched between sample numerosities
Supra-modal representation of individual numbers
Eger et al., Curr Biol 2009
From Classification
of Noisy Evoked
Response Patterns
To Predictions
from Noise
(Norman et al., 2006)
Spontaneous local variations in ongoing pre-stimulus
neural activity bias perceptual decisions
(Hesselmann et al., PNAS, 2008)
20 – 40s static (ISI)
right hMT+
coherent
random
355ms motion
**
*
[46 ± 4, -69 ± 6, 2 ± 3]
t
z=3
100x
(Hesselmann et al., J Neurosci, 2008)
The Brain builds a Predictive Model of the World
(Friston, Trends Cogn Sci, 2009)
Thinking allows beings to model the world and to represent it
according to their objectives, plans, intentions and desires.
(Wikipedia)
Brain Imaging and Its Impact on Society
What are
we thinking?
Who has the
right to know?
Cooperation
Consent