Brain Advance Access published March 31, 2015
doi:10.1093/brain/awv084
BRAIN 2015: Page 1 of 3
| e1
LETTER TO THE EDITOR
Differential functions of ventral and dorsal striatum
Yashar Zeighami1 and Ahmed A. Moustafa2
1 McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill University, 3801 University Street, Montreal, QC,
Canada
2 School of Social Sciences and Psychology and Marcs Institute for Brain and Behaviour, University of Western Sydney, Sydney, New
South Wales, Australia
Correspondence to: Dr Ahmed Moustafa,
School of Social Sciences and Psychology and Marcs Institute for Brain and Behaviour,
University of Western Sydney,
Sydney, NSW,
Australia
E-mail: [email protected]
Vo et al. also analysed learning and performance processes using computational models. Regression modelling
of current choice based on the previous choice and the
history of rewards revealed the following: in action-value
learning tasks, both Patient XG and healthy controls
demonstrated normal patterns of reinforcement learning
(i.e. exponential decline in weight of past trials). In contrast, Patient XG (compared to controls) did not show this
pattern in stimulus-value tasks. In fact, the regression
weights were not significantly different from zero, which
indicated lack of learning. Consistent with this, a three-parameter reinforcement learning model explained choice behaviour better than the null model for controls in both
tasks. For Patient XG, the model explained choice behaviour better for the action-value tasks, but could not explain
the behaviour in stimulus-value tasks.
These results address two different aspects of reinforcement learning: (i) the anatomical dissociation of the ventral
striatum and dorsal striatum in learning; and (ii) the behavioural dissociation between stimulus-value versus actionvalue learning. These two aspects are addressed separately
hereafter.
Studies investigating functional segregation of the striatum suggest that the ventral striatum is more involved in
goal-directed learning, whereas the dorsal striatum is more
involved in action related habitual learning (Balleine et al.,
2007; Balleine and O’Doherty, 2010; Redgrave et al.,
ß The Author (2015). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved.
For Permissions, please email: [email protected]
Downloaded from by guest on April 7, 2015
Sir,
The case study by Vo and colleagues (2014) aims to address the differential roles of ventral versus dorsal striatum
in learning, specifically, whether they are essential for learning or simply involved in it. The authors reported a dissociation between action-value (based on the outcomes,
values will be assigned to actions) and stimulus-value learning (values will be associated with the stimuli), and how
impairment of the dorsal striatum will affect each of these
processes.
To achieve this, Vo and colleagues tested a patient
(known as XG) who has bilateral damage to the dorsal
striatum, while the ventral striatum including the nucleus
accumbens is spared. To compare XG’s performance in
different tasks with a healthy population statistically, the
researchers tested 11 matched control subjects. Among
the seven reinforcement learning tasks employed, three
could be solved only by learning stimulus-values, one
only by learning action-values, and the remaining
tasks, with either strategy. Surprisingly, they found that
Patient XG was able to learn all the tasks involving
action-value learning and his performance resembled
those of healthy controls. However, he was impaired at
learning the tasks which could only be learned using
stimulus values; his performance in those tasks was significantly poorer than controls and was no different
from random.
e2
| BRAIN 2015: Page 2 of 3
the potential alterations in cortico-striatal connections in
Patient XG may elaborate on this knowledge gap.
Various theoretical models suggesting dissociation between stimulus- and action-value learning have been proposed. However, existing models contradict findings
reported by Vo et al. For example, Piray and colleagues
(2014) provided evidence for dissociation in patients with
Parkinson’s disease ON/OFF dopamine medication, those
with impulse control disorders and controls using an
Actor/Critic model. Their findings suggest that the ventral
striatum plays a role in stimulus-value learning while the
dorsal striatum in action-value learning whereas Vo et al.
suggest dorsal striatum being necessary for stimulus-value
learning process and not involved in action-value learning.
Differences in findings might be due to the structure of
the task used in the study by Vo and colleagues. In actionvalue learning tasks, Patient XG was presented with a
single state (trial type) whereas the stimulus-value learning
tasks consisted of multiple states. Impairment in the latter
task may therefore reflect the involvement of the dorsal
striatum in state representation, and its interaction with
working memory and not necessarily in learning stimulusvalue learning. Future study designs should introduce multiple state action-learning tasks and examine whether the
impairment occurs in state representation or stimulus-value
learning.
References
Balleine BW, Delgado MR, Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci 2007; 27: 8161–5.
Balleine BW, O’Doherty JP. Human and rodent homologies in action
control: corticostriatal determinants of goal-directed and habitual
action. Neuropsychopharmacology 2010; 35: 48–69.
Camille N, Tsuchida A, Fellows LK. Double dissociation of stimulusvalue and action-value learning in humans with orbitofrontal or
anterior cingulate cortex damage. J Neurosci 2011; 31: 15048–52.
Everitt BJ, Robbins TW. From the ventral to the dorsal striatum:
devolving views of their roles in drug addiction. Neurosci
Biobehav Rev 2013; 37 (9 Pt A): 1946–54.
Garrison J, Erdeniz B, Done J. Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies. Neurosci Biobehav
Rev 2013; 37: 1297–310.
Haber SN, Calzavara R. The cortico-basal ganglia integrative network:
the role of the thalamus. Brain Res Bull 2009; 78: 69–74.
Joel D, Niv Y, Ruppin E. Actor-critic models of the basal ganglia: new
anatomical and computational perspectives. Neural Netw 2002; 15:
535–47.
Niv Y. Reinforcement learning in the brain. J Math Psychol 2009; 53:
139–54.
O’Doherty , Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ.
Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science (New York, NY) 2004; 304: 452–4.
Piray P, Zeighami Y, Bahrami F, Eissa AM, Hewedi DH,
Moustafa AA. Impulse control disorders in Parkinson’s disease are
associated with dysfunction in stimulus valuation but not action
valuation. J Neurosci 2014; 34: 7814–24.
Porrino LJ, Lyons D, Smith HR, Daunais JB, Nader MA. Cocaine selfadministration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J Neurosci 2004; 24:
3554–62.
Downloaded from by guest on April 7, 2015
2010); this is consistent with findings in anatomical connectivity of the striatum. Moving in the ventral-dorsal axis,
the cortico-striatal connectivity changes from limbic/cognitive areas (ventromedial prefrontal cortex, orbitofrontal
cortex) towards associative (dorsal anterior cingulate
cortex, dorsal prefrontal cortex) and finally motor areas
(supplementary motor area) (Postuma and Dagher, 2006;
Haber and Calzavara, 2009). These theoretical findings
have been validated in rodent addiction studies where
goal-directed behaviour subserved by the ventral striatum
shifts towards habitual behaviour by the dorsal striatum as
addiction progresses (Porrino et al., 2004; Everitt and
Robbins, 2013). These findings can explain Patient XG’s
spared performance in tasks involving action-value learning, but fails to address the impaired stimulus-value
learning.
Stimulus- and action-value learning dissociations have
also been studied in cortical areas. Orbitofrontal cortex
lesions disrupt stimulus-value but not action-value learning,
whereas dorsal anterior cingulate lesions show the opposite
effects (Camille et al., 2011). Based on these findings and
the previously discussed anatomical connections between
dorsal striatum and dorsal anterior cingulate cortex,
dorsal striatum impairment in Patient XG should result in
action-value learning impairment while stimulus-value
learning remains intact, i.e. the exact opposite of what
was observed by Vo et al. Using functional and structural
connectivity analysis and functional MRI during task performance in Patient XG can enhance our knowledge about
the cortico-striatal connectivity and the brain structures
involved in each learning process and clarify the source
of this contradiction.
In the striatum, stimulus- and action-value learning dissociation has been studied using conditioning paradigms
including Pavlovian (stimulus-outcome association) and
instrumental (stimulus-action-outcome association) as well
as Actor/Critic models. The Actor/Critic model of the striatum suggests that ventral striatum plays a similar role to
the ‘critic’ by calculating the prediction errors, and dorsal
striatum is similar to the ‘actor’ that is involved in action
selection (Joel et al., 2002; Niv, 2009). Consistent with
these results, O’Doherty et al. (2004) found that dorsal
striatum is involved in instrumental conditioning while ventral striatum is involved in both instrumental and Pavlovian
conditioning. Schonberg and colleagues also showed a
selective impairment of prediction error-related activity in
dorsolateral but not ventral striatum in patients with
Parkinson’s disease, confirming dissociation between the
two areas (Schonberg et al., 2010). In contrast, a metaanalysis by Garrison and colleagues found that the ventral
striatum was chiefly responsible for instrumental condition
specific activity, whereas mainly cortical areas were identified in Pavlovian condition (Garrison et al., 2013). These
findings suggest that the ventral striatum may be responsible for intact action-value learning but does not explain
the impaired stimulus-value learning in Patient XG suffering from bilateral dorsal striatum damage. Investigations of
Letter to the Editor
Letter to the Editor
Postuma RB, Dagher A. Basal ganglia functional connectivity based on
a meta-analysis of 126 positron emission tomography and functional
magnetic resonance imaging publications. Cereb Cortex 2006; 16:
1508–21.
Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S,
Bergman H, et al. Goal-directed and habitual control in the basal
ganglia: implications for Parkinson’s disease. Nat Rev 2010; 11:
760–72.
BRAIN 2015: Page 3 of 3
| e3
Schonberg T, O’Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND.
Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson’s disease patients: evidence from a model-based fMRI study. Neuroimage 2010; 49:
772–81.
Vo K, Rutledge RB, Chatterjee A, Kable JW. Dorsal striatum is necessary for stimulus-value but not action-value learning in humans.
Brain 2014; 137 (Pt 12): 3129–35.
Downloaded from by guest on April 7, 2015