“Examining the relationship between virtual
reality driving and cognitive demands of driving
after brain injury”
Maria T. Schultheis, PhD.1,2
Emily Roseman, MA2
Jose Rebimbas3
Ronald Mourant, PhD4
Scott R. Millis, PhD5
1
2
Drexel University, Department. of Psychology
Drexel University, School of Biomedical Engineering, Science and Health Systems
3141 Chestnut Street, Philadelphia, PA 19104
3
Kessler Medical Rehabilitation Research & Education Corp.
1199 Pleasant Valley Way, West Orange, NJ
4
Northeastern University, Department of Mechanical Engineering
5
Rehabilitation Institute of Michigan
Abstract
Objective: This study examined the relationship between driving performance measures
generated by a clinical virtual reality driving simulator (VRDS) and driving performance
determined by a) a behind-the-wheel driving (BTW) evaluation and b) cognitive tasks relevant to
driving.
Methods: This study was conducted at an outpatient research setting and included twenty
eight individuals: 18 with acquired brain injury (BI) and 10 healthy controls (HC). Individuals
underwent cognitive testing, participated in the VRDS, and some received a BTW evaluation.
Driving performance on the BTW was examined both at the global level (i.e., pass/fail) and
specific level (i.e., rating on individual behaviors) VRDS variables were consistent with BTW
measures and included both specific behaviors and three global measures: 1) VRDS lane
management, 2) VRDS speed control and 3) VRDS turning/tracking. Cognitive tests specific to
driving performance included: TMT, Visual Cancellation Test, PASAT, Block Design.
Results: Bivariate correlations between VRDS measures and cognitive tests revealed
significant relationships. However, after a Hommel correction, only VRDS ‘lane management’
was significantly correlated with a measure of working memory (PASAT; p=.025). Due to the
small sample size, only an exploratory analyses was conducted examining the relationship
between VRDS and BTW measures. Bivariate correlations revealed no significant relationship
between the global VR and BTW measures, though significant relationships existed between the
VR and BTW subcomponents. VR speed control was significantly related to BTW speed control
(p<.05). Additionally, the BTW ‘road law’ subcomponent was significantly (p<.05) related to the
VR lane deviation, head turning standard deviation, average distance from stop sign, and
deceleration.
Discussion: Although exploratory the current findings may be suggestive of the possible
sensitivity that is offered by VR driving performance measures. Although additional research is
needed, the use of VR driving simulators may offer clinicians a new method to better define and
assess the components of driving following brain injury.
Résumé
DSC 2007 North America – Iowa City – September 2007
Background
1. Brain Injury
Each year in the United States, 1.4 million individuals sustain a traumatic brain injury
(Langlois,Rutland-Brown, & Thomas, 2006). With improvements in emergency medicine
and technology, the number of individuals living with a history of brain injury is
increasing (Langlois et al., 2006). Estimates suggest that there are approximately 5.3
million brain injury survivors (Thurman, Alverson, Dunn, Guerrero, & Sniezek, 1999).
The sequelae of brain injury (BI) can be enduring and can impact occupational and social
functioning. (National Institutes of Health,1998). Cognitive deficits, including impaired
attention, learning, memory, executive functioning, and language and communication are
often noted (McCullagh & Feinstein, 2005). Many BI survivors hope to return to premorbid levels of functioning, including returning to driving.
The age of peak incidence traumatic brain injury is 15 to 24 years. The risk of brain
injury in men is twice the risk in women, and the risk is higher in adolescents, young
adults and people older than 75 years.
1.2 BI and Driving
The operation of a motor vehicle is a functional behaviour (or, rather, a complete
repertoire of behaviours). It is a complex, involving myriad cognitive, sensory, and
behavioural components. Because driving requires the integration of these multiple
components, it is often an ability that becomes difficult for individuals with BI. Yet, the
ability to operate a motor vehicle is a means of maintaining independence and facilitating
community reintegration among individuals with BI (Formisano et al., 2005; Rapport,
Hanks, & Bryer, 2006; Schultheis, Matheis, Nead et al 2002) Indeed among, individuals
with BI, the cessation of driving has been related to difficulties in employment (DevaneySerio & Devens, 1994), higher incidence of depression, (Marottoli et al., 1997;LeghSmith et al., 1986) poor social integration and inability to engage in activities outside of
the home (Dawson & Chipman, 1995).
Given the negative impact that the inability to drive can have on everyday functioning, it
is not surprising that many survivors of BI persistently seek to regain their driving
privileges after their injuries. In many cases, persons with BI continue to drive despite
clinical recommendations (Leon-Carion, Domingues-Morales, & Martin, 2005).
Specifically, studies that have examined return to driving, estimate that approximately
38% to 80% of persons with acquired brain injury return to driving (Fisk, Schneider, &
Novack, 1998; Schultheis, et al,. 2002). Of those individuals that resume driving, the
majority report they drive daily (Fisk et al., 1998; Schultheis et al., 2002). Additionally,
return-to-driving status for persons with BI does not appear to be limited to minor or mild
injury, as it been reported that 60% of moderate to severe TBI survivors return to driving
(Fisk et al., 1998).
Yet, despite the importance of accurately determining the ability to return to driving,
current clinical tools are few and often fraught with limitations. In fact, the current “gold
standard” in driver rehabilitation, is the behind-the–wheel (BTW) evaluation, which
suffers from over-reliance on subjective observations and is a non-standardized procedure
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
which lacks evidence for validity and reliability. Other common tools, including driving
records and the use of neuropsychological tests are also limited in their ability to
accurately predict an individual’s performance on the road.
2. Virtual Reality Driving Simulation
Virtual reality (VR) has been promoted as a new method for evaluating and treating
individuals with brain injury, as it is able to objectively examine individual factors that
constitute a behaviour as it occurs in time (Schultheis & Rizzo, 2001). Specifically, VR
can allow the evaluation of the various elements of a functional behaviour in order to
assess where an individual is having difficulty. In the area of driver rehabilitation, VR
driving simulation offers a variety of unique assets that could address the limitations of
current clinical tools (Schultheis and Mourant, 2001). As such, providing clinicians with
new objective, detailed and standardized measurements of driving ability and
subsequently offering new information to help guide both driver assessment and retraining objectives.
2.1 Clinical VR driving simulators
It is important to first, clarify our working definition of “clinical VR driving simulators”.
Specifically, this does not refer to high-end simulators that have been used with clinical
populations and are often inaccessible to clinicians. Rather, we refer to a VR driving
simulator that requires low-end, affordable technology that is useful and usable by
clinicians. A system that preferably runs on a commercially available platform and does
not require extensive space or specialized personnel. The goal is a VR driving simulator
that is more informative (than traditional measures) but not as complex as high end
simulators (e.g. full vehicles).
To date, there are only a handful of studies that have examined the use of clinically
accessible VR driving simulators with individuals with brain injury. Most of these have
focused on determining VR driving simulations ability to discriminate between those
with and without brain injury, on basic driving performance measures (Wald, Liu,
Hirsekorn, & Taylar, 2000; Wald & Liu, 2001). A more recent study examined the
ability of VR driving measures to predict long-term driving performance in TBI (Lew,
Poole, Lee, Jaffe, Huang, Brodd, 2005). These researchers reported that the simulator
could provide additional, more sensitive information than the traditional road test.
Studies from our laboratory, using a virtual reality driving simulator (VRDS) has focused
on issues of user-interaction (Schultheis, Rebimbas, Mourant, Millis, 2007; Simone,
Schultheis, Roseman et al, 2006) and secondary side effects (Schultheis, Simone, Rizzo,
Hix, 2005).
3. Study Objective
While findings from these studies reinforce the potential value of VR driving simulation
as a clinical tool, additional work is required to establish the validity of this tool for
driver assessment following brain injury. In particular, our work has focused on the
development of a VRDS to provide assessment of the cognitive demands of driving. This
is of particular interest for clinical populations, such as brain injury, where cognitive
impairments are the most common and persistent deficits resulting from injury.
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
The current study examines issues of concurrent validity of the VRDS and examines the
relationship between VRDS measures and two clinical driving assessment methods: 1)
behind-the-wheel (BTW) performance and performance on cognitive measures relevant
to driving.
METHODS
1. Participants:
Twenty eight participants are included in the present study, 18 with BI and 10 HC. The
groups were matched for age, gender, race, and driving history. The HC group had a
significantly higher education level with a mean of 16.5 years, whereas the BI group had
a mean education of 14.3 years. Subject demographics are summarized below:
Table 1: Participant characteristics
Age (yrs)
Education (yrs)
Gender
female
male
Driving Experience (yrs)
BI Drivers
(n=18)
40.4
14.3
HC Drivers
(n=10)
35.8
16.5
33.3%
66.7%
21.5
40%
60%
18.6
Medical records obtained from rehabilitation hospitals and/or treating physicians
confirmed diagnosis of injury for BI participants. Participants with prior history of
severe psychiatric disturbances, extreme motion sickness, substance abuse or history of
prior TBI/CVA or any major medical/ neurological condition were excluded from the
study. Participants that required the use of assistive driving devises were not included.
At the time of testing all participants held a valid driver’s license in the states of New
Jersey, New York, and Connecticut. All participants were required to have a minimum
of one year of continuous driving experience, with BI participants meeting this
requirement prior to injury. Furthermore, participants were required to meet the
minimum visual requirement for their licensing state. Finally, in accordance with legal
and clinical requirements, all BI participants who were on anticonvulsant medication
were required to be seizure free for at least one year prior to testing.
2. Measures
2.1 The Virtual Reality Driving Simulator (VRDS)
The VRDS is a computer based system that uses a head mounted display (HMD) unit to
visually present computer-generated driving environments through which users can
“drive through” using a commercially available steering wheel and foot pedal (see Figure
1). The hardware includes a steering wheel and gas/brake foot pedals (Microsoft
Sidewinder), and a Proview™ XL50 HMD with 1024x768 resolution and 50° diagonal,
30° (V) x 40° (H) field of view, (Kaiser Electro-Optics, Inc.) with a motion track with 2
ms latency and 1° accuracy (Intersense I-Cube gyroscopic / geomagnetic sensor). The
virtual environments were delivered using a desktop computer and display (Gateway
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
multiprocessor 701 MHz Pentiums). The video update rate is 60 Hz (60 frames per
second). In addition to the visual feedback, the VR-DR
provides auditory (e.g. sound of car engine) and tactile
feedback (e.g. force feedback from the steering wheel) to
increase the VR experience for the user.
The VR-DS software was custom designed and includes a
driving route that represents several typical driving
scenarios that individuals could encounter while performing
daily activities (e.g., pedestrian crossing road, traffic). It
was patterned after the actual driving route used for clinical
driver evaluations conducted by the Kessler Institute for
Rehabilitation in West Orange, New Jersey. Total time to
complete the course is approximate 25- 35 minutes. The
entire VRDS driving course has nine separate areas (e.g.,
Fig.1 Participant at VRDS
residential, merging, commercial) each with unique driving
scenarios and demands. One specific area of the route was selected for this study. The
selected route represented a residential area which is the most commonly used type of
route in BTW evaluations. The area was approximately 4,075 feet in length and includes
one 4-way stop intersection and 2 regular stop sign intersections.
The VRDS automatically generates four quantitative output variables that are sampled
every 200 ms during simulation. No filtering is performed. These measures include
speed (in miles per hour), deviation from the center of the lane (in feet), distance to a
challenge (e.g. stop sign) in feet and head turn angle to the left or right (in degrees).
Consistent with the performance criteria of the BTW, three main VR driving performance
measures were calculated. Specifically, these three variables (See Table 2) represent
global performance in domains of driving. These VR variables were calculated by
summing the performance z-scores of other component VR driving measures.
Table 2: VR Driving Simulator Variables
VR Speed Control
VR Vehicle Management
VR Head Turning /Tracking
Mean Speed
Average speed in driving environment
Mean Lane Deviation
Average lane deviation over the entire
driving segment
Mean Head Turn
Average head turning over the
entire driving segment
SD of Lane Deviation
SD of the lane deviation over the entire
driving segment
SD of Head Turn
SD of head turning over the
entire driving segment
SD of Speed
Standard deviation of the speed
Stop Acceleration
Average acceleration over five seconds
from each of the three SS
Stop Deceleration
Average deceleration during the three
seconds prior to each stopping point
Distance stopped from the SS
Average distance an individual stopped
from the SS
SD = standard deviation, SS= stop sign
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
2.2 Behind the Wheel Driving Evaluation
A subcomponent of the participants in this study received the BTW evaluation.
This included a comprehensive “off-road”, pre-driving assessment to screen for potential
physical or cognitive limitations, followed by an, “on-road”, behind-the-wheel (BTW)
evaluation. The on-road component lasted approximately 30 minutes, during which time
participants were evaluated on various driving skills. The BTW evaluation was
completed by a certified driving evaluator and performance was documented using a
BTW checklist immediately following the on-route drive. The BTW checklist evaluates
five domains of driving related tasks observed during the on-route drive (Initial
Movement, Turning/Tracking, Speed Control/Management, Road Law, and Lane
Use/Management). The evaluator rated each item on the BTW checklist on a numerical
scale that reflected adequate (score of 1), marginal (score of 2), or failing (score of 3)
performance on each task, with lower scores reflecting better performance. A total
performance score for the BTW was calculated by summing the scores on 33 items,
whereby the lowest (best) and highest (worst) possible score a participant could earn was
33 and 99 points, respectively.
2.3 Cognitive Variables:
The cognitive variables used for this analysis were selected based on two criteria. A
thorough review of the clinical literature was conducted in order to determine the most
common cognitive tests associated with driving assessment and those that have shown
sensitivity to cognitive functioning and driving performance. Tests used assessed
following cognitive domains: executive functioning [Trail Making Test (TMT)]; visual
attention [Visual Cancellation Test}, working memory [Paced Auditory Serial Addition
Test (PASAT)] and visual spatial abilities [Weschler Block Design].
3. Procedures
All of the participants were screened by phone for inclusion and exclusion criteria.
Those meeting criteria were then scheduled for a testing session which lasted
approximately 3-4 hours. The testing session included administration of cognitive tests
as well as the VRDS. Prior to VRDS administration, all participants were provided with
verbal instructions on how to manipulate the various components of the VRDS, and were
given a practice trial to familiarize themselves with the devise
At the completion of the VRDS, participants were given a 15-30 minute rest to allow
them to re-accommodate to their surroundings. Following this, they were administered
the remainder of the protocol. Participants also completed a subjective driving history
questionnaire and simulation sickness monitoring questionnaires.
Results
First, we examined the relationship between VR driving measures and performance on
cognitive tasks relevant to driving. Bivariate correlations including all participants were
conducted. The results indicated a significant relations between VR Speed Control and
performance on the Block Design (r= .40, p<.05) and the PASAT (r= .45, p<.05). VR
Vehicle Management was significantly correlated with performance on the PASAT (r=
.52, p<.05) and VSAT (r= .43, p<.05). Given the small sample size and number of
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
correlations conducted, Hommel corrections were calculated. The results found only the
relationship between VR lane management and the PASAT (p=.025) was statistically
significant.
Second we examined the relationship between VR driving measures and measures of the
BTW evaluation. However, because the HC participants did not receive a BTW and not
every BI participant received a BTW examination, sample size for this analysis (n=11)
and subsequently power was considerably reduced. As such, we conducted these
analyses for exploratory purposes and hypothesized different relationships between the
VR driving performance measures and the two BTW measures (BTW total score and
BTW subcomponent scores).
As predicted, bivariate correlations showed no significant correlation between any three
VR driving measures with the BTW total score. By contrast, when bivariate correlations
were conducted including the individual VR subcomponents and the BTW
subcomponents significant relationship were found. VR speed control was significantly
related to BTW speed control (r=-.61, p<.05). There are four VR variables that are
significantly related to the BTW road law subcomponent. These are VR lane deviation
(r= .625, p<.05), head turning standard deviation (r= .627, p<.05), average distance from
the stop sign (r= -.72, p<.05), and deceleration (r=-.64, p<.05. Negative correlations are
to be expected, as higher scores on the BTW are indicative of a worse driving
performance, whereas higher scores on the VR driving simulator reflects better
performance.
Discussion
The current study examined issues of concurrent validity of a clinical VR driving
simulator that was developed for the assessment of driving ability following neurological
compromise (e.g. brain injury). Two commonly used clinical tools for assessing driving
were used: 1) performance on cognitive tests assessing domains relevant to driving and 2)
performance on a behind-the-wheel driving evaluation. VR driving performance was
summarized into three primary VR driving performance measures.
The data indicated that all three VR measures were significantly related to performance
in the cognitive domain of working memory (e.g. PASAT). Following a more
conservative analysis, the relationship between PASAT and VR Lane management
remained. It is interesting to note, that of the various cognitive tests examined, only the
PASAT was found to be significant. The fact that this measure of working memory is
related to VR driving is not surprising, as this cognitive domain has been found to be
related to other measures of driving performance in previous studies (Barkley et al, 2002;
Schultheis et al, 2001). What is notable is the fact that this test is not a commonly used
test in evaluating driving in many clinical populations (e.g., stroke, traumatic brain
injury) because of the difficulty level of the task.
The evaluation of the VR driving measures against the BTW performance was notable, in
the sense that no significant relationships were found at the global level, but significant
relationship were found when individual subcomponents of both tasks were compared.
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
Once again, from a clinical perspective, this finding is remarkable because the most
commonly used measure of BTW performance is a global pass/fail rating or score.
Although the current study included a small size and was exploratory in nature, some
inferences can be considered. In particular the findings may be suggestive of the possible
sensitivity that is offered by VR driving performance measures. That is, the current
sample included (for safety reasons) only individuals who returned to driving following
their brain injury. As such, it can be concluded that their driving performance was at a
higher level then individuals who were not able to return to driving (or more impaired).
Yet, despite a potentially higher functioning sample, VR driving measures were still
found to be correlated to working memory, a cognitive domain known to be relevant to
driving.
Similarly, the lack of findings when examining performance at a global level may suggest
that the beneficial aspects of VR driving simulation may be in its ability to generate
specific and discrete measures of driving. However, in contrast to the current subjective
ratings generated by the BTW, the VR driving performance measures may offer a method
for gathering objective measurement of where specific driving difficulties may be
occurring, thereby allowing a more individualized approach to evaluating driving skills.
In sum, driving is a complex behaviour that requires the successful integration of multiple
behaviours. The use of VR driving simulators may offer clinicians a new method to help
breakdown the components of driving following brain injury and perhaps increase our
understanding of the demands of driving. Although future studies should be conducted
with a larger and more diverse sample of individuals with brain injury, the current
findings begin to demonstrate the potential value of integrating this technology into
clinical practice to help meet an important clinical need---improved evaluations of
driving ability after neurological compromise.
References
Barkley RA, Murphy KR, Dupaul GJ, Bush T. (2002) Driving in young adults
with attention deficit hyperactivity disorder: Knowledge, performance, adverse outcomes,
and the role of executive functioning. J Int Neuropsychol Soc, 8: 655-672.
Dawson, D. R., & Chipman, M. (1995). The disablement experienced by
traumatically brain injured adults living in the community. Brain Injury, 9, 339-353.
Devaney-Serio, C., & Devens, M. (1994). Employment problems following
traumatic brain injury: Families assess the cause. Neurorehabilitation, 4, 53-57.
Fisk, G.D., Schneider, J.J., & Novack, T.A. (1998). Driving following traumatic
brain injury: Prevalence, exposure, advice and evaluations. Brain Injury, 12, 683-695.
Formisano, R., Bivona, U., Brunelli, S., Giustini, M., Longo, E., & Taggi, F.
(2005). A preliminary investigation of road traffic accident rate after severe brain injury.
Brain Injury, 19, 159-163.
Langlois, J.A., Rutland-Brown, W., Thomas, K.E. (2006). Traumatic brain injury
in the United States: Emergency department visits, hospitalizations, and deaths. Atlanta,
GA: Centers for Disease Control and Prevention, National Center for Injury Prevention
and Control.
______________________________________________________________________________________
DSC 2007 North America – Iowa City – September 2007
Leon-Carrion, J., Dominguez-Morales, M. R., & Barroso y Martin, J. M. (2005).
Driving with cognitive deficits: neurorehabilitation and legal measures are needed for
driving again after severe traumatic brain injury. Brain Injury, 19(3), 213-219.
Lew, H.L., Poole, J.H., Lee, E.H., Jaffe, D.L., Huang, H.C., & Brodd, E. (2005).
Predictive validity of driving-simulator assessments following traumatic brain injury: A
preliminary study. Brain Injury, 19, 177-188.
Marottoli, R.A., Mendes de Leon, C., Glass, T.A., Williams, C.S., Cooney, L.M.,
Berkman, L.F. et al. (1997). Driving cessation and increased depressive symptoms:
Prospective evidence from the New Haven EPESE. Journal of the American Geriatric
Society, 45, 202-210.
McCullagh, S., & Feinstein, A. Cognitive changes. In J.M. Silver, T.W.
McAllister, & S.C. Yudofsky (Eds.), Textbook of Traumatic Brain Injury (pp. 321-335).
Washington, DC: American Psychiatric Publishing.
Rapport, L.J., Hanks, R.A., & Bryer, R.C. (2006). Barriers to driving and
community integration after traumatic brain injury. Journal of Head Trauma
Rehabilitation, 21, 34-44.
Schultheis, M.T., Matheis, R.J., Nead, R., & DeLuca, J. (2002). Driving behaviors
following brain injury: Self-report and motor vehicle records. Journal of Head Trauma
Rehabilitation, 17, 38-47.
Schultheis, M. T., & Mourant, R. R. (2001). Virtual Reality and Driving: The
Road to Better Assessment of Cognitively Impaired Populations. Presence:
Teleoperators and Virtual Environments 10(4), 436-444.
Schultheis, M. T., & Rizzo, A. A. (2001). The Application of Virtual Reality
Technology for Rehabilitation. Rehabilitation Psychology, 46, 3, 296-311.
Schultheis, MT, Simone, L., Rizzo, A.A., Hix, D., (2005) Usability issues in the
clinical application of virtual reality technology. Proceeedings from the 28th
International Conference of the Rehabilitation Engineering and Assistive Technologies
Society of North America (RESNA), Atlanta, GA
Schultheis, MT, Rebimbas, J., Mourant, R. and Millis, SR. (2007) “Examining the
usability of a virtual reality driving simulator” Assistive Technologies , 19, (1), 1-8
Schultheis, MT, Garay, E. and DeLuca, J. (2001). The influence of cognitive
impairment on driving performance in multiple sclerosis, Neurology, 56; 1089-1094
Simone, L., Schultheis, MT, Rebimbas, J., Millis, SR. (2006) “Head Mounted
Displays for Clinical Virtual Reality Applications: Pitfalls in understanding user behavior
while using technology”. CyberPsychology and Behavior , 9(5),591-602
Thurman, D., Alverson, C., Dunn, K., Guerrero, J., & Sniezek, J. (1999).
Traumatic brain injury in the United States: A public health perspective. Journal of Head
Trauma Rehabilitation, 14, 602-615.
Wald, J., & Liu, L. (2001). Psychometric properties of the driVR: a virtual reality
driving assessment. Student Health and Technology Information, 81, 564-6.
Wald, J., Liu, L., Hirsekorn, L., & Taylar, S. (2000). The use of virtual reality in
the assessment of driving performance in persons with brain injury. Student Health and
Technology Information, 70, 365-7.
______________________________________________________________________________________