Graphically Speaking
Editor:
Miguel Encarnação
Games, Virtual Reality, and the
Pursuit of Happiness
Marientina
Gotsis,
University
of Southern
California
I
haven’t felt intellectually attracted to computing
research for some time, but as of recently, I feel
hopeful about the future of our field and humanity at large. Newer-generation games are finally exposing the masses to virtual reality (VR) concepts.
This month, I attended the 2009 Games for Health
(GFH) Conference in Boston, I was upgraded to first
class on my return flight, and I saw a rainbow at the
end of the runway before taking off. It all seems too
good to be true, and perhaps so do “health games.”
Technology focus has shifted dramatically toward
health in the past two years. Is it all a “bubble,” or
are we posing the right questions this time around?
The ancient Greeks believed that a healthy mind
exists in a healthy body. In the past few decades,
scientists have focused most of their attention on
developing technologies that sharpen only our
minds or relieve our minds and bodies of certain
duties. So, we’ve physically atrophied. We’ve become smarter but incredibly unhealthy.
Health games seem to offer one solution to this
problem. The concept of combining fun, technologyenabled interventions for motivation is probably
as old as the stick and the carrot. Entertainmentbased health interventions have a huge potential
to transform healthcare. Already, researchers have
been tasked with evaluating this emerging field
(for example, visit www.healthgamesresearch.org
or see “Playing for Real: Video Games and Stories
for Health-Related Behavior Change”1). For many
people, the concept of health games is like a rainbow at the end of a long, rainy day, with a pot of
gold potentially waiting at the end. But will this
fantasy come true?
Healthcare and Innovation
If your career depends on the hunt for innovation, you must be extremely skeptical to determine
whether a grand idea is just an empty bubble glistening in midair. We know that healthcare beyond
big pharmaceuticals is long overdue for major investment, so health-related computing is well on
its way to becoming the next big thing.
14
September/October 2009
I’ve observed, followed, and contributed my fair
share toward innovation and bubbles. I’ve happily
observed many novelties become fads, then industries, and then bankruptcies, only to be revitalized
by emerging fields. In my short life, I’ve already witnessed the birth of desktop publishing, VR, robotics,
the Web, the open source movement, games, sensor
networks, and other new concepts, products, and
movements with intertwined fates. Each time innovation occurs, many brave, smart, and/or greedy
souls have used it to seek the next phase of healthcare applications.
Our society often penalizes those who choose
an unconventional way to practice what they’re
trained to do, whether they’re artists, doctors,
engineers, or other professionals. In the field of
innovation, those who do well are most certainly
outsiders. The GFH Conference consisted mostly
of outsiders, and I felt at home. The rooms were
full of people who sought a spark and demonstrated both fear and passion. It was joyful to see
so many trained professionals cross their knowledge boundaries and look for answers. However, as
with all new fields, I also observed the ubiquitous
bunch of folks looking for the pot of gold.
Unfortunately, the first spaghetti strand thrown
on the wall almost never sticks. If it did, my father would have been a millionaire. Only a few
years ago, he worked on an online counseling service—what we would now dub a “Health 2.0”2 application—for a company with stock options. That
company didn’t succeed because the concept was
ahead of its time. But in 2009, real humans can receive counseling in virtual worlds such as Second
Life,3 and real therapists can practice on virtual
humans (see Figure 1).4 However, even today, it’s
hard to figure out who will pay for these innovations, and the health-games concept is in dire
need of a business model.
Health and Happiness
According to Epicurean philosophy, happiness is the
absence of suffering. Much of Epicureanism has
Published by the IEEE Computer Society
0272-1716/09/$26.00 © 2009 IEEE
been distorted by notions of hedonism, but Epicurus’s teachings are the closest to modern humanism as we know it. Science and technology should
help humans become happier. Because disease is
what most often compromises our happiness, we’ve
always sought and found ways to use scientific and
technological advances to improve human health.
In the past, we tried to improve our health by
segregating ourselves from nature. This plan has
backfired because humans are inescapably embedded in nature. Technology has sheltered us from
the storm, and science has healed some of our ills,
but in doing so we seem to have contributed to
nature’s demise, which in turn has affected our
health and happiness. Health games offer no solutions to healing nature, but they certainly seem
to help many people reduce their suffering. Such
innovations are worth paying attention to.
Most people love games: playground games, tabletop games, card games, text-based games, console games, PC games, and mobile games. Play is a
fundamental mode of expression, fulfills the human need to connect with the “other,” and, may I
dare say, can even be fun. Serious play is also great
exercise for the mind and spirit.
My late grandfather lived to be 97 and died with
a smile on his face. His epiphany in life came
when he was an obese, middle-aged man who had
broken his leg and was spending a long time in
the hospital. Too scared to leave his fate to modern medicine again, he found his health through
exercise. His happiness stemmed from defying his
two archenemies: doctors and lawyers (he was one
himself). He enjoyed hiking in the mountains surrounding Athens. His only source of news was his
FM radio, but he demonstrated the expectation
that technology could bring us closer to nature.
Conventional research has moved past wearing
headphones while hiking to interfacing with Nintendo Wiis and other computer-based technology.
The hottest topic in computer games right now
is exergames. This cleverly annoying term simply
refers to games that make you sweat. We might
not be able to cure cancer yet, but we know that
exercise can help us achieve optimal health and
fight against time and our genetic predispositions.
The Wii was a good start for motivating people
to get off the couch (see Figure 2 for an example).
Unfortunately, hardcore sensor aficionados are
probably still on the couch. The Wii isn’t good
enough for VR fans because we can cheat the
sensor too easily, but for millions of consumers
who bought the Wii and are hopping around in
their living rooms, it’s the first step toward liberation. For those people, the Wii is as close to VR as
Figure 1. Virtual Justina from the Virtual Patient Project at the University
of Southern California’s Institute for Creative Technologies. This virtual
patient helps provide feedback on a trainee’s psychiatric interviewing
skills. (Source: Patrick Kenny; used with permission.)
Figure 2. Extreme Wii Boxing at the Zemeckis Media Lab, a 14-screen
projection room at the USC School of Cinematic Arts, Interactive Media
Division. The spectacle created by playing with the Wii compensates for
a low-fidelity sensor. (Source: Mike Brazil; used with permission.)
they’ve ever gotten. Sadly, the Wii and the aforementioned exergames are as close as VR has ever
gotten to reaching the greater public.
I don’t think we’ll dethrone the qwerty keyboard
anytime soon and replace it with body-based interfaces, but we can certainly replace our seated
leisure activities with something that burns more
calories and gives our bodies a workout. I don’t
think my grandfather would give up the hills for
an Expresso Fitness bike (http://expresso.com/
products_services) or EA Sports Active (www.
easportsactive.com), but not everyone lives in Greece.
IEEE Computer Graphics and Applications
15
Graphically Speaking
Figure 3. Dancers from the Anatomical Dance Theater Company at the
University of Illinois at Chicago’s Electronic Visualization Laboratory
explore the Inside_Out project. This project used a tetherless optical
tracking system to create a mixed-reality improvisation stage.
The Promise of VR
In my quest for happiness, I pursued computer
graphics (CG) and VR, but something was always
missing. I sat in one-too-many paper presentations
about fire and smoke particle systems, cloth and
hair simulations, industrial demos, virtual walkthroughs of temples and museums, and avatars in
networked worlds. Of course, we marvel at CGgenerated animation’s results (even if the plot really
sucks), and we nearly choke with excitement if the
technique demonstrated can be rendered in “real
time.” However, we’ve all probably fallen asleep at
least once during a presentation about how to improve an algorithm that renders translucence.
This promise of real-time realism sold VR output systems for nearly two decades. Yet if you ask
those who experienced VR worlds what their most
memorable emotional experience was, they might
tell you it was a nonrealistically rendered world—
something fantastic, extraordinary, and makebelieve. Or maybe it was something they had never
seen before, such as a far-away galaxy or a mythical world with curious creatures. It might appear
that I just dismissed CG and VR for their focus on
realism, and in fact, I did. Not because the cause
wasn’t honorable, but because in our quest for realism, we somehow forgot that there’s not only the
rest of our body available for interaction but also
the rest of our imagination.
I’ll leave the details to neuroscience, but a fully
engaged brain engages the entire body in a physical
response. Arousal is scientifically measurable, but
one obvious way to determine whether someone is
emotionally engaged involves simple observation.
Just look for smiles, frowns, raised eyebrows, or
tears, or listen for laughter. We might not always
be in touch with our emotions or be able to describe or interpret what we feel, but our body language tells the truth, as does our sweat.
I found myself rebelling against VR during
my last year in graduate school. The resolution
of some of my frustration ironically came about
because of a combination of injuries aggravated
16
September/October 2009
through computer use. Writing code without
breaks in the middle of the night most certainly
called for an intervention. I took dance classes to
rehabilitate my posture and attempted to create a
full-body VR experience for my thesis (see Figure
3).5 The idea was ambitious but earnest. The project miraculously worked, but the bigger victory
lay in the inevitable breaks between writing code
and testing my application, which required a lot
of moving around.
When I graduated and no longer had access to
high-fidelity VR technologies, I abandoned the
field for several years. Many VR fans went back to
the drawing board for similar reasons. I helped a
new department in interactive media get off the
ground at the University of Southern California
while the digital-games field was taking off. At the
time, digital games were popular, but no serious
curricula offered accredited degrees in interactive
entertainment. It was great to start fresh, but I
was bitter because there were no PC or console
games I really cared for anymore and the students
spoke a language I didn’t understand. Once again,
I was an outsider.
But I soon realized I was the luckiest outsider
there ever was because our students started creating games and other interactive experiences
that were fun. I played and enjoyed almost all the
games and, most important, was surrounded by
people who had hope that what they were doing
was changing the world around them.
Five years later, independent games (as opposed
to mass-produced games from big corporations)
aren’t just a fad, and the serious-games movement
is making some dents in the world. Health games
and sensor-based experiences have recently become the new, shiny promise of hope and change
in our field. However, an affordable, untethered
full-body VR experience is still the Holy Grail. Microsoft’s Natal project, which offers a controllerfree gaming experience, hints at the future, but
I’m afraid that for the greater VR community, it
will feel like the Wii all over again.
The Pursuit of Happiness
Epicurus maintained a famous garden outside
Athens; he also admitted women and slaves to his
school. Away from the worries of the city, the allinclusive modest pleasure of tending to nature sent
a message of optimism. Many people packed in
dense cities dream of nature, such as Jenova Chen,
a young game designer from a packed city in China.
His game company (www.thatgamecompany.com)
released Flower for the Sony PlayStation 3 in January 2009 (see Figure 4).
Figure 4. Flower for the Sony PS3 game console, in which the player uses the controller’s tilt sensor to navigate through a
landscape. Thousands of blades of grass interact with the player in a stunning immersive visual experience that’s emotionally
engaging. (Flower is a trademark of Sony Computer Entertainment America Inc. © 2008 Sony Computer Entertainment America
Inc. Developed by thatgamecompany.)
Playing Flower was as close to VR as I’ve gotten
in the past five years. Liberated from complex controller buttons, I role-played a flower petal by using
the controller’s tilt sensor to navigate a sublime
landscape. It was exhilarating, and for once, I appreciated all the special real-time graphics effects
that compensated for the absence of head tracking and stereoscopy, luxuries I’ve always expected
from VR. Sure, I didn’t break into a pouring sweat,
but my stress melted away, and I transported myself into an endless field of green grass and beautiful nature. My only major complaint of the health
games movement is that it’s so focused on health
that it at times misses out on obvious factors that
contribute to overall health, such as happiness.
Stress is as detrimental to health as excess weight,
so in our quest for health games, let’s not forget
life’s simple pleasures.
Our happiness is connected not only to simple
pleasures but also to our deeper understanding of
who we are and where we came from. In dealing
with health issues, we most often focus on illness and its consequences. We’re unprepared for
the illness journey, and we often look at illness
as a single isolated event in our lives. For some,
illness has a narrative arc; that is, it might be
resolved through a cure or death. For others, it’s
a constant state, as in the case of chronic illness.
Most everyone would agree that every event in
our lives, whether it represents pleasure or illness,
is part of the story of our life and the lives of others around us.
If life is a journey, our health is a very complex
record of stories parallel to life, and our happiness
depends on it. Stories provide a context for seemingly isolated occurrences that might otherwise be
meaningless. Our life stories are encoded in not
only our living memories but also our genetic material. Our sense of self, place, and provenance are
deeply interconnected. We’ve always used art and
technology to help preserve and prolong our longevity beyond our current selves. Already, decoding
our genetic history might help us understand the
past and perhaps help us circumvent an unfortunate future, but how do we reconcile our past and
future with our present? We exist through a complex layer of storytelling and memories of things,
people, and places.
If you could combine the best aspects of games,
VR, and sensors, the output would be augmented
reality (AR), a once clumsy, expensive technology
that made people and their devices look silly but is
now making a splash through cell phones. AR can
reveal context-aware information in the natural
and built environment, on demand, but also as intended by content authors. AR’s Holy Grail should
IEEE Computer Graphics and Applications
17
Graphically Speaking
Figure 5.
TwittARound,
an augmentedreality Twitter
viewer for
the iPhone.
(a) Users
can overlay
location-based
tweets in the
phone viewer,
which (b) lets
them leave
virtual notes in
situ. (Source:
Michael
Zoellner;
used with
permission.)
be the sharing of our stories across space and time.
In AR’s quotidian form, it’s like hiding a present for
someone in the park. But what if you could stumble
upon a message left for you from a long-gone family
member or friend, or preserve a moment in time
you shared with someone at some amazing place
(see Figure 5)? What if your children could visit
that location and view that story or find something
you left for them? This sounds like science fiction
now, but a whole generation younger than I am will
be able to track their parents’ digital location-based
history long after their parents are gone.
I’ve stopped pursuing VR to achieve happiness.
Games have given me some hope and sparked my
imagination. However, they keep me tethered, and
We have to release our precious ideas
into the world, hoping they’ll mutate and
reappear in some other shape or form.
virtual humans on my flatscreen overwhelm me because I can barely find time for my real friends. The
future of my health is uncertain, so I’ll continue
to seek new ways to preserve my past and augment
my present through computing. I think I would be
happy if someone who cared for me could invoke a
rainbow through AR at the end of a rainy day.
Outsiders for a Sustainable Future
Happiness can be hard to find but easy to measure.
For many, it has a simple binary value, but measuring success in a material world has become a 64bit operation.6 The pot of gold at the rainbow’s end
is a metaphor for a wide gradient between fame
and fortune. In our field, measuring scholarship
alone is a moving target.7 To complicate matters,
our compensation depends more on our primary
appointment at a university—which might not be
in a computer science department—than on our
contribution to the field. Publications and patents
18
September/October 2009
certainly play a major role in promotion within
ranks, and most universities have created aggressive technology transfer centers that make deals
with industry before a single pixel is drawn on the
screen. Will all this excitement last?
I’ve always felt that you should measure success
by the sustainability of innovation. Sustainability
usually refers to a commitment toward improving
social and environmental conditions.8 In our field,
this sustainability entails enduring the frustration
of seeing technologies we experimented with decades ago suddenly reach mainstream and be considered new again. Camera-tracking applications,
voice recognition, gesture-based interfaces, sensors,
3D, stereoscopy, and other favorites are now finding
their way into consumer pockets, backpacks, and
living rooms.
However, it’s no longer possible to accommodate
all possible usage needs and scenarios, nor can we
sit on ideas for too long in an era when anyone can
release an application on the Apple App Store. In
CG, our job is never done. From an educational perspective, this means that looking at past innovation
in our field is now even more important. Screens
get small, then large, then small again. They change
shape, brightness, and surface material. Don’t even
get me started on resolution. As for input, we have
to factor in the qwerty keyboard, mice, pens, joysticks, game controllers, musical instruments, and
moving body parts. Just when you thought a modality was going to disappear, it returns.
Managing this complexity can be overwhelming,
but if we adopt a model of sustainability, maybe
we’ll stop being so frustrated. We have to release
our precious ideas into the world, hoping they’ll
mutate and reappear in some other shape or form.
Maybe if something keeps coming back, it isn’t
a bad cold but the ultimate proof of our success.
This is why health games is an exciting research
area today. It’s an opportunity to reuse what we’ve
learned from decades of CG, VR, and digital-game
research, merge that with knowledge with other
areas, and release new ideas into the world.
I
urge you to become an outsider and collaborate
on health topics with researchers in medicine,
psychology, genetics, neuroscience, sports, architecture, geography, music, dance, and social work.
Right now, the field is fresh and exciting, and your
ideas will be welcomed. If you’re successful, these
ideas will be quickly appropriated. If you’re lucky,
these ideas will improve the quality of life for ourselves and others in the world.
References
1. T. Baranowski et al., “Playing for Real: Video Games
and Stories for Health-Related Behavior Change,” Am.
J. Preventive Medicine, vol. 34, no. 1, 2008, pp. 74–82.
2. “Health 2.0,” The Economist, 16 Apr. 2009; www.
economist.com/displayStory.cfm?story_id=13437940.
3. M.N. Kamel Boulos et al., “Web 3D for Public,
Environmental and Occupational Health: Early
Exam­ples from Second Life,” Int’l J. Environmental
Research and Public Health, vol. 5, no. 4, 2008, pp.
290–317.
4. T.D. Parsons et al., “A Virtual Human Agent for
Assessing Bias in Novice Therapists,” Medicine Meets
Virtual Reality 17—NextMed: Design for/the Well Being,
IOS Press, 2009, pp. 253–258; http://booksonline.
iospress.nl/Content/View.aspx?piid=11450.
5. M. Gotsis, “Inside_Out,” master’s thesis, Graduate
College, Univ. of Illinois at Chicago, 2003; www.evl.
uic.edu/core.php?mod=4&type=3&indi=386.
6. S.J. Dubner, “How Can We Measure Innovation? A
Freakonomics Quorum,” blog, 25 Apr. 2008; http://
freakonomics.blogs.nytimes.com/2008/04/25/howcan-we-measure-innovation-a-freakonomics-quorum.
7. M.Y. Vardi, “Conferences vs. Journals in Computing
Research,” Comm. ACM, vol. 52, no. 5, 2009, p. 5; http://
cacm.acm.org/magazines/2009/5/24632-conferencesvs-journals-in-computing-research/abstract.
8. L. Kaufmann et al., “Sustainable Success,” Wall Street
Journal, 22 June 2009; http://online.wsj.com/article/
SB10001424052970203334304574159330047219184.
html?mod=googlenews_wsj.
Marientina Gotsis is the Media Lab Manager at
the Interactive Media Division of the University
of Southern California’s School of Cinematic Arts.
Contact her at [email protected].
Contact department editor Miguel Encarnação at
[email protected].
IEEE Computer Graphics and Applications
19