Using Ubiquitous Computing to
enhance experience in pervasive gaming
Henrik Berggren ([email protected])
Jesper Svenning Andersen ([email protected])
Ubiquitous Computing
DSV, 2008
Table of contents
Abstract
3
Introduction to Ubiquitous Computing
3
Introduction to our paper
3
Technologies
4
Augmenting environments
4
Augmenting clothes
5
Augmenting the human body
7
Implementations
8
Conclusions
9
References
9
Abstract
This paper describes how ubiquitous computing can be used to enhance the experience for players of
pervasive games. With a short introduction to the term ubiquitous computing, the terms augmentedenvironments, clothing and bodies are explained and discussed. We will then exemplify some specific
technologies implemented in each area and discuss advantages and disadvantages. Alternate reality
games have been using some of these concept to increase the players role and make the games more
interactive. We have looked at two different games and have briefly described their specific
implementations. Some conclusions will in the end be drawn on how ubiquitous technologies supports,
but also complicates, the participation in new forms of gaming.
Introduction to Ubiquitous Computing
Ubiquitous Computing was first mentioned in as early as 1988 by Mark Weiser. He then described
ubiquitous computing like this:
"It [ubiquitous computing] is invisible, everywhere computing that does not live on a
personal device of any sort, but is in the woodwork everywhere[...] its highest ideal is to make
a computer so embedded, so fitting, so natural, that we use it without even thinking about
it" (ubiq.com, 2008)
One of the goals is that technology should be available everywhere, hence the term ubiquitous. The
means for making the computer available everywhere is to take it apart into smaller pieces. These
devices will handle different tasks and thereby substituting post-its, books, paper and blackboards
(Weiser, 1991). The devices should under optimal conditions be available in hundreds of numbers in a
single room. They should be connected to a larger centralized local unit that stores the data. This unit
should also perform various calculations for the devices to minimize the power consumption of each
device. Furthermore the devices should not be carried around like a laptop but should be left lying
around so the next user will be able to use it for whatever purpose she sees fit. To make this possible,
the devices should of course be very cheap, otherwise no one would just leave a device behind. This
will result in the devices being absorbed in our everyday life and people will use them to carry out tasks
without reflecting over how they do it. The computer (or other digital artifacts) should fade into the
background of our attention. The focus should be on what we do and not how we do it. When a
person uses technology it should be as natural as eating cereals.
Ubiquitous computing is often described as the opposite of virtual reality. Virtual reality strives to
simulate our world (or a completely other world) and place the user inside the digital world. The user
almost becomes imprisoned by the technology not being able to move freely and getting entangled in
the wires interconnecting all the necessary equipment. Instead of this, ubiquitous computing sets the
the user "free" and tries to help out and improve the world that we live in.
It's now 20 years since Mark Weiser introduced the term ubiquitous computing and much have
happened in the last two decades. The basic programs we used to have locally installed on our personal
computers is now available through online applications. From here we can, for example, create, edit and
view images, documents, spreadsheets and presentations. Combined with the fact the more and more
of our personal content is being stored online our "personal" computer becomes less and less
"personal". We can use any computer to do the things we normally only did with our own computer.
We no longer think about what we do when we use mobile phones to check our e-mail or write an SMS
to a friend. But when our mobile phone starts ringing during an important meeting we are once again
reminded of the bluntness of technology. So even though the technology has evolved tremendously
and that we're getting closer and closer to the vision of Mark Weiser there is still a few blocks to go.
Introduction to our paper
The aim of this paper is to investigate how technology based on ubiquitous- and pervasive computing
can enhance the gaming experience for a player in a pervasive gaming environment. By enhance, in this
case, we refer to enabling the user to perform actions in the physical world that is also reflected in the
digital world. This gives the player extended possibilites to affect and be affected by the game and it's
story.
Before digging into technologies and details we will briefly explain the basic idea behind pervasive
gaming and Alternate Reality Games (ARG's). These rather new kind of games are based on the old
idea of live roleplaying. But instead of dressing up like an elf and run around with a plastic sword
trying to kill people, pervasive gaming is about playing a role that somewhat resembles yourself. The
goal of the designers of the game is to create a layer of uncertainty in between reality and the game. By
doing this the plan is to let the players wonder if things happening around them are a part of the game
or just based on plain coincidence. This gives the game a whole new dimension and also gives the game
designers a possibility to orchestrate the game and it's players in new ways. ARG's are often played in
larger cities where interaction with a lot of different people is common and where a story easily can be
embedded without being too noticeable. To be able to steer the players in the right direction of the
game and be able to monitor their progress, technology is needed. Techniques commonly used is GPS
for location and movement tracking, mobile phones for communication and RFID for object
interaction. In this paper we will try to give some examples of technologies that can be used for
supporting functions that are needed by these games and thereby enhancing the experience of the
game.
We will also exemplify some of the technologies by walking through two ARG's recently run in
Sweden, Momentum (MO) and Sanningen Om Marika (The Truth About Marika, SOM) which also
won an Emmy in Cannes this year. Although both games used various kinds of technology, MO was
quite experimental while SOM focused more on mainstream technology.
We wanted to investigate this since we are interested in new forms of gaming and how to make
technology usable in new contexts. The connection between this field in entertainment and ubiquitous
computing is also especially interesting because the aim of the gameplay and the aim of the technology,
to make it more pervasive that is, seem to match quite well. We hope that this paper can shed some
light over how different applications of technology can be used to orchestrate and monitor people in
urban enviroments.
Technologies
Ubiquitous computing is strongly connected to a wide variety of areas such as pervasive computing and
interactive spaces. All the areas have their own implementation of making computing ubiquitous. This
section will cover three of these areas; augmenting environments, augmenting clothes and augmenting
bodies. With the help of examples we will describe the areas and discuss usage, advantages and
disadvantages.
Augmenting environments
One way of making our everyday life more exiting and more interactive is to embed computers in our
near environment. Building computers into everyday objects in our homes and in the city, it is usually a
city, is something that has become quite common over
the last few years. Nowadays we can rent bikes,
browse through ads and read tourist information on
touchscreens and monitors nicely boxed on the street,
where we work or in our favorite cafe. However, is the
screen-based interaction model good enough for this
new context? After all it was designed for sitting at a
desk with a screen in front of you and not standing up
on a sidewalk. There are forces trying to make the
interaction model of computers augmenting our
environment to be more pervasive, easier to use and
to keep the properties of intuitive design. We will walk
you through one example that we think might succeed in doing that in the future.
This example is called Mobile Barcoding. It was originally developed by some of japans bigger mobile
phone companies and is based on the basic idea of barcodes. The old form of barcodes, that you can
see in your local food store, can't however hold so much data, so a new more sophisticated version was
developed. The new implementation is called 2D barcodes and from the name you might guess that it's
now two dimensional. These barcodes are basically a matrix of light and dark pixels (typically black and
white) which are placed in a square. The important fact here is that they can be read and interpreted by
a machine with the help of computer vision. This would have been quite unusable if wasn't for the fact
that we all walk around with a computer-vision-enabled machine in out pockets - the mobile phone. So
these graphical matrices can, with the help of a mobile phone and it's camera, be decoded into a
message, an URL or even a picture.
So, how do we augment the environments with this technology? Well, a number of projects has been
done around this concept. The barcodes can easily be generated by a computer to hold the assigned
message or URL and can then be placed in a newspaper, around the city in forms of stickers or even
displayed on a monitor. In japan the barcodes are as common as billboards and enables people to, with
a few simple clicks, download promotional songs, buy clothes right off ads for stores and book tickets.
The barcodes represent a loophole or a door into another world where the user can interact in an
extended way. We like to think that this concept represents the door between the physical and the
digital world where you use your mobile phone as the key.
Although this concept is huge in japan it hasn't really caught on in Europe yet. We think that this has a
lot to do with the mobile phone manufacturers who has chosen not to include the decoding software in
the hardware when delivered to the customer. And since downloading applications on to your phone
still is very immature, it makes the group of potential users quite small. However, Nokia has recently
broken this trend by adding the software to their latest N-series phone and we hope that more
manufacturers will follow.
Another common critique is that they are quite ugly an not very, in a sense, calm. They are quite easy to
spot around town and doesn't blend with the environment as well as one could wish. However,
developments are made continuously and new, more sophisticated, types of graphical representations
are emerging constantly. We believe that in just a few years most types of graphical objects can be used
as an interface to the digital world and that it will revolutionize the way we interact with our
environment.
Augmenting clothes
Augmented clothes is another way to achieve ubiquitous computing. Because the technology is getting
physically smaller and smaller, augmented clothes is one of the areas wich is rapidly evolving . It's now
possible to make micro-controllers, displays,
buttons etc. small, flexible and light weight
enough to sew into clothes without changing
the look and feel of the garment. With
augmented clothes you also have the
possibility to substitute your electronic devices.
Ideally it's no longer necessary to remember to
bring your personal electronic devices, just
wear your clothes as usual and head out the
door. With augmented clothes, technology will
be closer to you, but it is of course not
flawless. We will in the section below describe
the positive and negative sides of augmented
clothes, as well as the various ways augmented
clothes can be used and realized
A way of realizing augmented clothes is with the Lilypad1. The Lilypad is a modified version of the
Arduino micro controller2 . The Lilypad is flexible and can be washed. It has 14 digital I/O pins and 6
analog input pins. The Lilypad can easily be woven into fabric and the relevant sensors can be
connected to it. With this you can add something new to your everyday life. It can for example register
what the user is doing (running, overheated etc.) and perform some sort of reaction depending on
what is registered. So without performing any specific task the user interacts with the devices
connected to the clothes. This will give the feeling that the environment is adapting to the user. For
example if the body temperature rises above normal the clothes sends a signal to turn on the aircondition. This could of course also be used to control light, music and much more. But not only can
augmented clothes be used to pervasively interact with the environment, but also to interact with user
itself. Instead of sending a signal to another device, a signal could be sent directly to the user. This
could be done with light, sound or on a display. The user could also be notified about registrations
from the clothes. For example if the wearers pulse is to high or posture is bad the user can be warned.
Augmented clothes can also be used to interact with you electronic devices with more intentional
gestures. One of the first commercial examples that surfaced a couple of years ago was the "mp3
jacket". With the the "mp3 jacket" you can plug in your mp3-player and then control it with buttons on
the jacket 3. This could be implemented with the Lilypad but it could also be extended with other ways
of interaction with the devices. Besides regular buttons, interaction could be made with sound or
different body gestures. This way of interaction would be to prefer because this would minimize the
use of buttons. Like the "mp3 jacket" most of the augmented clothes have wires and buttons woven
into the fabric, but with nanotechnology, a new form of augmented clothes is being developed. The
company Eleksen4 have made fabric touch sensitive and opened up for a far more advanced form of
augmented clothing than seen before. Imagine the possibilities in wearing a fabric made touch screen.
By removing buttons the clothes would look more "normal" and thereby physically hiding the
technology even more. Instead of using regular buttons it could also be clever to disguise them to look
like things we are used to seeing on clothes. An example of this is the ZipperSensor5. The
ZipperSensor can sense how much a zipper is zipped/unzipped. An obvious use of the sensor is to
have a light in your pockets or purse to be lit when you unzipped the zipper, making it easier to find
your stuff in you pockets or purse. But you could also imagine that you could use the ZipperCensor to
control the volume on you mp3-player.
These are all examples that either already exists or could be possible to build today. All though these
new forms of clothing sound very promising there are still a lot of issues that needs to be dealt with.
One of the main issues is that the technology provided in augmented clothes can't comped with
handheld devices on the technical specifications. For a user to switch technology the new choice must
exceed the current available technology in at least one technical aspect, for example batterylife,
computational power, better interaction etc. While interaction with a simple device as a portable music
player, augmented clothes may work fine, but when it comes to writing a SMS or e-mail, it would still
be easier to use your phone. Because the technology is bound in the clothes you would either have to
wear that particular piece of clothes everyday or have technology built into all of your clothes. If you
didn't wear your "right" jacket you would be reminded of the missing technology, hence making the the
technology less invisible. Another major issue is power consumption. Are you supposed to recharge all
of your clothes every time you come home? What would happen if you run out of power in the middle
of the day? You would then have to be connected to a wire while charging and thereby being unable to
move until you are recharged.
1
http://www.cs.colorado.edu/~buechley/LilyPad
2
http://www.arduino.cc
3
There is a lot of these jackets, one of them is found at http://www.kenpojacketforipod.com/
4
http://www.eleksen.com/?page=technology08/elektex.asp
5
http://itp.nyu.edu/physcomp/sensors/Reports/ZipperSensor
Will there be a reason to embrace this technology, when it comes to it? Even though it could be
intelligent to have the air-condition turned on automatically, could we really rely on the accurateness of
the sensors? Wouldn’t it be easier to just grab the remote and turn the air-condition on instead? Maybe
that would be more transparent than using a technology that is harder to control if it doesn't act as
expected.
Augmenting the human body
Augmented bodies takes augmented clothes a step further. With the technology directly inserted in
your body you can't really leave it behind. Many of the examples in this section are still just conceptual
ideas or merely proof-of-concept prototypes. Still they serve to clarify the development and use of
augmented bodies.
One of the persons who have experimented with augmented bodies is the artist and scientist Stelarc.
The most known piece from Stelarc is the "Extra Ear" project 6. The main goal of the project was to
make an extra ear with real tissue and
connect it to his body. I addition to this,
Stelarc inserted a blue-tooth microphone in
his extra ear right under the skin. When
people now called him, his mobile phone was
connected to the ear. People would then be
able to hear what the extra ear was hearing.
This actually worked quite good until the
microphone caused an infection and had to
be removed. Stelarcs idea was to combine the
microphone in the ear with a loudspeaker in
the mouth. If his mouth was closed, only he
could hear the callers voice, but if his mouth
was open you could hear the callers voice
through Stelarcs mouth7. This is a quite
interesting idea although it must be modified
a little before put to commercial use. Because
it’s a rather simple technology and that the use of headsets is quite popular this could become a
commercial product.
Another project Stelarc have done is the "URL Body"8. Stelarc had a number of electrical pads attached
to his muscles. When an electrical current was send to a pad, the muscle would flex. The overall idea
was that Stelarcs body movements should visualise traffic on his own website. Electrical pads was
therefore connected to a computer that then sent an electrical current to the different pads. Instead of
the "random" movements it could also be used to control the body in a more strict fashion. This could
be used to "touch" people from a distance and thus creating a more physical element to long distance
communication. It could even be used to control another person, like having an avatar in the real world.
Even though some of these examples seem very arty and not very useful at first, they have elements
that could have a beneficial outcome.
Yet another example is from Jim Mielke who is an industrial designer. Jim Mielke's concept is a touch
sentive screen, that is inserted just under the skin9. The screen is powered by a blood fuel cell, so
basically the screen runs on the same energy as the human body. The screen is also with bluetooth so it
can be connected to any bluetooth device. For example it acts as a display for a mobile phone and
6
http://www.stelarc.va.com.au/extra_ear/index.htm
7
http://www.we-make-money-not-art.com/archives/2007/02/stelarcs-talk-a.php
8
http://www.stelarc.va.com.au/urlbody/introVR.html
9
http://www.physorg.com/news122819670.html
because the display is touch sensitive you would also be able to control your mobile phone with it. The
screen could of course also act "just" as a tattoo, but with possibility of morphing into something new
whenever you want. Because the device also continuously scans your blood, implanting this device
could actually mean that certain illnesses could be detected before they get severe or warn you about
other, smaller, health risks (blood alcohol level, overheating etc).
Because augmenting the human body is still a relative new area we still don't know enough about the
health risks connected to these procedures. Examples of health risks are radiation and bloodpoisoning.
These health risks could very easily be the biggest reason why most people would not have technology
operated into them yet. It's still a rather complicated procedure to insert an technological implant and
therefore you wouldn't want to have the implant changed very frequently. But because the technology is
evolving so rapidly the more advanced the device implanted in you gets, the more often you would
probably have to have it replaced. Not because you have to, but because most people are often keen on
having the latest technology available.
To avoid this it should be a rather simple device that should be inserted in you, like a RFID tag or GPS
tracker. We have also seen examples of pills with built in technology that is already used in sports. The
player swallows a pill before a game and the coach can then keep health statics of the player during the
match. Naturally the pill is only in the body for a certain amount of time, and unless it's a very cheap
form of technology there is also an issue concerning recovering the pill again. The bottom line is that
people are probably not ready to undergo a medical procedure to have their bodies augmented. The
advantages are simply too small to have a rather complicated surgical procedure. We will most likely see
augmented animals before people will start to get used to the thought of having an augmented body.
Implementations
The interest for new forms of gaming and thereby ARG's and pervasive gaming has grown
tremendously over the last years. There is quite heavy research going on at the moment and one of the
key players behind it is the market driven research institute - The Interactive Institute. A while back
they formed the so called Mobile Life Centre where the research focus lies on mobile usage and
technology. Among the researchers at this centre is a group looking at game play and future
implementations of new forms of gaming. This group, called Game Studios, has over the last years run
a couple of games of various sizes and designs. We will briefly walk through two of them and try to
explain how they used ubiquitous technologies to enhance the gaming experience.
The first game, Momentum (MO), was run in the city of Stockholm in the end of 2006. The goal was
to untie magical knots that were placed around the city with the use of RFID tags. The game was
played around the clock for about a month and the players could choose when and where they wanted
to play. The 30 participants had their headquarters down in Swedens first nuclear reactor, R1. A lot of
different technologies were used to be able to orchestrate and run the game. The knots that were
represented by RFID tags was searched for using low frequency radio receivers and clues were given to
the players by teleprinters. When a team thought that they had found a knot they used a custom made
glove that was augmented with a RFID reader and a mobile phone. The idea was that the players were
supposed to look for the knot by moving their hands near the RFID tag. When they came close enough
the glove responded with a vibration and registered the knot. This piece of information was then sent
via the mobile phone to the game masters orchestration software that marked that knot as completed.
This is a good example of humans, via augmented clothing, interacting with augmented environments
in a quite pervasive way. Although the players had to wear a glove the technology itself was never seen
and since the RFID tag was placed in hidden places, like inside a book in a public library, the very
action carried out became the point of focus, not the technology.
The second game was called Sanningen om Marika, SOM. This game was played last year all over
Sweden with the focus on larger cities. The goal of the game was to form a cell with a few friends and
then find and carry out missions together. These missions was placed all around town with the help of
mobile barcodes. These barcodes (missions) could be found online at the game’s web application that
was run by the game masters. After downloading a barcode it was then supposed to be printed by the
cell and be posted somewhere in the cell's home town. Each cell had one main mission but could also
complete other cell's missions and thereby gain more points and reach higher levels. The missions were
accepted by a cell by one of the members mobile phones. An application was downloaded to the phone
and the players could then use that application to decode the barcodes and recieve information about a
specific mission. There were also barcodes placed along the way of a mission to help the players with
clues. This is an ideal implementation of the mobile barcoding concept which augmented the players
enviroment in a non intrusive way. Although the matrices are easy to spot regular people didn't know
how to use them and therefore didn't care about them while players in the game saw them as
possibilities for a higher score and therefore were constantly on the lookout for the unique graphical
images. SOM ended up having hundreds of players all over Sweden and gained quite a lot of attention
from mass media (DN, 2008).
Conclusions
We think that the use of technologies that are pervasive or ubiquitous contributes to the experience of
a game when well implemented. Whether it's about building custom clothes, augmenting a special
environment or mounting implants on the human body the possibility to perform actions usually
limited by the physical world is exciting and interesting. However the technologies are still extremely
immature and it takes a lot of customizing and creative solutions to make it work well enough. It works
well for the group of people used to playing games because of their usual early-adopter-attitude. For
regular people, the technologies has to become far more transparent to be accepted and used. Gaming
is also a good implementations due to the limited amount of people playing at a given moment.
However we are sure that when the technology matures the applications will be better and will then suit
more people.
There are still some key problems that needs to be solved. Power consumption is one of them, a lot of
the current implementations are limited due to power shortage and until we can come up with a better
way to power and re-charge it will be hard to largely scale technology-intense games.
We also think that we will need to shift our focus from the technology to the actual usage. After all,
computes are experienced by us humans and we are the ones they are supposed to help. Therefore we
need to look beyond the regular screen-based-interaction model and further develop more pervasive
and intuitive ways to control computers given the new contexts they now are available in.
References
The Computer for the 21st Century, Weiser Mark, 1991
http://www.we-make-money-not-art.com/archives/2007/02/stelarcs-talk-a.php, 2008-04-26
http://www.ubiq.com/ubicomp/, 2008-04-26
http://itp.nyu.edu/physcomp/sensors/Reports/ZipperSensor, 2008-04-26
http://www.svd.se/kulturnoje/nyheter/artikel_369116.svd, 2008-04-26
http://www.prosopopeia.se, 2008-04-26
http://www.dn.se/DNet/jsp/polopoly.jsp?d=2374&a=759576, 2008-04-26
http://www.stelarc.va.com.au/extra_ear/index.htm, 2008-04-26
http://www.stelarc.va.com.au/urlbody/introVR.html, 2008-04-10
http://www.physorg.com/news122819670.html, 2008-04-10
http://www.kenpojacketforipod.com/, 2008-04-10
http://www.eleksen.com/?page=technology08/elektex.asp, 2008-04-26
http://www.cs.colorado.edu/~buechley/LilyPad, 2008-04-10
http://www.arduino.cc, 2008-04-10