Faculty of Computer Science
Effects of different materials for
tangible objects for interacting
with a word game on a reactive
table
Olav Hermansen, Tom Erik Hvring,
William Killerud
Subject:
Mobile Applications
Date:
Saturday 16th February, 2013
Supervisor:
Harald Holone
Contents
Glossary
3
1 Introduction
1.1 The word games Boggle and Boggle Flash . . . . . . . . . . .
1.2 Tangible interfaces and reactive tables . . . . . . . . . . . . .
1.3 Taking a Boggle Flash-like word game to a reactive table . . .
4
4
5
5
2 Related work
10
2.1 Most frequently used characters . . . . . . . . . . . . . . . . . 10
2.2 Tactile properties of objects . . . . . . . . . . . . . . . . . . . 13
3 Methodology
15
3.1 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Semi-structured interview . . . . . . . . . . . . . . . . . . . . 15
4 Implementation
4.1 reacTIVision . . . . . . . . . . .
4.2 C# TUIO library . . . . . . . .
4.3 Unity 3D game engine . . . . .
4.3.1 TUIO as input . . . . .
4.3.2 Creating the game world
4.4 Table . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
16
17
17
17
17
17
17
5 Results
18
6 Discussion
19
7 Future work
20
7.1 Chip size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2 Using dice as chips . . . . . . . . . . . . . . . . . . . . . . . . 20
7.3
Magnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2
Glossary
Chip
A square, flat object, used for playing the word
game. Has a fiducial marker on one face, and a
character on the other.
Fiducial marker A glyph used as an object identifier
3
Chapter 1
Introduction
1.1
The word games Boggle and Boggle Flash
R
Boggle1
is a word game played using a four by four grid of lettered dice.
The object of the game is to find words spelled out by adjacent dice in the
grid. The minimum length of a word is three characters, with longer words
granting more points, up to eight characters long (even longer words do not
grant more points). Abbreviations and names are not allowed.
Boggle Flash (also marketed as Scrabble Flash) is an electronic version of
the Boggle game. Instead of dice the game comes with five small electronic
devices. These devices each have a screen that displays one character. For
each round of the game the letter on the screens change. The object of
the game is to physically connect the different devices to compose words of
between three to five characters in length. The devices sense when a word
has been composed using a form of wireless communication, and looking up
1
Boggle is a registered trademark of Parker Brothers, owned by Hasbro
Word length Points
3, 4
1
5
2
6
3
7
5
8+
11
Table 1.1: The different scores in Boggle
4
the combination in a stored dictionary. When a word has been combined
and approved, the score for that word is calculated and added to the total
score of the player.
1.2
Tangible interfaces and reactive tables
In 1998 Gorbet [3] was exploring tangible objects as a interface for manipulation of data. The project used triangles with microprocessors inside, and
linking the triangles together would build different structures.
By combining these triangles you could add or remove attributes to the
final structure on the fly. A magnetic plate allowed the different triangles
to connect to each other to form new shapes. Depending on which sides of
the different triangles that where linked would shape a different signal to a
computer connecting all of the triangles [3].
The principles described above - tangible objects as an interface for manipulation of data - are still used today. The methods have changed slightly,
however. Instead of triangles with microprocessors, tangible interfaces are
built using image recognition technology to recognize physical objects.
To simplify the image recognition aspect, special shapes called fiducial
markers (see figure 1.1) are attached to, or otherwise incorporated in, the
objects. Different fiducial markers are attached to different classes of objects.
The reacTIVision and TUIO frameworks are open source implementations
of this type of image recognition software [5].
Using relatively simple components we can build a table structure as illustrated in figure 1.2 that enables the use of tangible objects tagged with
fiducial markers, as well as multi-touch finger gestures, to be used for interacting with the system.
1.3
Taking a Boggle Flash-like word game to
a reactive table
A reactive table such as the one described in chapter ?? opens for touch
interaction as well as interaction with tangible objects tagged with fiducial
markers. Touch interaction can be used to drag letters projected onto the
surface around to combine them into words. The application could generate
a random set of letters and have them float around in disarray, leaving it
5
Figure 1.1: Examples of fiducial markers
6
Figure 1.2: The different components in a reacTIVision reactive table [5]
7
up to the user to construct as many words from them as possible within the
given time.
While touch interaction certainly is feasible it will not be the main interaction style used in this project, simply because we wish to work with
tangible interaction. Touch interaction may still be used to some extent for
starting games and navigating menus.
By using flat, square objects (from now on referred to as chips) you can
have a fiducial marker on one side and a letter on the other side. The fiducialletter combination can be a simple sheet of printed paper, the paper can be
attached to a piece of plywood, burnt into the plywood itself, or a different
material altogether can be used.
This approach would mean having a pool of chips, and for each game
a number of these objects would have to be randomly selected. The chips
would then be rearranged by the player to form words. The system would
sense fiducial markers in close proximity to each other and run a dictionary
check on the combination, giving points if it is a valid word that has not been
played already. This would be a game very similar to Boggle Flash.
As mentioned, there are many ways such a chip can be constructed. It
may be made of plywood, plastic, metal, or glass. The material affects the
feel of the interaction and the effect of the visual feedback. For instance, a
transparent object will not block light. Will this make the visual feedback
of the game more prominent?
In this paper we will look at the different effects of playing the game with
chips made of wood, cardboard, and plexiglass. Specifically, we wish to look
at the effect the different materials have on the following:
• Visual feedback
• Feel when playing
• Physical error rates
The game will give visual feedback around the chips in a few ways. Firstly,
each chip will be surrounded by a digital border, with indicators on the left
and right face of the chip where the player will ”connect” the chips. If the
player connects two or more chips, the border will light up - green if the word
is valid and gives a point, and red if the word is invalid.
The different materials we wish to test will have different attributes and
feel different in a player’s hand. We wish to look closer at what differences
8
there are, and what advantages there may be to choosing one material over
the other, for example reduced physical error rates. A physical error would
for instance be losing grip of a chip, or having a chip drop off the table.
9
Chapter 2
Related work
2.1
Most frequently used characters
Being that the game being developed in this project is a word game it is of
interest to know the usage frequency of characters in the target language in
order to provide an appropriate proportion of the characters when playing a
game.
Based on the Concise Oxford Dictionary, Oxford Dictionaries came up
with the character frequency of the English vocabulary, shown in figure 2.1.
The table shows the usage of each character proportionally compared to Q,
the least used character in English. Of all characters in the English language,
E is the most used, at 11.16%, followed by A and R at 8.50% and 7.58%
respectively. At the opposite end of the scale we have Q, J, and Z, at 0.20%,
0.20% and 0.27% respectively.
A similar overview of most frequently used characters in Norwegian was
unavailable to us. Running a character frequency analysis on Rapport fra
22. juli-kommisjonen published by Norges offentlige utredninger [2] gave us
the results shown in figure 2.2. Just like in English, E is the character that
is most frequently used, at 15.08%, with T, R, and N following at 9.01%,
8.55%, and 7.64% respectively. A bar chart of the results can be seen in
figure 2.3.
The fourth column in figure 2.2 shows the number of occurrences of the
character in proportion to the character Ø. We decided that characters with
a frequency lower than Ø should not be taken into consideration for the game,
as the difference in number of occurrences was huge between the lowest (Q,
10
Figure 2.1: ”The third column represents proportions, taking the least common character (q) as equal to 1. The character E is over 56 times more
common than Q in forming individual English words.” [1]
11
Figure 2.2: The results of the character frequency analysis of [2]
12
Figure 2.3: The results of the character frequency analysis of [2]
at 37) and the highest (E, at 200546).
2.2
Tactile properties of objects
In [4] Jeanine Kierkels and Elise van den Hoven ”investigat[e] the role of
material hardness in the haptic experience of tangible artifacts”[4]. The
project looks at children’s experience in particular.
Kierkels and van den Hoven found that there are two ways of describing
materials, called engineering dimension and perception dimension [4]. The
engineering dimension is based on the physical and technical attributes of the
object. The perception dimension is based on how the objects are percieved
by the senses - whether the object feels smooth or rough, soft or hard, warm
or cold, or flexible or stiff [4].
Running the experiment with children between 10 and 13 years old, the
participants were given several objects inside a ”blind box”, where both hands
were inserted, but which did not allow visual contact with the objects.
Kierkels and van den Hoven observed that children had no problem classi13
fying objects by hardness, identifying soft items as ”cute, speedy and warm”,
and hard objects as ”boring, sad and old-fashioned”[4]. They recommend
paying attention to this when designing tangible user interfaces for use by
children.
In [6] Ullmer and Iishi, working on the metaDESK project, created a
transparent acrylic model of the Great Dome at the Massachusetts Institute
of Technology campus for use on their reactionary table. They chose acrylic
to minimize occlusion (obstructing vision of the table beneath), for the model
to blend somewhat seamlessly with the graphics on the table, and to use
lighting effects to affect the model [6].
14
Chapter 3
Methodology
The questions we have are about the impact the material of the chips have
on the visual feedback of the game, the feel of the chips in hand, and the
number of errors the player has using the chips. The visual feedback and
the feel of the chips are things that are not naturally measurable, meaning
a qualitative approach has to be made. Finding the number of errors will
require observation of the player.
To test the game we will recruit five test subjects that will each play one
round of the game with each of the three sets of chips. Each round lasts X
minutes, and the player will choose five chips from the pool of available chips
each round. Score achieved in the game does not factor in to the results.
During the game the subject will be observed by one of the test administrators, as outlined in chapter 3.1. At the end of each round, the subject
will be interviewed as outlined in 3.2, before the next round is started with
chips of a different material.
3.1
Observation
3.2
Semi-structured interview
15
Chapter 4
Implementation
The project uses the following components and frameworks:
• Dell *** wide-angle projector
• Microsoft LiveCam Studio camera
• Infra-red light source
• Glass surface, sand-blown on one face
• Wooden structure
• reacTIVision framework
• C# TUIO library
• Unity 3D game engine
16
4.1
reacTIVision
4.2
C# TUIO library
4.3
Unity 3D game engine
4.3.1
TUIO as input
4.3.2
Creating the game world
4.4
Table
17
Chapter 5
Results
18
Chapter 6
Discussion
19
Chapter 7
Future work
7.1
Chip size
In this paper chip size has not been discussed, and a size that would make
the game more accessible might be available. Investigating this aspect of the
game is a possible line of future work.
7.2
Using dice as chips
In this paper we have looked at chips that are flat squares. A possible different
approach from flat squares is to use dice. To make the dice both human- and
machine readable a special acrylic can be used that allow infra-red light to
shine through, but not light visible to human eyes. In this way, a fiducial
marker can be placed underneath a layer of acrylic, and a letter be placed on
top of that layer. Using such dice will take the game closer to its analogue
origin in Boggle.
The dice will be taller. Are they then easier to grab hold of on the table?
The flat squares will obscure less of the table surface when viewed from an
angle. Will this have a positive effect on the visual feedback from the surface?
Does it matter that there is a roll of the dice instead of picking letters from
a pile? Does it feel more fair?
20
7.3
Magnets
Tactile feedback on connection [3]. Supplements GUI border around chip.
21
Bibliography
[1] Oxford Dictionaries, What is the frequency of the letters of the alphabet
in english?, http://oxforddictionaries.com/words/what-is-the-frequencyof-the-letters-of-the-alphabet-in-english.
[2] A.B. Gjrv, R.L. Auglend, L. Bokhari, E.S. Enger, S. Gerkman, T. Hagen,
H.B. Hansen, G. Hjeltnes, L.M. Paulsen, and K. Straume, Rapport fra
22. juli-kommisjonen, Tech. report, Norges offentlige utredninger, 2012.
[3] M.G. Gorbet, M. Orth, and H. Ishii, Triangles: tangible interface for manipulation and exploration of digital information topography, Proceedings
of the SIGCHI conference on Human factors in computing systems, ACM
Press/Addison-Wesley Publishing Co., 1998, pp. 49–56.
[4] J. Kierkels and E. van den Hoven, Children’s haptic experiences of tangible
artifacts varying in hardness, Proceedings of the 5th Nordic conference on
Human-computer interaction: building bridges, ACM, 2008, pp. 221–228.
[5] J. Mickelson, M. Canton, and W. Ju, Pattern poses: embodied geometry
with tangibles and computer visualization, Proceedings of the 10th International Conference on Interaction Design and Children, ACM, 2011,
pp. 242–245.
[6] B. Ullmer and H. Ishii, The metadesk: models and prototypes for tangible
user interfaces, Proceedings of the 10th annual ACM symposium on User
interface software and technology, ACM, 1997, pp. 223–232.
22