Geometric Games for Assessing
Cognitive, Working Memory, and Motor
Control Skills
Beatrice Floyd
Abstract
Case Western Reserve University
This paper presents the authors ongoing work in
designing and building a set of blocks with embedded
electronics for the cognitive, working memory, and
motor control assessment of children. The sensorintegrated geometric blocks (SIG-Blocks) act as game
controls for a computerized game, called TaG-Games.
Three types of TaG-Games, Assembly, Shape Matching,
and Sequence Memory, have been developed. These
games are currently being tested on young adults for
preliminary evaluation, which will be followed by an
additional evaluation study focusing on children. Game
administration and behavioral/performance
assessments during play are fully automated by
embedded sensors and wireless communication enabled
between the blocks and a host computer.
Dept. of Mechanical and Aerospace Engineering
Cleveland, OH 44106
[email protected]
Donghwa Jeong
Case Western Reserve University
Dept. of Mechanical and Aerospace Engineering
Cleveland, OH 44106
[email protected]
Kiju Lee
Case Western Reserve University
Dept. of Mechanical and Aerospace Engineering
Cleveland, OH 44106
[email protected] (Primary Contact)
Keywords
Tangible Game Interface, Geometric Toys,
Cognitive/Intelligence Assessment, Play-based
Assessment
ACM Classification Keywords
Copyright is held by the author / owner(s).
TEI 2012, Kingston, Ontario, Canada, February 19 – 22, 2012.
ISBN 978-1-4503-1174-8/12/0002 $10.00
H5.2 Information interfaces and presentation: User
interfaces - Prototyping
Introduction
figure 1. Four SIG-Blocks assembled
in a 2-by-2 configuration.
Young children on average spend approximately halfan-hour playing games on the computer every day [2].
Many people are concerned about the effect that this
computer usage could have on children and have
studied its impact [4,8,12]. These studies have
identified the problems with computer use as being the
exposure of child to inappropriate content, advertising,
and safety concerns, as well as, health concerns like
sleep and physical activity displacement, vision
problems, and addiction [12]. On the other hand these
studies also discuss the evidence that computer use
can have positive effects in regards to development of
cognitive, fine motor, social interaction, and visual
processing skills [12]. Individual studies have shown
that computers can help children develop a variety of
skills including cognitive and motor development [1],
problem solving [10], numeracy [9], and handwriting
[7].
We believe that a child’s computer time should be
made as productive as possible in order to avoid the
negative effects. We developed TaG-Games as a
computer game for children using a tangible interface
that will not only improve their fine motor and cognitive
skills, but also perform assessment of these skills in a
simple, non-intrusive manner. Early diagnosis of
cognitive and fine motor control disorders means earlier
treatment resulting in better outcomes in the long term
[3]. It is thus important that assessment happens early
on, but assessments of such skills traditionally take
place in a controlled environment and must be
overseen by a trained professional. By integrating the
assessment into a fun environment that children
already participate in on a daily basis, we hope to
provide an assessment tool that can help objective
assessments of cognitive problem-solving, working
memory, and motor control skills in a less controlled
environment.
System Description
TaG-Games is comprised of a set of SIG-Blocks and a
graphical user interface (GUI) that interact by way of
wireless communications [5,6]. The SIG-Blocks are
used as game controls to play TaG-Games and the GUI
gives the user feedback and assigns the user new tasks
to complete.
A
B
C
D
E
figure 2. Schematic of a SIG-Block [11]: (A) batteries, (B)
wireless module, (C) accelerometer, (D) reflective optical
sensor, (E) microprocessor
Hardware: SIG-Blocks
Each SIG-Block contains a tri-axial accelerometer
(MMA7260), a microprocessor (ATmega328), six
reflective optical sensors, a wireless module (XBee),
and four AAA batteries. A schematic of a block can be
seen in figure 2. The accelerometer communicates
whether and how much the block is moving and the
optical sensors, which are placed on each face of the
block, indicate whether each block face is covered. The
wireless module is used to transfer the accelerometer
and optical sensor data to the computer. The
rechargeable batteries provide up to 5 hours of battery
life. Each block has a unique ID so the computer is able
to distinguish the blocks from each other and to
reconstruct the 3D relative locations of the blocks.
Physically, the blocks are 2-inch cubes with different
images on each of their six faces. The images can be
changed manually depending on what type of game is
being played. The most commonly used face images
are black and white geometric patterns with 1-, 2-, and
4-fold symmetry [5] and can be seen in figure 1.
Software: Graphical User Interface
OpenGL for animations. Its features include the ability
to select games, display 3D animation of blocks in realtime, perform performance analysis, save data for
reference, and an interface that is attractive and user
friendly. The GUI has two difference appearances based
on who is using it. The first is intended for parents or
professionals that want to look at the collected data
[5,6] and the second is intended for children. The
professional GUI has a simpler appearance, but has
more information and options. The children’s GUI does
not display all the possible information, but has more
playful appearance and features. Figure 3 shows a
layout of the children’s GUI, which comes with a variety
of options for layouts and colors.
The Games
Buttons: exit, settings, help
Three main games have been designed for the initial
trial of TaG-Games. Each is intended to test a different
sub-set of psychological, developmental, and motor
skills, as shown in table 1, as well as be interactive and
fun for the user. A player proficiency at a certain game
as reflected in the time taken to complete it and the
accuracy of their answer. Additional behavioral data,
such as spatial manipulation skills, overall speed of
motions, dominant frequency of hand motions,
repetitive or hyperactive activities, and estimated
manipulation trajectories, can be also obtained by
analyzing the sensor data. The designed games are
called Assembly, Shape-Matching, and Sequence
Memory and are described in detail below.
Game problem display
Real-time assembly
configuration
Pattern size selection
Level selection
Visual feedback
Buttons: start, stop
figure 3. Layout of children’s GUI: a player can choose the
background image from several options.
The GUI receives data from the blocks and interprets it
in order to provide the user with feedback on their
performance. It is written in Visual C# and utilizes
Types
Assembly
Associated Cognitive Skills
Fine-motor proficiency; Visual-motor
integration
Shape
Matching
Cognitive problem solving; Conceptual
reasoning; Visual-motor integration;
Working memory
Sequence
Memory
Working memory; Visual-motor
integration; Fine-motor proficiency
table 1. Three types of TaG-Games with related cognitive
skills and behavioral data available from TaG-Games.
figure 4. Assembly game examples of
2x2 and 3x3 games patterns with and
without dividing lines
figure 5. Shape-Matching game
examples of 2x2 and 3x3 patterns
games
Assembly Game
The assembly game is a puzzle game in which the user
is trying to recreate a displayed image using the SIGBlocks. The user is given a pattern like those seen in
figure 4 and they must rotate and rearrange the blocks
to find the correct images in order to recreate the
pattern. The difficulty and interest of this task comes
from the manipulation of the blocks in the hands to find
the correct images, the size of the pattern, and the
distinguishability of the images on the block faces
within the image. The distinguishability of the patterns
can be decreased in order to make them more difficult
by removing the lines separating the pattern
components. This change results in the big picture
being more prominent in the patterns then the
individual block face images. Figure 4 demonstrates
this by showing the same three patterns without and
with dividing lines in the first and second rows,
respectively. Most noticeably, the far right matched
image looks like a ‘W’ when viewed without dividing
lines but when lines are added in, it becomes clear that
it is simply made up of four triangles.
Shape Matching Game
The shape matching game is a problem solving game in
which the user tries to figure out the visual
relationships within an image in order to complete
them. It is played by placing a SIG-Block with the
correct image in the correct orientation face up. Some
example tasks can be seen in figure 5. From these it
can be seen that a variety of methods need to be used
in order to figure out what goes in the blank location.
The most basic is by symmetry (or a mirrored image),
as seen in the top left about the vertical axis. Rotation
is important of solving the top middle and bottom left.
For the top right, the compliment of the image must be
taken. And, finally, for the bottom left the difference
between images within rows in used to solve for the
solution.
Sequence Memory Game
The sequence game is a memory game and involves
remembering a sequence of images and then replaying
them back using a SIG-Block. The sequences can be
displayed by showing all the images at once for a
certain amount of time or each image can be flashed
one at a time in sequence. The player replays the
image sequence by rotating a block and placing it with
the correct face up sequentially. Two different face
image pattern sets have been used; a set of six
different colors (red, blue, green, yellow, purple, and
white) and the set of geometric patterns used in the
other two games. The difficulty of remembering a
sequence is related to what image set is used, the
length of the pattern, and the number of repeating
elements within the sequence.
Other Game Ideas
The three TaG-Games described above are designed
primarily for performance assessment. However, SIGBlocks can function as a platform technology that can
be used for a wide range of fun games for entertaining
and educational purposes. Currently, there are two
additional games that we have begun creating. The first
figure 6. Maze game example
is a maze traversal game for which the player is
presented with a maze that can be navigated by
rotating a SIG-Block in the direction they wish to travel.
This game is mostly just for fun without any
assessment uses, but it does require working memory
and problem solving skills in addition to good fine
motor control. The game currently can be played as a
single player or a multiple player game with players
competing to get to the goal location. The mazes are
randomly generated and have the options to
incorporate extra points for visiting certain locations in
the maze other than the goal location, sound effects for
moving different directions, and a choice of cursor
pictures. A sample maze can be seen in figure 6. The
second game we have developed is called ShakeShake
and is intended to measure the response time to
stimuli. It is played by shaking SIG-Blocks in each hand
to response as fast as possible to visual and audio
signals. The game interface with a visual stimulus can
be seen in figure 7.
Technology Evaluation
figure 7. ShakeShake game example
Evaluation of our system takes place in two parts; the
first is to test its assessment capabilities to see how
well different items correlate with different skills and
the second is to test children’s interest and response to
it. In order to test its assessment capabilities, we are
currently running a human subject study on young
adults for initial feasibility evaluation. The study
involves performing our three games, Assembly, Shape
Matching, and Sequence Memory, as well as the Block
Design, Digit Span, and Matrix Reasoning subtests of
the Wechsler Adult Intelligence Scale. The Wechsler
Intelligence Test is well studied and each of the subtests has been correlated with different cognitive
abilities [11,13]. By comparing the player’s
performance on the Wechsler Intelligence test and our
own games, we will be able to demonstrate the ability
of our games to measure certain cognitive skills.
Currently, we have tested 110 subjects of our goal of
150 subjects. The subjects are between 18 and 30
years of age and are all members of the Case Western
Reserve University community. After this feasibility
evaluation, the games with a proper set of comparison
modules (e.g., Wechsler Intelligence Scale for Children)
will be tested on young children. We will also assess
children’s response to the games for tuning the game
items and improving the GUI design. One of the
foreseeable advantages of our technology is that it will
be fun for children and thus an easier way to test the
cognitive capabilities of children. Therefore, it is
important that children find it engaging.
Discussion
With every prototype there are parts that do not work
as well as could be hoped for. In this case, we have a
couple of technical concerns that will be addressed in
the next generation. The first is the connection sensing
mechanism. Optical sensors have the advantages of
being simple, low energy consumption, and accurate in
detection, but are set off by anything covering them.
Any object that is close to the sensor will reflect the
emitted light and set off the sensor, meaning that it will
be set of not only by other blocks, but also by a
person’s hands as they are manipulating the blocks.
Our algorithm combines optical sensor data with the
accelerometer to detect assembly among the blocks,
but about an error rate of 2% was still caused by false
detection in optical sensors. The second concern is the
weight of the blocks. Currently, they weigh 350 grams,
which feels heavy for a 2-inch cube and may be too
much for a young child to handle comfortably. The third
problem is that the orientation of blocks can only be
detected relative to each other and not relative to the
some global frame of reference. In order to play games
with a single block, spatial orientation of the block
would be useful.
[3] First L, Palfrey J. 1994. The infant or young child
with developmental delay. The New England Journal of
Medicine, 330(7): 478-483.
Conclusion and Future Work
[5] Jeong, D., Kerci, E. and Lee, K. 2010. Sensorintegrated geometric blocks: towards interactive playbased assessment of young children. Intl. Workshop on
Interactive Systems in Healthcare.
We have presented TaG-Games as an alternative or
supplementary assessment tool for cognitive, working
memory, and fine motor control skills with potential
advantages of not requiring oversight and being
enjoyable to play. The game control interface, called
SIG-Blocks, was introduced and the three games,
Assembly, Shape Matching, and Sequence Memory,
were described to show the different types of skills
being test for. Currently, we are conducting a
preliminary evaluation study involving human subjects
to validate our design and demonstrate its usefulness
as an assessment tool. We are also developing the next
generation of SIG-Blocks addressing the technical
problems identified in the current design.
Supporting Web Documents
http://www.case.edu/mae/robotics
Acknowledgements
This project is funded by National Science Foundation
under the REESE Program (Award No. 1109270).
Citations
[1] Atkins M. 2004. Early childhood computer
experience and cognitive and motor development.
Pediatrics, 113(6): 1715-1722.
[2] Christakis D, Ebel B, Rivara F, Zimmerman F. 2004.
Television, video, and computer game usage in children
under 11 years of age. J. Pediatr., 145(5):652-656.
[4] Jacobs K, Baker NA. 2002. The association between
children's computer use and musculoskeletal
discomfort. Work 18(3): 221-6.
[6] Jeong, D. Kerci, E., and K. Lee. TaG-Games:
tangible geometric games for assessing cognitive
problem-solving skills and fine motor proficiency," IEEE
MFI 2010, pp. 32-37.
[7] Kastc M, Meyera M, Vogelic C, Markus F. 2007.
Computer-based multisensory learning in children with
developmental dyslexia. Res. Neu. & Neuroscience,
25(3-4): 355-369.
[8] Kraut R, Greenfield P, Gross E. 2000. The impact of
home computer use on children's activities and
development. Future Child, 10(2): 123-144.
[9] Räsänen P, Salminen J, Wilson AJ, Aunio P,
Dehaene S. 2009. Computer-assisted intervention for
children with low numeracy skills. Cog. Dev., 24(4):
450-472.
[10] Serin O. 2011. The effects of the computer-based
instruction on the achievement and problem solving
skills of the science and technology students. Turk
Online J. Educ. T., 10(1): 183-201.
[11] Silverstein A. 1985. 2-subtest and 4-subtest short
forms of the WAIS-R: A closer look at validity and
reliability. J. Clin. Psy., 41(1): 95-97.
[12] Straker L, Pollock C, Maslen B. 2009. Principles for
the wise use of computers by children. Ergonomics,
52(11):1386-1401.
[13] Wechsler, D. 1955b. Manual for the Wechsler Adult
Intelligence Scale. New York. The Psychological
Corporation.