Spellotron
Using Kinect for educational purposes
Austin Keener
Faculty Mentor: Dr. Rahman Tashakkori
Department of Computer Science
Appalachian State University
http://cs.appstate.edu/~rt/kinect/
Goal of the Project

Create an interactive and fun environment to help children learn to spell

Promote exercise in the classroom

Start creative process to promote problem solving

Introduces applications of computer science to children
http://cs.appstate.edu/~rt/kinect/
Spellotron is a learning game

Players spell out the words on the screen by making poses that match the
letters of the English alphabet used in the word

Players are given visual clues on how to complete the word

Score is given based on how quickly the player is able to spell the word
http://cs.appstate.edu/~rt/kinect/
How Spellotron works

User selects difficulty level based on desired grade level

A word is displayed on the screen for users to spell

Individual letters are shown representing the current letter in the word

Kinect tracks the player’s pose and determines the correlation of the pose to
the current letter

The game will display the highest correlated pose as the match to the letter

Player learns the spelling of the words while performing physical activity
http://cs.appstate.edu/~rt/kinect/
Modification for Color Blind

Originally the program indicated closeness of the pose of the player to the
correct pose by changing colors in-between green and red.

Proved to be problematic to Color blindness

Added a bar that indicated correct pose to supplement correctness feedback
http://cs.appstate.edu/~rt/kinect/
Technology behind the game

Kinect sensor to track player


Windows Presentation Foundation


The sensor tracks the player through use of an infrared sensor
WPF is used as the main display system of the program.
Implemented using C#

C# was used for access to the Kinect SDK and ease of use.

Also used for easy integration of other systems like WPF.
http://cs.appstate.edu/~rt/kinect/
Program Examples
Grade Level Select

Provides options for Pre-School
through 8th grade spelling.

Each grade level has it’s own
library of words

The text, buttons and pictures
displayed in this window are
arranged using an xml format
understood by WPF.

Each button has an event call when
it is pressed, and is handled by the
parent C# class.
http://cs.appstate.edu/~rt/kinect/
Program Examples
Game Play

The elements on the screen are
arranged using WPF’s xml format

The time is determined by using
the computer’s current time with
the one that is stored when the
new word is displayed

The visual stream from the Kinect
is not handled by WPF’s xml file,
but instead is drawn directly on to
the window.

The bar is used to show how close
the player is to matching the pose.
The bar also works much better
than just color for color blind
players.
http://cs.appstate.edu/~rt/kinect/
Acknowledgements

Dr. Rahman Tashakkori

Jack Jordan

Clint Guin

Appalachian State Department of Computer Science

National Science Foundation (NSF)
Conclusion and Questions
Austin Keener
[email protected]
Additional information and downloads are available at:
http://cs.appstate.edu/~rt/kinect/