Kinect in Greenfoot
Lecture 4. Greenfoot
1
Pablo Romero, Department of Informatics
Lecture plan
Installing kinect drivers
 The sample scenarios



Kinect in Greenfoot
The Painting scenario
Modifying the code
2
Pablo Romero, IIMAS, UNAM
The scenarios
Painting
 Ball
 Depth
 Mirror
 Trail
 Xray

Kinect in Greenfoot
3
Pablo Romero, IIMAS, UNAM
The Painting scenario

World Classes

KinectWorld
• PaintWorld

Actor Classes
Instructions
 Label
 Brush
 Thumbnail

Kinect in Greenfoot

Other Classes
Joint
 KinectClient
 UserData
 Point3D

4
Pablo Romero, IIMAS, UNAM
KinectWorld


The basic world class for kinect scenarios
Make a subclass for the specific scenario



Methods





Kinect in Greenfoot
super(…) in constructor
super.act() in act()
isConnected()
getTrackedUsers()
getAllUsers()
getThumbnail()
getDepthAt(int x, int y)
5
Pablo Romero, IIMAS, UNAM
KinectClient
Establishes communication and fetches
data from the kinect
 Used by the world classes (kinectWorld
and its specific subclass)
 Methods

IsConnected(), update(), disconnect()
 getUsers()
 getThumbnail()

Kinect in Greenfoot
6
Pablo Romero, IIMAS, UNAM
UserData

Holds data from the kinect for one user

id, state, an array of joints, image, etc.
getTrackedUsers(), getAllUsers() from
kinectWorld return an array of UserData
 Used frequently by other classes
 Methods

drawStickFigure, getAllJoints,
getNearestJoint, getHighestJoint
 getters and setters for its fields

Kinect in Greenfoot
7
Pablo Romero, IIMAS, UNAM
Joint
Holds data for one joint as a Point3D
 15 different types of joints
 Used by UserData but also frequently by
other classes
 Methods


Kinect in Greenfoot
getX(), getY(), getJointIndex()
8
Pablo Romero, IIMAS, UNAM
So far

Greenfoot environment provides a
connection to use kinect
KinectWorld and KinnectClient main classes
provided to interact with kinect
 Particular application world created as a
subclass of KinectWorld
 UserData, Joint and Point3D also useful, reusable classes

Kinect in Greenfoot
9
Pablo Romero, IIMAS, UNAM
The design
Informal presentation
 Game should be challenging

Without kinect (with xbox control, intellectual
part)
 Only with physical part
 Both intellectual and physical


Kinect in Greenfoot
Intellectual and physical well integrated
10
Pablo Romero, IIMAS, UNAM
Exercise

Now change the program so that a the
person’s image and not the stick figure
is displayed

In the act
•
•
•
•
Kinect in Greenfoot
Comment drawing the stick figure
Create a user image (getCombinedUserImage)
Scale that image to the width and height
Draw that image (after drawing the
background)
11
Pablo Romero, IIMAS, UNAM
The brush

In the constructor


Creates a green dot as a greenfoot image
In the act
Positions and paints the dot on the right
hand’s coordinates
 If the left hand is higher than the head clears
the screen

Kinect in Greenfoot
12
Pablo Romero, IIMAS, UNAM
Exercise

How would you change the behaviour
of the brush so that it paints as if the
image had been flipped both vertically
and horizontally?

In Brush.java
• Modify the coordinates of the brush so that
they are deducted from the dimensions of the
world (getWidth() and getHeight() from World)
Kinect in Greenfoot
13
Pablo Romero, IIMAS, UNAM
Exercise

How would you change the behaviour
of the brush so that what it paints falls
down the screen until it disappears?
This can be done independently from
the previous exercise

In the act of Brush.java
• Create an arrayList to store the coordinates of
each position of the right hand
• Increment each of the y coordinates in the
arrayList every time to make them fall. If they
go out of the screen delete them
Kinect in Greenfoot
14
Pablo Romero, IIMAS, UNAM
Abstraction and modularization
Modularization is the process of dividing
a whole into well-defined parts, which can
be built and examined separately, and
which interact in well-defined ways.
 Abstraction is the ability to ignore details
of parts to focus attention on a higher
level of a problem.

Kinect in Greenfoot
15
Pablo Romero, IIMAS, UNAM
Falling drawings
Create a class for the coordinates
 Create an ArrayList to a group of
coordinates

Declare the ArrayList at the beginning of
Brush.java
 Initialise it in the constructor
 Store the coordinates in the act method

Modify the Y values of the ArrayList
 Redisplay the modified coordinates

Kinect in Greenfoot
16
Pablo Romero, IIMAS, UNAM
Big exercise

Write a simple game that requires
people to remember a set of postures.
The game could consist of two parts:
A program to record the set of postures to
use in the game
 The game proper where the set of
postures recorded before are compared
with the user’s postures

Kinect in Greenfoot
17
Pablo Romero, IIMAS, UNAM
Recording postures
Use the starting program for postures. This
program tracks a user, displays the instructions
of the game and a white background. It has a
PostureWorld, subclass of KinectWorld, that
does those three tasks. Additionally it has a
Posture, subclass of Actor that has the user’s
image as its image and displays it
 Give the user a 3 sec. warning to make a pose
and make a beep sound
 Record the postures in a file as a set of joint 2D
coordinates

Kinect in Greenfoot
18
Pablo Romero, IIMAS, UNAM
The second step
if (leftHandUp == -1) {
leftHandUp = System.currentTimeMillis();
} else if (System.currentTimeMillis() - leftHandUp > 5000)
postureWarning = new Label("3",100);
import java.awt.Toolkit;
…
Toolkit.getDefaultToolkit().beep();
Kinect in Greenfoot
19
Pablo Romero, IIMAS, UNAM
Handling errors
Not always programmer’s error
 File processing particularly error-prone

Missing or corrupt files
 Lack of appropriate permissions


Errors of this type have to be handled and
reported (exception handling)

Kinect in Greenfoot
Done with the try statement
20
Pablo Romero, IIMAS, UNAM
The try statement

Code catching an exception must protect
the call with a try statement:
try {
Protect one or more statements here.
}
catch(Exception e) {
Report and recover from the exception here.
}
Kinect in Greenfoot
21
Pablo Romero, IIMAS, UNAM
The try statement
1. Exception thrown from here
try {
addressbook.saveToFile(filename);
tryAgain = false;
2. Control transfers to here
}
catch(IOException e) {
System.out.println("Unable to save to " + filename);
tryAgain = true;
}
Kinect in Greenfoot
22
Pablo Romero, IIMAS, UNAM
Text input-output

Input-output is particularly error-prone.

It involves interaction with the external
environment.
The java.io package supports inputoutput.
 java.io.IOException is a checked
exception.

Kinect in Greenfoot
23
Pablo Romero, IIMAS, UNAM
Text output to file

Use the FileWriter class.
Open a file.
 Write to the file.
 Close the file.


Kinect in Greenfoot
Failure at any point results in an
IOException.
24
Pablo Romero, IIMAS, UNAM
Text output to file
try {
FileWriter writer = new FileWriter("name of file");
while(there is more text to write) {
...
writer.write(next piece of text);
...
}
writer.close();
}
catch(IOException e) {
something went wrong with accessing the file
}
Kinect in Greenfoot
25
Pablo Romero, IIMAS, UNAM
The postures game
Displaying the postures
 Comparing the player’s posture to the
recorded posture

Kinect in Greenfoot
26
Pablo Romero, IIMAS, UNAM
Displaying the postures

Different program but can use the starting
program

The Posture world class responsible for the
overall game
• Reading the data file to fill in an ArrayList of
UserData
• Displaying the postures

Kinect in Greenfoot
The Posture class responsible for drawing
the stick figure and changing the posture
27
Pablo Romero, IIMAS, UNAM
Text input from file
Use the FileReader class.
 Augment with BufferedReader for linebased input.

Open a file.
 Read from the file.
 Close the file.


Failure at any point results in an
IOException.

Use the split string method
Kinect in Greenfoot
28
Pablo Romero, IIMAS, UNAM
Text input from file
try {
BufferedReader reader =
new BufferedReader(new FileReader("filename"));
String line = reader.readLine();
while(line != null) {
do something with line
line = reader.readLine();
}
reader.close();
}
catch(FileNotFoundException e) {
the specified file could not be found
}
catch(IOException e) {
something went wrong with reading or closing
}
Kinect in Greenfoot
29
Pablo Romero, IIMAS, UNAM
Filling in the ArrayList
Use the UserData class as it can draw the
stick figure.
 Fill in an ArrayList of UserData

Just fill in the joints
 Make the positionWorld and positionScreen
the same
 The z coordinate can be missing
 Confidence can be 0

Kinect in Greenfoot
30
Pablo Romero, IIMAS, UNAM
Displaying the postures
In the act of PostureWorld (remember the
act is within a loop)
 Use a field to keep the index of the
current posture and to avoid out of
bounds
 Use another field to indicate whether it is
time to change the posture (similar to the
one to keep the elapsed time when
warning the user to make a pose)

Kinect in Greenfoot
31
Pablo Romero, IIMAS, UNAM