SM1205 Interactivity
Topic 10:
Motion Tracking Part II
Spring 2010
SCM-CityU
1
Motion Tracking Library
• In cityu.scm package (developed by Hongbo Fu)
– Webcam class
• An easy-to-use wrapper for webcam access
import cityu.scm.Webcam; // import Webcam class
// create a webcam with size 400 x 300
var webcam:Webcam = new Webcam(400, 300);
// webcam will dispatch READY event when the webcam is ready to use
webcam.addEventListener(Webcam.READY, onCameraReady);
function onCameraReady(e:Event): void {
addChild(webcam);
}
Spring 2010
SCM-CityU
2
Motion Tracking Library
• In cityu.scm package (developed by Hongbo Fu)
– MotionTracker class
var tracker:MotionTracker=new MotionTracker(webcam);
Spring 2010
SCM-CityU
3
MotionTracker Class
• Properties
– trackX: Number; trackY: Number
• Center of tracked region
• With respect to top-left corner of webcam (instead of stage)
• Methods
(0, 0)
– hitTestObject
– hitTestPoint
– hideTrackedArea
– hideTrackedCenter
Spring 2010
(trackX, trackY)
Spring 2010
SCM-CityU
4
4
Demos
Spring 2010
SCM-CityU
5
Clipping Regions
• By default, motion tracking is done over the entire
video domain
– Check MotionTracker-Skeleton.fla
Spring 2010
SCM-CityU
6
Clipping Regions
• What if motion only within specific regions (called
clipping regions) are needed?
– Motion outside those clipping regions will be ignored
• Demo
clipping regions
Spring 2010
SCM-CityU
7
Clipping Regions
• Specify a clipping region when creating a motion tracker
tracker = new MotionTracker(webcam,
cl, ct, cr, cb);
– cl, ct, cr, cb are with
respect to top-left corner
of webcam
– trackX, trackY are also with
respect to top-left corner
of webcam
– You can have only ONE
clipping region for
one tracker
Spring 2010
ct
cb
cl
cr
SCM-CityU
Clipping Region
8
Multiple Clipping Regions
• To have multiple clipping regions, you can create
multiple trackers
– All trackers share the same Webcam instance
– Each tracker has its own associated clipping region
// initialize 1st motion tracker
tracker = new MotionTracker(webcam, 0, 0, 200, 200);
// initialize 2nd motion tracker
tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);
Spring 2010
SCM-CityU
9
Class Exercise
• Extend the previous example to have two clipping
regions as follows
– Hint: create two trackers with their own clipping regions
– How trackers work at the overlapped areas of clipping regions?
Spring 2010
SCM-CityU
10
Example: Ping-Pong Game
ball
box2
barL
barR
box
Spring 2010
SCM-CityU
11
Example: Ping-Pong Game
• Use tracker and tracker2 to control barL and barR,
respectively
Spring 2010
SCM-CityU
12
Class Exercise
• Make ball bounce against stage borders
– Change ball’s x and y properties continuously in onTimer
event listener
– Check if ball hits
stage borders
Spring 2010
SCM-CityU
13
Hit Test between Ball and Bars
• Let’s just use hitTestObject method
if (ball.hitTestObject(barR)) {
trace("barR got hit");
}
if (ball.hitTestObject(barL)) {
trace("barL got hit");
}
Spring 2010
SCM-CityU
14
Hit Test between Ball and Bars
• Collision response
Case 2:
Ball is on right side
of barR
Case 1:
Ball is on left side
of barR
Spring 2010
SCM-CityU
15
Hit Test between Ball and Bars
• But how to determine which case to handle?
• Solution: use signs of ball’s speedX
– speedX > 0  Case 1
• Response: put ball
exactly at the left side
of barR
– speedX < 0  Case 2
• Response: put ball
exactly at the right side
of barR
Spring 2010
SCM-CityU
16
Class Exercise
• Add sound effects when ball hits stage borders/bars
– You can use Flash built-in sounds
• Hints
– Export sounds for
ActionScript
var ws:WoodSound = new WoodSound();
ws.play();
Spring 2010
SCM-CityU
17
Moving Speed
• Use MotionTracker’s speedX and speedY to get the
moving speed of tracked object
trace(tracker.speedX, tracker.speedY);
• speedX (in pixels)
– Positive value if object moves along positive x-axis
– Negative value if objects moves along negative x-axis
• speedY (in pixels)
– Positive value if object moves along positive y-axis
– Negative value if objects moves along negative y-axis
Spring 2010
SCM-CityU
18
Example
• Transfer moving speed of tracked object to ball
– E.g., hand
• Demo
Spring 2010
SCM-CityU
19
Example
• Let’s modify the previous ping-pong example
– Keep one tracker only, without any clipping region
– No vertical bars
– Add friction
Spring 2010
SCM-CityU
20
Example
• Change ball’s speeds only if there is some intersection
between ball and moving object
if (tracker.isTracked() &&
tracker.hitTestPoint(ball.x, ball.y, true)) {
speedX += tracker.speedX;
speedY += tracker.speedY;
}
Spring 2010
SCM-CityU
21
Example
• Interact with more balls (demo)
Spring 2010
SCM-CityU
22
Assignment 3
• Topic: Webcam Interaction with Motion
• Requirements
– Use motion tracker
– Interact with motion
– Have music/sound elements
– Meaningful system
– Documentation (in Word format)
• Title, motivation, how-to-use, screenshots, references
Spring 2010
SCM-CityU
23
Assignment 3
• Submission
– Via ACS
• Max total file size: 50M
• .fla, .swf, .doc (documentation) files and other files
– Deadline: 23:59:59, 12 May (Wed), 2010
• No late submission is allowed
• In-class presentation, 12 May, 2010
– Each student has roughly 5 minutes for presentation
Spring 2010
SCM-CityU
24
Assignment 3
• Assessment
– Based on
• Implementation difficulty + UI design + creativity (80%)
• Presentation performance (10%)
• Documentation (10%)
– Note: if you use any technique which has not been
taught in this course, you must state its source or
reference in the documentation clearly
• Otherwise, some marks will be deducted
Spring 2010
SCM-CityU
25
Summary of MotionTracker
• Make sure your Flash application is under the
same folder as the tracking library
Spring 2010
SCM-CityU
26
Summary of MotionTracker
• Without clipping region
tracker = new MotionTracker(webcam);
• With clipping region
tracker = new MotionTracker(webcam,
cl, ct, cr, cb);
– cl, ct, cr, cb are with respect to top-left corner
of webcam
Spring 2010
SCM-CityU
27
Summary of MotionTracker
• Properties
– trackX, trackY: center of tracked region
• With respect to top-left corner of webcam
(0, 0)
– speedX, speedY
• Moving speed of
tracked object
(trackX, trackY)
Spring 2010
28
Spring 2010
SCM-CityU
28
Summary of MotionTracker
• Methods
– hitTestObject
• To test if bounding boxes of display object and tracked
region intersect with each other
– hitTestPoint
• To test if a given point is within the tracked region
– isTracked
• To check if any moving object is tracked
– hideTrackedArea
– hideTrackedCenter
Spring 2010
SCM-CityU
29