Capturing, Encoding and
Rendering Gestures using the
Kinect
Adan Patience
Supervisor: Mr. Mehrdad Ghaziasgar
Co-Supervisor: Mr. Reginald McDonald Dodds
Mentor: Mr. Kenzo Abrahams
Overview
▪ Recap
▪ Design Decisions and System Changes
▪ Interface
▪ Implementation
▪ Tools Used
▪ Project Plan
Recap Introduction
▪ Many countries lack networking infrastructures which
support video chatting applications such as Skype.
▪ Develop towards a video chatting application using the
Kinect, that could possibly be used extensively by the deaf
in the future.
Recap Work Flow
Recap High Level Design
Input
Capture
Encode
Render
Video
Frames
Locate and
Track User
Get Joint
Coordinates
Get Frame
and Read
Coordinates
Write
Coordinates
to File
Inert Key
Frame
Animate
Design Decisions and System Changes
Previously
Currently
▪ No Configuration File
▪ Create Configuration file
▪ Append Coordinates to
tracking file
▪ Overwrite tracking file
containing coordinate points
▪ Used line number in tracking
file to determine frame number
▪ Create new file to store the
frame count
▪ Key frame every 24th set of
coordinates
▪ Key frame every 10th set of
coordinates
▪ No real-time
▪ real-time key framing
User Interface
▪ Setting foundation for the application
▪ Application not at a mature enough state for
a user interface
Implementation
Input
Update
Skeleton
Capture
Encode
Render
Process
Skeleton
Get Joint
Coordinates
and Frame
Count
Configure
User and Get
Frame Count
Draw
Skeleton
Write
Coordinates to
File and Frame
Count
Get
Coordinates
Map
Coordinates
Insert Key
Frame
Animate
Configure User
▪ Assume 3 Positions for the Configuration File
– Rest Arms
– Arms Up
– Spread Arms
Maximum
y-position
centre
shoulder
▪ Coordinates are stored in CONF.csv
– Average of the centre shoulder position is
calculated
– Minimum and Maximum of both x and y
coordinates are determined for both hands.
Minimum
x-position
Minimum
y-position
Maximum
x-position
Map Coordinates
▪ Right Wrist
– 𝑥 = 0 − 6.5 ∗ 𝑃, 𝑤ℎ𝑒𝑟𝑒 𝑃 =
𝑥𝑐𝑜𝑜𝑟𝑑𝑖𝑛𝑎𝑡𝑒 − 𝑥𝑎𝑣𝑔𝑠ℎ𝑜𝑢𝑙𝑑𝑒𝑟 Right Wrist
𝑥𝑚𝑖𝑛 −𝑥𝑎𝑣𝑔𝑠ℎ𝑜𝑢𝑙𝑑𝑒𝑟
– 𝑦 = 0.2 + 9.6 ∗ 𝑃, 𝑤ℎ𝑒𝑟𝑒 𝑃 =
Centre Shoulder
𝑦𝑐𝑜𝑜𝑟𝑑𝑖𝑛𝑎𝑡𝑒 − 𝑦𝑚𝑖𝑛
𝑦𝑚𝑎𝑥 − 𝑦𝑚𝑖𝑛
Left Wrist
▪ Left Wrist
– 𝑥 = 0 + 6.5 ∗ 𝑃, 𝑤ℎ𝑒𝑟𝑒 𝑃 =
𝑥𝑐𝑜𝑜𝑟𝑑𝑖𝑛𝑎𝑡𝑒 − 𝑥𝑎𝑣𝑔𝑠ℎ𝑜𝑢𝑙𝑑𝑒𝑟 -6.5 Units
– 𝑦 = 0.2 + 9.6 ∗ 𝑃, 𝑤ℎ𝑒𝑟𝑒 𝑃 =
𝑥𝑚𝑎𝑥 −𝑥𝑎𝑣𝑔𝑠ℎ𝑜𝑢𝑙𝑑𝑒𝑟
𝑦𝑐𝑜𝑜𝑟𝑑𝑖𝑛𝑎𝑡𝑒 − 𝑦𝑚𝑖𝑛
𝑦𝑚𝑎𝑥 − 𝑦𝑚𝑖𝑛
Origin
6.5 Units
Tools Used
▪ Platform
– Windows 7 x86
▪ Applications
– Blender 2.49b
– Visual Studio 2013
– Kinect SDK
▪ Languages
– Python 2.6
– C++
Project Plan
Term1
Term2
Term3
Term4
• Identify User Requirements, Analyse User Requirements, Literature Survey
• Become Acquainted with OpenCV and C++
• Identify Objects and Classes in the OOD and analyse the Class’ methods and
attributes, Design UI and Choose the Render Application
• Familiarise myself with the Kinect SDK and API
• Create the Capture and Encode Classes leveraging from the Kinect
• Develop the mapping function. Create UI and finalise application
• Test the application against the User Requirements
• Optimise where possible and iron out bugs