Reflection for Tools Development
Jeremy Walker
Senior Tools Programmer
Ubisoft Vancouver
Table of Contents
• Understanding the Costs of Tools Development
• The Ultimate Technique for Data Driven Solutions
• A Reflection-Powered Infrastructure for Tools
Hard Coded vs. Data Driven Systems
Data Driven System
Hard Coded System
Animation
Animation
Animation Tools
Animation Build System
Hard Coded
Animation System
Animation Serialization
Animation Engine
What's the Cost of Great Workflows?
Sounds
Animation
Scenes
Tools
SoundAnimation
Editing Tools
Scene Editing Tools
Sound
Build System
Animation
Build System
Scene Build System
Animation
Serialization
Sound
Serialization
Scene Serialization
Sound Playback
AnimationEngine
Engine
Scene Rendering Engine
Great Workflows for All Types of Content
Animation
Sounds
Scenes
SHARED
SHARED
SHARED
Textures
AI
Levels
SHARED
Input
Front End
Cameras
Particles
SHARED
SHARED
SHARED
Models
Shaders
Materials
HUD
SHARED
SHARED
SHARED
SHARED
Curves
Videos
Skeletons
Music
Sequences
State Flows
SHARED
SHARED
SHARED
SHARED
SHARED
Characters
Props
SHARED
Scripts
Cut Scenes
SHARED
The Keep-It-Simple Approach
Animation
Sounds
Scenes
Input
Front End
Cameras
Particles
Characters
Hard
Coded
Hard
Coded
Hard
Coded
SHARED
SHARED
SHARED
Shaders
Materials
HUD
Props
Hard
Coded
Hard
Coded
Hard
Coded
SHARED
SHARED
SHARED
Textures
AI
Levels
Models
Hard
Coded
SHARED
SHARED
SHARED
SHARED
SHARED
Videos
Skeletons
Music
Sequences
State Flows
Curves
Hard
Coded
Hard
Coded
Hard
Coded
Hard
Coded
Hard
Coded
SHARED
SHARED
SHARED
SHARED
SHARED
SHARED
Scripts
Cut Scenes
SHARED
The Monolithic Engine Approach
Animation
Sounds
Scenes
Monolithic Engine
Input
Front End
Cameras
Particles
Characters
Materials
HUD
Props
Videos
Scripts
Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras,
Particles, Characters, Textures, AI, Levels, Models, Shaders,
Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Textures
AI
Levels
Animation,Models
Sounds, Scenes, Input, FrontShaders
End, Cameras,
Particles, Characters, Textures, AI, Levels, Models, Shaders,
Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras,
Particles, Characters, Textures, AI, Levels, Models, Shaders,
Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Skeletons
Music
Sequences
State Flows
Curves
Animation, Sounds, Scenes, Input, Front End, Cameras,
Particles, Characters, Textures, AI, Levels, Models, Shaders,
Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
The Problem
Low
Development
Cost
Game and
Genre-Specific
Needs
Adaptable
To Change
Reusable
Systems
Stable
Tools
Great
Workflows
The Solution
For all types of content:
• Minimize the cost of developing great workflows.
• Design reusable systems while accommodating gamespecific needs.
• Develop stable tools that adapt to constant change
during production.
The Package Release Cycle
Source
Code
Release
Candidate
Release
yes
yes
Programmer
Modifies Code
Automated
Build Process
OK?
no
Testing
Process
OK?
no
Artist Gets
Package
Release
Artist
Requests New
Features
The Monolithic Package Release Problem
Monolithic
Package
Source Code
Monolithic
Package
Release
The Benefit of Decoupled Packages
Package
Source Code
Package
Release
Package
Source Code
Package
Release
Package
Source Code
Package
Release
Package
Source Code
Package
Release
Package
Source Code
Package
Release
The Lockstep Release Problem
Animation Tool
Source Code
Animation
Tool Release
Compile Time
Dependency
Run Time
Dependency
Sound Tool
Source Code
Compile Time
Dependency
Game Source
Code
Lockstep
Release Cycle
Sound Tool
Release
Run Time
Dependency
Game Release
Reducing Cost and Complexity
Hardcoded Engine
Monolithic Engine
Decoupled Systems
Animation
Animation
Sounds
Sounds
Scenes
Scenes
Animation, Sounds, Scenes, Input, Front End,
Cameras, Particles, Characters, Textures, AI, Levels,
Models, Shaders, Materials, HUD, Props, Skeletons,
Music, Sequences, State Flows, Curves, Videos,
Scripts, Cut Scenes
Cameras
Particles
Cameras
Particles
Character
s
Character
s
Animation, Sounds, Scenes, Input, Front End,
Cameras, Particles, Characters, Textures, AI, Levels,
Models, Shaders, Materials, HUD, Props, Skeletons,
Music, Sequences, State Flows, Curves, Videos,
Scripts, Cut Scenes
Materials
HUD
Props
Materials
HUD
Props
Animation, Sounds, Scenes, Input, Front End,
Cameras, Particles, Characters, Textures, AI, Levels,
Models, Shaders, Materials, HUD, Props, Skeletons,
Music, Sequences, State Flows, Curves, Videos,
Scripts, Cut Scenes
Hard Coded
Animation, Sounds, Scenes, Input, Front End,
Cameras, Particles, Characters, Textures, AI, Levels,
Models, Shaders, Materials, HUD, Props, Skeletons,
Music, Sequences, State Flows, Curves, Videos,
Scripts, Cut Scenes
Low Cost
Simple
Hard to Reuse
Poor Workflow
Medium Cost
Complex
Reuse All or Nothing
Great Workflow
Reflection
Low Cost
High
VerySimple
Complex
Very
Very Reusable
Reusable
Great Workflow
Workflow
Great
What is Reflection?
From Wikipedia:
“…Reflection is the process by which a
computer program can observe and modify
its own structure and behaviour.”
Reflection is Used Everywhere
Case Study: C++ Compilation
Discover
Function B
calls
Function A
C++
Compiler
Validate
Function
Declaration
Reflection
Binding to
Function
Definition
Expose
C++
Compiler
Implement
Function A
Reflection for Language Interoperability
Discover
C# Code
C#
Compiler
Validate
Type
Definitions
Reflection
Type
Binding
Expose
Visual Basic
Compiler
Implement
Visual Basic
Code
Reflection for Scripting in Games
Discover
Lua Script
Script
Adapter
Validate
Type
Definitions
Reflection
Type
Binding
Expose
C++
Adapter
Implement
C++ Classes
Reflection for Serialization in Games
Discover
Serialized
Objects
Serializer
Validate
Type
Definitions
Reflection
Type
Binding
Expose
C++
Adapter
Implement
C++ Classes
Common Language Specification
Type Definitions
Reflection
Type Binding
•Type size and alignment
•Allocate objects
•Native types
•Binary representation
•Classes
•Invoke constructor, get/invoke members
•Enums
•Get/set enum value
Reflection in Action
Example: Print all fields of type “float”
const Type* type = TypeOf<SimpleVehicle>();
const char* name = type->Name();
size_t size = type->Size();
printf(“Type: %s\n Size: %d\n”, name, size);
Instance vehicle = type->CreateInstance();
vehicle.As<SimpleVehicle&>().m_MaxSpeedKPH = 193.12f;
for (int i=0; i<type->NumMembers(); i++)
{
const Member* member = vehicle.GetMember(i);
if (member->IsField())
{
const Field* field = (Field*)member;
if (field->GetType() == TypeOf<float>())
{
Instance value = vehicle.GetField(field);
printf(“Field: %s\n Offset: %d\n Value: %.2f\n”,
field->Name(), field->Offset(), value.As<float>());
}
}
}
Output
Type: SimpleVehicle
Size: 4
Field: MaxSpeedKPH
Offset 0
Value: 193.12
Popular Approaches to C++ Reflection
• Macros
• Code Parser
• Type Definition Language
Reflection with Macros
Macros Example
class SimpleVehicle : public Entity
{
public:
DECLARE_TYPE();
float m_MaxSpeedKPH;
void
Reset(bool useDefaults);
float GetMaxSpeedMPH() const;
void
SetMaxSpeedMPH(float maxSpeedMPH);
};
//In a separate .CPP file:
DEFINE_TYPE(SimpleVehicle)
BASE_CLASS(Entity)
FIELD(“MaxSpeedKPH”, m_MaxSpeedKPH)
METHOD(Reset)
PROPERTY(GetMaxSpeedMPH, SetMaxSpeedMPH)
DEFINE_TYPE_END()
Pros
• No external tools
Cons
• Awkward to implement
• Hard to debug
• Run-time discovery
Reflection with Code Parser
Code Parser Example
/// [Class]
class SimpleVehicle : public Entity
{
public:
/// [Field(“MaxSpeedKPH”)]
float m_MaxSpeedKPH;
/// [Method]
void Reset(bool useDefaults);
/// [Property]
float GetMaxSpeedMPH() const;
/// [Property]
void
SetMaxSpeedMPH(float maxSpeedMPH);
};
Pros
• Easier to implement
• Compile-time discovery
Cons
• Slow pre-build step
Reflection with Type Definition Language
Type Definition Language Example
class SimpleVehicle : Entity
{
float MaxSpeedKPH;
void
Reset(bool useDefaults);
float MaxSpeedMPH { get; set; }
};
Pros
• Easiest to implement
• No slow pre-build
Cons
• Can’t reflect existing
classes
Exporting Type Definitions for Game
Generated
Binder Code
C++ Reflected
Types
(game-side)
Reflection System
(game-side)
Code Parser
Types.xml
C++
Class
Types.xml
/// [Class]
<type
name=“SimpleVehicle”>
class
SimpleVehicle
: publictype=“float”/>
Entity
<field
name=“MaxSpeedKPH”
{ <method name=“Reset” returntype=“void”>
public:
<parameter name=“useDefaults” type=“bool”/>
/// [Field(“MaxSpeedKPH”)]
</method>
float
m_MaxSpeedKPH;
<property
name=“MaxSpeedMPH” type=“float” hasget=“true” hasset=“true”/>
/// [Method]
</type>
void Reset(bool useDefaults);
/// [Property]
float
GetMaxSpeedMPH() const;
/// [Property]
void
SetMaxSpeedMPH(float maxSpeedMPH);
};
Exporting Type Definitions for Tools
C++ Reflected
Types
(game-side)
Code Parser
Types.xml
Reflection System
(tool-side)
C#Types.xml
Proxy Class
<type
[ProxyType(“SimpleVehicle”,
name=“SimpleVehicle”> 0x81c37132)]
public
<field
partial
name=“MaxSpeedKPH”
class SimpleVehicle
type=“float”/>
: Entity
{ <method name=“Reset” returntype=“void”>
public
<parameter
floatname=“useDefaults”
MaxSpeedKPH
type=“bool”/>
</method>
{
<property
get { name=“MaxSpeedMPH”
return this.Instance.GetField(“MaxSpeedKPH”).Get<float>();
type=“float” hasget=“true” hasset=“true”/>
}
</type>
set { this.Instance.GetField(“MaxSpeedKPH”).Set<float>(value); } }
}
public float MaxSpeedMPH { ... }
public void Reset(bool useDefaults) { ... }
}
C# Proxy
Generator
Primary Uses for Reflection in Tools
Types.xml
Serialization
Reflection System
Generated UI
(PropertyGrid,
TreeView, etc.)
Client-Server
Remoting
Client-Server Remoting
Tools (Client)
Discover
Generated
C# Proxy
Classes
C# Proxy
Generator
Validate
Type
Definitions
Reflection
Type
Binding
Game (Server)
Expose
Remoting
Client
Implement
Discover
Remoting
Server
Validate
Type
Definitions
Reflection
Type
Binding
Expose
C++ Adapter
Implement
C++ Game
Classes
Problems and Workarounds
Type definitions out of sync
•
•
•
•
Detect type checksum mismatch
Detect problems early
Auto-synchronization of type information
Auto-migration of data
Problems and Workarounds
Tools tightly coupled to the game
•
•
•
Avoid overuse of generated proxy classes
Use generated UI where possible
Use polymorphic proxy classes
Tools
Code
has proxy
Base Class
Derived
Class A
Derived
Class B
Derived
Class C
has no proxy
Problems and Workarounds
Excessive memory usage
•
•
Strip type information based on usage
Auto-detect unused reflected types
Other Uses for Reflection
•
•
•
Marshalling events for multi-processor architectures
Client-Server Remoting for Online
Serialization of saved game data
Content Framework
Tool-side
Game-side
3rd Party
Content Framework
Asset
Repository
Package System
Asset Explorer
Build System
Asset Manager
Asset
Previewing
Reflection
(Demo Videos)
Promising Results
• Fast tools development
• Great workflows for all types of content
• Decoupled systems with improved reusability and
resilience to change
Questions?