Fundamentals of Secure
Development
Dave Weinstein
Security Development Engineer
Microsoft
What is Secure Development?
A process, not a
magic bullet or secret
technology
What is Secure Development?

Security Development Lifecycle

Secure by Design


Secure by Default


The default configuration should be secure; normal
users should not have to go searching through options to
lock things down
Secure in Deployment


If you add security as an afterthought, it is just an
afterthought
Provide tools and guidance for securing the application,
and a solid and easy patch and update path
Communications

Be prepared for the discovery of vulnerabilities and
communicate openly and responsibly with end users
and/or administrators to help them take protective action
SD3+C
What could a major exploited
vulnerability mean?

If player run servers are vulnerable…


If clients searching for games are vulnerable…


No one will play with strangers
If replays, save games, or mods are
vulnerable…


No one will play games
If clients in a game with a malicious client are
vulnerable…


No one will host games
The community may die
If your game is vulnerable…

The franchise may die
Myths and
Misperceptions about
Security
Myths and Misperceptions

People writing malicious code are doing it
to prove they can

People writing malicious code are doing it
primarily for the money. Stolen identities,
botnets, stolen credit card information, and
so on.

The security research community, on the other
hand, has a great many people who investigate
security issues because they find it interesting.
Conflating the researchers who tell you about
security violations with the bad guys who exploit
them is another example of misperception.
Myths and Misperceptions

This is an operating system issue, not an
application issue.

As operating systems become hardened, the
attacks are moving up the application stack.
Malware has been found in the wild exploiting
instant message clients and audio players.
Success in the mass market makes you a
target.
Myths and Misperceptions

This isn’t important enough to miss a ship
date for

“Market seems to act on this information and
punishes a vendor, who on an average, loses
around 0.63% of its market value on the day
a vulnerability is reported in its products.”

[Telang R and S Wattal (2004)]
The most important thing to
realize…
Data is evil
(Validated Data is only mostly evil)
Why Data is Evil



Data affects code execution. If you didn’t
change software behavior based on the data,
you wouldn’t be loading it at all.
Software which naively trusts data from outside
sources is easy prey
Without training programmers defend against
bugs during development, not against
deliberately malformed data in the real world

Even with training, simple mistakes can and will
happen
Why Data is Evil

Small errors can have devastating results

while (*pwszTemp != L'\\')
*pwszServerName++ = *pwszTemp++;

Blaster involved more than 1.5 million
compromised computers, all from a two line
coding error.
Why Data is Evil

Data types are not as safe as you think
they are

Image based vulnerabilities that can result in
remote execution of code have been found
on all major operating systems and across
the application stack.
Code Vulnerability:
Bad APIs
Code Vulnerability:
Bad APIs
 There
are classes of functions
which are commonly misused, and
are extremely prone to security
vulnerabilities.
 strcpy()
 strncpy()
 sprintf()
…
Code Vulnerability:
Bad APIs

Buffer constants are often subject to change,
and the information isn’t necessarily
propogated across the code:




…
fgets( buffer, 256, file );
LogCommand( users[curUser].Name, buffer );
…
void LogCommand( char *user, char *cmd )
{
char logString[128];
sprintf( logString, “User: %s Command: %s”, user, cmd );
…
}
Code Vulnerability:
Bad APIs

What about the “n” functions?
// Code verifies pszSrc is <= 50 chars
 #define MAX (50)
 char *pszDest = malloc(sizeof(pszSrc));
 strncpy(pszDest,pszSrc,MAX);


Human error: sizeof is 4 bytes, not 50
Code Vulnerability:
Bad APIs

What about the “n” functions?
#define MAX (50)
char szDest[MAX];
 strncpy(szDest,pszSrc,MAX);


Latent error: The string not null-terminated
if len(pszSrc) >= MAX
Code Vulnerability:
Bad APIs

What about the “n” functions?
#define MAX (50)
char szDest[MAX];
 strncpy(szDest,pszSrc,strlen(pszSrc));


Human Error: The limiting size is derived from the
source buffer, instead of the destination
Attacks:
An Introduction to the
STRIDE Taxonomy
STRIDE: Spoofing

What it is:


What it can gain:


Impersonating a person or machine
Anything that the impersonated entity can do
Countermeasures:

Authentication
STRIDE: Spoofing

Identification


Authentication


Proof as to their claim of identity
Authorization


Who someone claims to be
Proof of their ability to do something
Do not mistake identification for authentication,
or authentication for authorization
STRIDE: Spoofing

Example: SSL Certificates
An SSL certificate for a web site will confirm
that you are talking to the web site the
certificate was issued for.
 This means that you were not subject to DNS
poisoning, or other means of redirection
 It does NOT mean that the web site the
certificate was issued for is the web site you
SHOULD be talking to. Authentication is not
authorization.

STRIDE: Spoofing

Types of authentication
Something you are
 Something you have
 Something you know


Username / Password


Single factor authentication
Smartcard / PIN

Two factor authentication
STRIDE: Tampering

What it is:


What it can gain:


Altering data or data flows
Injection of false or malicious data
Countermeasures:

Integrity Checking

Integrity checking and prevention of Information
Disclosure are the focus of the Cryptography
portion of this presentation
STRIDE: Repudiation

What it is:


What it can gain:


Denying that an action or transaction took
place
Avoiding the consequences of an action
Countermeasures:

“Non-repudiation services generate evidence
which will convince a disinterested party that
a specific subject performed a specific action”
STRIDE: Repudiation

Repudiation attacks
Primarily involve people making claims, not
software issues
 Can be escalated in some cases to entities
outside the developer or publisher

Credit card companies
 Legal System

STRIDE: Information
Disclosure

What it is:


What it can gain:


Gaining access to information that should not
be available to the attacker
Passwords, account information, personally
identifiable information…
Countermeasures:
Confidentiality
 Encryption

STRIDE: Denial of Service

What it is:


What it can gain:




Stopping a service or system from running
Revenge for perceived slights
The ability to leave a game without “quitting”
Blackmail
Countermeasures:


Availability
Robustness
STRIDE: Elevation of Privilege

What it is:


What it can gain:


Using one level of privilege to gain a higher
level
All the capabilities of the higher level of
privilege
Countermeasures:

Authorization
STRIDE: Elevation of Privilege

Remote execution of code by an
unauthenticated user is the ultimate
Elevation of Privilege.
This is the goal of an attacker.
 UDP sockets that are listening on the Internet
are particularly vulnerable.

STRIDE: Elevation of Privilege

Additional Note: Software should run with
the smallest set of privileges possible.
There is no reason that a game should
require high level privileges to run.

The answer to people trying to break into a
building is not to remove the locks. Or the
doors.
Code Vulnerability:
Format Strings
Code Vulnerability:
Format Strings

What is wrong with this?
…
char buffer[256];
if( strlen( description ) < 256 )
{
sprintf( buffer, description );
}
…
What happens if description consists of
“%u%u%u%u%u%...”?
An Introduction to
Attack Surfaces and
Threat Modeling
Attack Surface

The attack surface of an application is
the set of sources of data it accepts.
Anything that is providing you with data
externally is a potential attack vector.







Game traffic
Matchmaking traffic
Voice traffic
Player provided skins and icons
Player created maps
Game mods
Replays
Attack Surface

A code defect is not a vulnerability unless
there is an open attack surface that lets an
attacker provide data to it.

However, do not patch a vulnerability by simply putting
guard code in the discovered attack surface. There are
powerful binary analysis tools available that make it trivial
to determine the changes in the code. By only providing a
work around for the initially discovered attack surface, you
make it easy for attackers to isolate the vulnerable
component and trace other surfaces that may provide
access to it.
Threat Modeling

Data becomes interesting when it
crosses “trust boundaries”.

Trust boundaries occur whenever you cross
machines, users, privilege levels, or authors.
Loading the official missions
 Loading a player created map
 Loading a player created mod
 Playing a multiplayer game
 Viewing a replay in engine
 Viewing a replay from another user in engine

Threat Modeling

Threat models describe the way the data
flows across the application, and
highlight where the trust boundaries are
crossed.

You can consider the threat model to be a
profiler indicating where you should
concentrate your efforts in securing the
application.
Threat Modeling:
Data Flow Diagrams
• Real People
• News feeds
• Data feeds
• Events
• Notifications
• Etc…
• Services
• Web Services
• Assemblies
• DLLs
• EXEs
• COM object
• Etc…
• Function call
• Network traffic
• Shared
memory
• Etc…
• Database
• File
• Registry
• Shared
Memory
• Queue/Stack
• etc…
FPS: Context Diagram
Leaderboard
Master
Server
List
Client
Local Player
Server
Local Player
Game
Service
Player
Rankings
Remote
Server Admin
FPS: Partial DFD Level 0
Leaderboard
List
Servers
Master
Server
List
Report
Servers
Client
Report
Leaders
Request
Server List
Update
Server List
Request
Server List
Game
Service
Get
Rankings
Player
Rankings
Report Game
Update Rankings
Report Score
Server
FPS: Partial DFD Level 0
Transfer
Mods/Maps
Change
Settings
Report
Settings
Client
Update
Game State
Report
Actions
Get
Settings
Assets Default
Game
Assets
Server
Report
Settings
Maps
Change
Settings
Remote
Server Admin
Manage
Players
Game
Settings
Changed
Game
Functionality
Player
Created
Maps
Player
Created
Mods
Reference: Primary Threat Chart By Asset
S
External Entity
T
X
R
I
D
E
X
X X X X X X
Process
Data Store
Dataflow
X X X X
X
X X
Code Vulnerability:
ASSERT
Code Vulnerability:
ASSERT Considered Harmful
void
Packet::CopyContents( void *buffer, unsigned int maxSize )
{
ASSERT (buffer );
ASSERT( ContentsSize() <= maxSize );
memcpy( buffer, Contents(), ContentsSize() );
}
Code Vulnerability:
ASSERT Considered Harmful

What ASSERT is good for?
Checking for coding errors within the same
trust level
 Checking for code integration defects
 Finding cases where the assumptions of one
part of the code base have been changed


When is ASSERT dangerous?

When it is used to validate untrusted data
sources
A (Very) Basic Introduction
to Cryptography
Basic Introduction to
Cryptography
Cryptography is not a simple subject
 Even using Cryptography effectively is
not a simple subject
 Unless you are a skilled cryptographer,
and you present your findings publicly,
and they hold up over years of analysis,
using cryptographic techniques you
invented is at best only a bad idea...

Basic Introduction to
Cryptography
At the most basic, cryptography involves
secrets; some combination of things that
one or all parties involved know, but that
no one else does
 How effective cryptography is depends
on the strength of the secrets, the way in
which they are used, and the state of
current technology.

Basic Introduction to
Cryptography: Confidentiality


To defend the confidentiality of data, we need to
ensure that only the appropriate people can
decipher the contents.
We could do this with a symmetric key
encryption algorithm. That is to say, both the
encryption and decryption key are the same.

If the key is in fact secure, then only the people with
the key can read the contents, assuming the key is
long enough that brute force cracking attempts are not
usable.
Basic Introduction to
Cryptography: Confidentiality

But is confidentiality the only thing we
care about?

If the data is intercepted, the bad guy may
not be able to read it, but it can certainly be
tampered with.
At best the data that was deciphered would be
useless
 At worst it would look right, but be wrong

Basic Introduction to
Cryptography: Confidentiality

To detect whether the data has been
tampered with, we need a separate way
to validate that the data was unchanged.

MAC (Message Authentication Code)
Generate a hash of the actual data
 Encrypt the hash with a different shared secret


If the MAC matches the data, then it
hasn’t been tampered with.

If it doesn’t match, all you know is that the
data was wrong, you cannot repair the data.
Basic Introduction to
Cryptography: Key Problems

All that the symmetric key encryption
does is verify that the sender was
someone who had the secret key.
If the encryption key is shipped as part of the
software, the potential sender set is arbitrarily
large
 If the encryption key is transmitted as part of
the game session, it is vulnerable to man-inthe-middle attacks

Basic Introduction to
Cryptography: Key Exchange

Public Key Encryption provides a
framework for secure and authenticated
key exchange
In public key encryption, the encryption keys
are asymmetric; each key can encrypt a
message that only the paired key can
decrypt.
 If the public keys of both parties are known,
then the symmetric key used for the session
can be transferred securely.

Basic Introduction to
Cryptography: Key Exchange

Why not just use public key encryption
for the data?


Public key encryption is computationally
intensive (and therefore slow), especially as
compared with symmetric encryption.
What about SSL?

SSL authenticates the remote machine to the
user, it does not authenticate the user to the
remote machine in general usage
Code Vulnerability:
Integer Over/Underflows
Code Vulnerability:
Integer Overflows/Underflows
1 bottle of beer on the wall, 1 bottle of
beer! Take one down, pass it around, 0
bottles of beer on the wall.
0 bottles of beer on the wall, 0 bottles of
beer! Take one down, pass it around,
4294967295 bottles of beer on the wall.
Code Vulnerability:
Integer Overflows/Underflows

Where does this show up in real applications?
int dataSize = packet->Size() – HEADERSIZE;
if( bufferSize > dataSize )
{
memcpy( buffer, &packet->Buffer()[HEADERSIZE], dataSize );
}
What if the size field embedded in that packet is less than
HEADERSIZE?
Code Vulnerability:
Integer Overflows/Underflows

If Chris wants to buy beer by the case,
and each bottle takes 1 unit of storage,
calculate how much storage Chris will
need…
UInt16 CalcBottles( UInt16 numCases )
{
return( numCases * BOTTLESPERCASE );
}
What if numCases is 2731?
Code Vulnerability:
Integer Overflows/Underflows

Ok, so…
UInt16 CalcBottles( UInt16 numCases, bool& overflow )
{
UInt32 storageNeeded = numCases * BOTTLESPERCASE;
overflow = storageNeeded > MAX_UINT16;
return( (UInt16) storageNeeded );
}
Better, but…
Code Vulnerability:
Integer Overflows/Underflows
What happens if the number of cases was
2730?
Bottle* SysAllocBottleStorage( UInt16 bottles )
{
UInt16 bottlesNeeded = (bottles + 31) & ~31;
return( MakeBottleRacks( bottlesNeeded ) );
}
This would only be a problem if memory allocation
functions padded allocation requests to be boundary
aligned.
Which of course, they do.
An Introduction to
Static Code Analysis
Finding High Risk Code

Finding dangerous API usage can be
done via source code analysis, or by
checking the linker information for
imports.
It is not that difficult to grep for strcpy()
 Ensuring that developers don’t immediately
write their own version of the dangerous API
requires training.

Finding High Risk Code

Commercial source code analysis tools
exist to find high risk code patterns
(including arithmetic over/underflows)

Gimpel Lint (PC-lint and FlexeLint)


Note: If using Lint, be very careful with the use of
ASSERT or equivalents to disable Lint warnings
PREfast
Included with Visual Studio 2005 Team Edition
 Driver and Windows Mobile Editions also are
available

Analyzing Changes

Commercial tools exist to analyze and
isolate the differences between two
versions of an application
BinDiff from SABRE Security can also
identify libraries and components that have
been previously analyzed.
 Attackers are in a position to take advantage
of weaknesses that are highlighted in a
patch, or that have been found in other
applications using the same middleware.

An Introduction to
Fuzzing
“Fuzzing”

Fuzzing is the automatic application of
malformed data.

It can range from sending pure garbage or
flipping random bits to detailed, format or
protocol aware intelligent malformation.
“Dumb fuzzing” almost invariably works against
unhardened applications.
 Games are almost invariably unhardened
applications. You do not want to be the low
hanging fruit on the application stack.

“Fuzzing”

Simple dumb fuzzing against all of the trust
boundaries identified in your threat model will go
a long way towards making the application
secure, as well as significantly more robust.



This includes fuzzing the exposed API of any SDK you
make available for mod developers.
We know we will get errors with fuzzed data, what we
care about are access violations, memory spikes
above expected norms, and CPU spikes above
expected norms.
It is more time consuming to figure out if an access
violation is exploitable than it is to fix it, so just fix it.
Integrating Security
into the Development
Process
Integrating Security into the
Development Process
The Principle of Least Privilege
 Automated compiler protections
 Dangerous API / Code Practice Removal
 Attack surface reduction
 Threat modeling
 Fuzzing
 Security Development Lifecycle

The Principle of Least Privilege

No application should ever demand more
privileges than it actually needs to do its
work
As a corollary, if you find yourself running
with additional privileges, and the platform
permits it, rescind them and run minimally
privileged
Automated compiler
protections

There are compiler and Operating
System options which can serve as a
defense in depth



/GS or -fstack-protector or equivalent compiler options put
protection against overwritten return addresses into the code
/SAFESEH is a linker option which puts protection in place
help ensure that the exception handlers are those which
were registered by the developer, not by malicious code
/NXCOMPAT is a compiler option which works with the OS
and the CPU to limit the ability of malicious code to execute
on the stack
Dangerous API / Code
Practice Removal




There are available replacements for dangerous
API elements, and the dangerous functions can
be easily found in existing code.
There are available libraries to check for
arithmetic over and underflows.
There are commercial tools for static code
analysis for dangerous code practices.
Use of these tools will greatly reduce the
vulnerability of your underlying source code.
Attack Surface Reduction


Reducing the Attack Surface makes it harder
for code defects to become vulnerabilities, and
for vulnerabilities to become exploits.
For any exposed attack surface, ask:




Is this used by 80% of the players?
Is it open even when it isn’t in use?
How can access to this be restricted without
impacting play?
Is access to this surface authenticated and
authorized? Are we authenticating the right
component?
Threat Modeling

Do not overcomplicate your Threat Model

If the Threat Model is looking more like a Call
Graph, it is far too complex. A Threat Model
should consist of a context, and at most two
levels below the context (DFD 0 and DFD 1)
There are Threat Model tools freely
available
 Building the Threat Model can be done as
part of Pre-Production


Be sure to update the Threat Model if
assumptions or components change
Fuzzing
There are both free and commercial
fuzzing tools available.
 Basic dumb fuzzing can be implemented
inline in code, simply by randomly flipping
bits in a buffer immediately after reading
it from the data source.
 Apply fuzzing techniques to the high risk
data flows found by the Threat Model.

Reference: The Security
Development Lifecycle
Security Training
Security
Kickoff
Security Security Arch &
Design Attack Surface
Review
Best
Threat
Practices
Modeling
Use Security Create
Development Security
Prepare
Tools &
Docs
Security
Security Best and Tools Response
Dev & Test
For
Plan
Practices
Product
Security
Push
Pen
Testing
Final
Security
Review
Security
Servicing &
Response
Execution
Traditional Software Product Development Lifecycle Tasks and Processes
Feature Lists
Quality
Guidelines
Arch Docs
Schedules
Requirements
Design
Specifications
Functional
Specifications
Design
Testing and Verification
Development
of New Code
Implementation
Bug Fixes
Verification
Code
Signing +
Checkpoint RTM
Express
Signoff
Release
Product Support
Service Packs/
QFEs Security
Updates
Support
&
Servicing
Questions?
References

Writing Secure Code, 2nd Edition



Threat Modeling



Michael Howard and David LeBlanc
Microsoft Press, 2003
Frank Swiderski and Window Snyder
Microsoft Press, 2004
Fuzzing Tools Wiki
http://rtos.trinux.org/secwiki/FuzzingTools

Security Development Lifecycle
http://msdn.microsoft.com/library/en-us/dnsecure/html/sdl.asp

Best Security Practices in Game Development
http://msdn.microsoft.com/library/en-us/directx9_c/Best_Security_Practices_in_Game_Development.asp

Impact of Software Vulnerability Announcements on the Market
Value of Software Vendors – an Empirical Investigation
http://www.heinz.cmu.edu/~rtelang/event_study.pdf
References

Strsafe Library for String Functions
http://msdn.microsoft.com/library/en-us/winui/winui/windowsuserinterface/resources/strings/usingstrsafefunctions.asp

SafeCRT Library
http://msdn.microsoft.com/msdnmag/issues/05/05/SafeCandC/default.aspx

SafeInt Class (C++)
http://msdn.microsoft.com/library/en-us/dncode/html/secure01142004.asp

IntSafe Library (C)
http://blogs.msdn.com/michael_howard/archive/2006/02/02/523392.aspx

Microsoft Threat Modeling Tool
http://msdn.microsoft.com/security/securecode/threatmodeling/default.aspx