Verification
Verification of
of Cyber-Physical
Cyber-Physical
Software
Software Systems:
Systems: Challenges
Challenges
and
and Recent
Recent Advances
Advances
Sanjit A. Seshia
EECS Department
UC Berkeley
Students:
S. Jha, J. Kotker, D. Sadigh, J. Ferguson, S. Jain, M. Xu, A. Chan
Collaborators: A. Rakhlin, A. Tiwari
August 2012
Cyber-Physical
Cyber-Physical Systems
Systems (CPS):
(CPS):
Orchestrating
Orchestrating networked
networked computation
computation
with
with physical
physical systems
systems
Avionics
Building Systems
Transportation
(Air traffic
control at
SFO)
Telecommunications
Automotive
Instrumentation
(Soleil Synchrotron)
E-Corner, Siemens
Power
generation and
distribution
Factory automation
Daimler-Chrysler
Military systems:
Courtesy of Doug Schmidt
Courtesy of
General Electric
–2–
Courtesy of Kuka Robotics Corp.
Cyber-Physical
Cyber-Physical Systems
Systems (CPS):
(CPS):
Orchestrating
Orchestrating networked
networked computation
computation
with
with physical
physical systems
systems
Avionics
Building Systems
Transportation
(Air traffic
control at
SFO)
Telecommunications
Automotive
Instrumentation
(Soleil Synchrotron)
E-Corner, Siemens
Power
generation and
distribution
Factory automation
Daimler-Chrysler
Military systems:
Courtesy of Doug Schmidt
Courtesy of
General Electric
–3–
Courtesy of Kuka Robotics Corp.
Cyber-Physical
Cyber-Physical Properties
Properties

Properties whose specification or verification
requires reasoning about
BOTH cyber and physical components
of the system

Typically involve physical parameters
of the system:
real time, power, energy, velocity, etc.
–4–
Cyber-Physical
Cyber-Physical Properties
Properties -- Examples
Examples
Does the brake-by-wire software
always actuate the brakes within
1 ms?
Safety-critical embedded systems
How much energy must the sensor
node harvest for RSA encryption?
Energy-limited sensor nets,
bio-medical apps, etc.
–5–
Formal
Formal Verification
Verification

Prove or disprove that a system meets its
requirements (formal specification)
System
Model
Environment
Model
VERIFICATION
TOOL
VALID 
ERROR 
Specification
–6–
Automobiles:
Automobiles: A
A Challenging
Challenging Domain
Domain for
for
Verification
Verification of
of Cyber-Physical
Cyber-Physical Systems
Systems
Today’s automobiles
“run on software”
in a
“networked world”
• Nearly 100 million lines of code
• cf. ~ 6.5 million lines of code for Boeing 787
• Running on 70 to 100 networked microprocessorbased electronic control units (ECUs)
[IEEE Spectrum, Feb. 2009]
–7–
Trends
Trends Compound
Compound the
the Challenge
Challenge
Increasing Size
& Complexity of
SOFTWARE
Multicore Processors,
Concurrent Software,
Networking,
Energy-aware architectures, …
Increasing Complexity of
PLATFORM
(ENVIRONMENT)
–8–
Where
Where are
are the
the Problems?
Problems? Viewing
Viewing the
the
Stages
Stages of
of Model-Based
Model-Based Design
Design
Control Design and Modeling
High-level Models
Optimization and Mapping
Low-level Code
Verification of Implementations
Focus of much
of the talk
–9–
Exemplar:
Exemplar: Timing
Timing Analysis
Analysis
Does the brake-by-wire software
always actuate the brakes within
1 ms?
Safety-critical embedded systems
NASA’s Toyota UA report (2011) mentions:
“In practice…there are significant limitations”
(in the state of the art in timing analysis).
CHALLENGE: ENVIRONMENT MODELING
Need a good model of the (complex) platform.
– 10 –
Roadmap
Roadmap for
for the
the Talk
Talk

Why Verification of Cyber-Physical Software is
Challenging
Example: Timing Analysis

Sciduction: A Promising Direction

– Combining Inductive Inference + Deductive
Methods

Future Directions
– Other Stages of Model-Based Design
– 11 –
Roadmap
Roadmap for
for the
the Talk
Talk

Why Verification of Cyber-Physical Software is
Challenging
Example: Timing Analysis

Sciduction: A Promising Direction

– Combining Inductive Inference + Deductive
Methods

Future Directions
– Other Stages of Model-Based Design
– 12 –
The
The Human
Human Aspect
Aspect
Auxiliary Inputs
No Result invariants,
(abstraction,
compositional lemmas, etc.)
System
Model
Environment
Model
DON’T 
KNOW
VERIFICATION
TOOL
VALID 
ERROR 
Specification
DEBUG
– 13 –
Verification
Verification “=”
“=” Synthesis
Synthesis


Often, the hard steps in verification are
“synthesis sub-tasks”
Artifacts synthesized in verification:
– Inductive / auxiliary invariants
– Auxiliary specifications (e.g., pre/post-conditions,
function summaries)
– Environment assumptions / Env model / interface
specifications
– Abstraction functions / abstract models
– Ranking functions
– Intermediate lemmas for compositional reasoning
– …

Correct-by-construction Synthesis needs
Verification
– 14 –
[See Seshia, DAC 2012 for more details]
Main
Main Challenge
Challenge for
for Timing
Timing Analysis:
Analysis:
Environment
Environment Modeling
Modeling

Timing properties of the Program depend heavily
on its environment
– Other cyber-physical properties too

Environment =
Processor & Memory Hierarchy
+ Operating System, other processes/threads, …
+ Network
+ I/O Devices
+…

Modeling the environment is hard!
– Many components, crossing h/w-s/w boundary
– “Gray-box” --- proprietary technology, untrusted
components
– Tedious for engineers, requires high expertise
– 15 –
The
The End
End Goal
Goal

Improve designer / programmer
creativity and productivity
– Automate tedious tasks
– Enable user to express creative insights

How do we take the tedium out of
Environment Modeling?
– 16 –
Roadmap
Roadmap for
for the
the Talk
Talk

Why Verification of Cyber-Physical Software is
Challenging
Example: Timing Analysis

Sciduction: A Promising Direction

– Combining Inductive Inference + Deductive
Methods

Future Directions
– Other Stages of Model-Based Design
– 17 –
Timing
Timing Analysis
Analysis Problems
Problems




Worst-case execution time (WCET) estimation
Estimating distribution of execution times
Threshold property: can you produce a test case
that causes a program to violate its deadline?
Software-in-the-loop simulation: predict
execution time of particular program path
All four problems can be solved if we could predict
the execution time of arbitrary program paths.
– 18 –
The
The Problem
Problem



Program = Sequential,
terminating program
Runs uninterrupted
Environment = Processor
+ Memory Hierarchy + OS
+ Other Tasks + Network
+…
– 19 –
Current
Current State-of-the-art
State-of-the-art for
for
Timing
Timing Analysis
Analysis


Program = Sequential,
terminating program
Runs uninterrupted
PROBLEM:
Takes several manmonths to construct!
Also: limited to
extreme-case analysis
Abstract Timing Model

Environment =
Single-core Processor +
Instruction/Data Cache
– 20 –
Existing
Existing Approaches:
Approaches: One-size-fits-all?
One-size-fits-all?



Why construct a SINGLE
timing model for ALL
programs?
We are only interested
in analyzing a specific
program.
Can we automatically
infer from
measurements a
program-specific
timing model?
 GameTime
[ICCAD ’08, ACM TECS]
– 21 –
Programs
Programs 
 DAGs
DAGs
flag!=0
while(!flag)
{
flag = 1;
(*x)++;
}
*x += 2;
flag=1; (*x)++;
flag!=0
flag=1; (*x)++;
flag!=0
*x += 2;
*x += 2;
Control-flow graph
(loop bound = 1)
CFG unrolled
to a DAG
– 22 –
Ideal
Ideal Platform:
Platform: Fixed
Fixed Instruction
Instruction Timings
Timings




Each instruction takes a fixed
number of cycles
Platform assigns weight w(i) to
basic block (edge) i
w(i)
An Approach: Execute basic
block i and measure w(i)
Predict execution time for any
path by summing measurements
along that path
– 23 –
The
The Ideal
Ideal Platform
Platform Doesn’t
Doesn’t Exist!
Exist!
On a processor
with a data
flag!=0
cache
x
flag=1;
(*x)++;
flag!=0
*x += 2;
CFG unrolled
to a DAG
Timing of an edge (basic
block) depends on:
• Path it lies on
• Initial platform state
Challenges:
• Exponential number of
paths and platform states!
• Lack of visibility into
platform operation
– 24 –
Roadmap
Roadmap for
for the
the Talk
Talk

Why Verification of Cyber-Physical Software is
Challenging
Example: Timing Analysis

Sciduction: A Promising Direction

– Combining Inductive Inference + Deductive
Methods
– Application to Timing Analysis

Future Directions
– Other Stages of Model-Based Design
– 25 –
Human-Computer
Human-Computer Interaction
Interaction in
in
Verification,
Verification, TODAY
TODAY
Deductive
Proof Search
Model
true / b
!a / b
a / !b
Human User
Creates
Complete Model
– 26 –
DESIRED
DESIRED Human-Computer
Human-Computer Interaction
Interaction in
in
Verification
Verification
Inductive Inference
of Model
+
Deductive
Proof Search
Observations /
Measurements
Model
Template
Human User
Expresses
Creative Insight
– 27 –
Desired
Desired Human-Computer
Human-Computer
Interaction
Interaction in
in Verification
Verification

User Identifies hard task
– “Generate environment model”
and expresses creative insight
– The form of a mathematical model
STRUCTURE HYPOTHESIS

Tool automates “tedious” search
– Learning an environment model from
systematically generated measurements
DEDUCTION: General to specific
+
INDUCTION: Specific to general
– 28 –
Sciduction
Sciduction
[Details in DAC 2012 paper]
Structure-Constrained Induction and Deduction
Inductive Reasoning
(Active Learning: Generalizing from Examples)
+
Deductive Reasoning
(“Lightweight” Logical inference &
Constraint solving)
+
Structure Hypotheses
(on artifacts to be synthesized)
– 29 –
Demonstrated
Demonstrated Applications
Applications in
in My
My Group
Group
Floating-point
to fixed-point
Controller
synthesis
[ICCPS 2010,
EMSOFT 2011]
Program
synthesis
[S. Jha, PhD 2011]
Structure Hypothesis
+
Inductive Inference
Timing analysis
of software
(GameTime)
+
Deductive Reasoning
[ICSE 2010]
Synthesis from TL
[ICCAD 2008,
ACM TECS]
RTL
verification
[FMCAD 2011]
[MEMOCODE 2011]
– 30 –
Sciduction
Sciduction for
for Timing
Timing Analysis
Analysis
Structure Hypothesis:
Platform adversarially picks weights (w + )
+
Inductive Inference:
Infer platform weights (w + ) from Basis Path
Measurements with learning algorithm
+
Deductive Engine:
SMT solving for generating tests for basis paths
Publication: S. A. Seshia and A. Rakhlin, “Quantitative Analysis of Systems Using
– 31 –
Game-Theoretic Learning”, ACM Trans. Embedded Computing Systems.
Platform
Platform Model
Model
Models path-independent timing
Weights on edges of unrolled CFG
&
Path-specific perturbation
w(i)
+
(i)
Models path-dependent timing
– 32 –
Game
Game Formulation
Formulation

Model as a 2-player game: Tool vs. Platform
– Program paths controlled by tool
– Platform states uncontrollable (controlled by
adversary)

Problems:
– How to select paths?
– What’s a reasonable mathematical model of the
platform and how to learn it?
– Tomorrow’s Workshop Talk will describe these
steps in more detail
– 33 –
Some
Some Experimental
Experimental Results
Results
(details in ICCAD, ACM TECS, FMCAD papers)

GameTime is Efficient
– E.g.: 7 x 1016 total paths vs. < 200 basis paths

Accurately predicts WCET for complex platforms
– I & D caches, pipeline, branch prediction, …

Basis paths effectively encode information about
timing of other paths
– Found paths 25% longer than sampled basis

GameTime can accurately estimate the distribution
of execution times with few measurements
– Measure basis paths, predict other paths
Moving to Industrial Deployment
– 34 –
Recent
Recent Results
Results

Timing analysis of interrupt-driven programs
[FMCAD 2011]
– Idea: context-bounded analysis + GameTime

Energy estimation on embedded devices
– Use GameTime algorithm with iCount hardware
– 35 –
Roadmap
Roadmap for
for the
the Talk
Talk

Why Verification of Cyber-Physical Software is
Challenging
Example: Timing Analysis

Sciduction: A Promising Direction

– Combining Inductive Inference + Deductive
Methods
– Application to Timing Analysis

Future Directions
– Other Stages of Model-Based Design
– 36 –
Verification
Verification &
& Synthesis
Synthesis Problems
Problems Tackled
Tackled
by
by Our
Our Approach
Approach
Control Design and Modeling
Switching Logic
Synthesis
High-level Models
Optimization and Mapping
Floating-point
to
Fixed-point
Low-level Code
Verification of Implementations
Verification of Human-inthe-loop Systems
Timing
Analysis
– 37 –
Motivating
Motivating Problem:
Problem: Controller
Controller Synthesis
Synthesis
Synthesizing distributed control system for
building energy management
Control HVAC,
lighting and other
systems so as to
meet building usage
and comfort
requirements and
minimize energy
consumption
[Image: newenglandroof.com]
– 38 –
Synthesizing
Synthesizing Switching
Switching Logic
Logic for
for Hybrid
Hybrid
Automata
Automata with
with Nonlinear
Nonlinear Dynamics
Dynamics
?
?
?
?
• SAFETY: Room Temperature x must lie between 20 and 22 C.
• OPTIMALITY: Minimize switching between modes to save energy
– 39 –
Synthesizing
Synthesizing Switching
Switching Logic
Logic for
for Hybrid
Hybrid
Automata
Automata with
with Nonlinear
Nonlinear Dynamics
Dynamics
?
?
CHALLENGE: ?
UNDECIDABILITY!
?
• SAFETY: Room Temperature x must lie between 20 and 22 C.
• OPTIMALITY: Minimize switching between modes to save energy
– 40 –
Our
Our Approach
Approach
?
Guards are Hyperboxes
?
?
+
Hyperbox Learning from +/- Examples
?
(safe/unsafe switching
states)
+
Numerical Simulation (constraint solving)
• SAFETY: Room Temperature x must lie between 20 and 22 C.
• OPTIMALITY: Minimize switching between modes to save energy
Papers: S. Jha et al., ICCPS 2010 and EMSOFT 2011.
– 41 –
Vision
Vision for
for Verification
Verification &
& Synthesis
Synthesis
Inductive Inference
of Model
+
Deductive
Proof Search
Observations /
Measurements
Model
Template
Human User
Expresses
Creative Insight
– 42 –
Need
Need Seamless
Seamless Interaction
Interaction with
with Feedback
Feedback
Loop
Loop
Inductive Inference
of Model
+
Deductive
Proof Search
?
Model
Template
?
?
?
NO SOLUTION
– 43 –
Need
Need Seamless
Seamless Interaction:
Interaction: Support
Support for
for
Multiple
Multiple Structure
Structure Hypotheses
Hypotheses &
& Feedback
Feedback
Model
Inductive Inference ION
T Template
U
of Model OL Y
+ O S ODA
N T
Deductive
Proof Search
?
?
?
?
Algorithms need to be more
expressive and flexible
– 44 –
Conclusion
Conclusion


Many Challenges in the verification of cyberphysical software systems
Need to examine the Human Aspect
– Verification “=” Synthesis
– Environment Modeling

Sciduction: a new approach to verification and
synthesis
– GameTime: timing analysis of embedded software

Vision: A Seamless Interaction between Human
User and Verification/Synthesis Tool
– 45 –
Relevant
Relevant Publications
Publications






S. A. Seshia, “Sciduction: Combining Induction, Deduction, and
Structure Hypotheses for Verification and Synthesis”, DAC 2010.
S. A. Seshia and A. Rakhlin, “Game-Theoretic Timing Analysis”,
ICCAD 2008.
S. A. Seshia and A. Rakhlin, “Quantitative Analysis of Systems
Using Game-Theoretic Learning”, ACM TECS, to appear.
J. Kotker, D. Sadigh, and S. A. Seshia, “Timing Analysis of
Interrupt-Driven Programs Under Context Bounds”, FMCAD 2011.
S. Jha, S. Gulwani, S. A. Seshia, and A. Tiwari, “Synthesizing
Switching Logic for Safety and Dwell-Time Requirements”, ICCPS
2010.
S. Jha, S. A. Seshia, and A. Tiwari, “Synthesis of Optimal
Switching Logic for Hybrid Systems”, EMSOFT 2011.
– 46 –
Anatomy
Anatomy of
of GameTime
GameTime
Game-Theoretic Online Learning +
Satisfiability Solving Modulo Theories (SMT)
i1 42
i2 75
i1
PROGRAM
i3
CONTROL-FLOW
GRAPH
EXTRACT BASIS PATHS
SMT SOLVER GENERATES
TEST INPUTS
i3 101
LEARNING
online ALGORITHM
…
…
i2
MEASURE
EXECUTION
TIMES
PREDICT
TIMING
PROPERTIES
(worst-case,
distribution,etc.)
Publication: S. A. Seshia and A. Rakhlin, “Quantitative Analysis of Systems Using
– 47 –
Game-Theoretic Learning”, ACM Trans. Embedded Computing Systems.