Sparse Coding for Specification
Mining and Error Localization
Wenchao Li, Sanjit A. Seshia
University of California - Berkeley
[email protected]
Runtime Verification
September 26, 2012
Assertion-Based Verification
Runtime Verification 2012
Generate stimulus to patch coverage holes
Tests
Simulator
Coverage
Circuit/Program
Assertions
Find bugs with assertions
Problem: assertions are created manually
“…typically 20% of specifications pass vacuously during the first
formal verification runs of a new hardware design…” [IBM Haifa]
2
Error Localization
Runtime Verification 2012
010101010101
011011010101
010111111010
10101
Challenges:
• Limited observability
• Long error detection latency
• Transient and hard-to-reproduce bugs
Where?
Idea: assertions can provide local
observability and correctness checks
3
Related Work
Runtime Verification 2012
• Specification Mining:
– Programs: single-state invariants, pre-/post-conditions, automata
learning, alternating patterns
– Circuits: fixed-delay pairs, temporal logic patterns
– Require templates
• Error Localization:
– Programs: model checking, predicates
– Circuits: instruction footprints, SAT-based, mined assertion-based
– Require system model and good observability
– Require templates
Our technique is template-free and does
not require having the system model
4
What can you tell by just observing a trace?
Runtime Verification 2012
1
0
0
1
1
1
0
0
0
0
0
0
1
1
1
0
0
0
1
0
1
0
0
1
0
1
0
0
0
0
0
0
1
0
1
0
0
0
1
0
Obj1.m1()
Obj1.m1()
Obj1.m2()
Obj2.m1()
Cloud
5
Obj2.m1()





Hardware trace
Program trace
Human interaction/behavior
Sensor network
Distributed system
A Sparse Coding Approach
Runtime Verification 2012
1 1 0 0 1
0 0 1 0 1
0.8* *
 0.8
x
 0.8 *
0.3**
++0.3
f3
+ 0.3 *
++0.5
0.5* *
f30
+ 0.5 *
f61
Sparsity helps to uncover latent structure of the data
6
Key idea: Express each subtrace as a Boolean
combination of a few “basis subtraces”– a (sparsityconstrained) Boolean matrix factorization problem.
Contributions and Outline
Runtime Verification 2012
• A new formalism for discovering structure in a trace
• A definition of the sparsity-constrained Boolean matrix
factorization problem and an algorithm for solving it
• Applications to specification mining and error
localization
– Does not rely on redefined templates
– Simultaneous perform error localization and explanation
• Outline:
 Problem formulation
 Algorithm
 Error localization and explanation
 Results
7
Problem Formulation
Runtime Verification 2012
columns are sparse
1 1 0 0 1
0 0 1 0 1
basis
1
1
0
0
8
1
0
0
1
Subtrace
=
coefficient
○
Multiplication as “AND”
Addition as “OR”
Sparsity-Constrained Boolean Factorization
Runtime Verification 2012
Given a data matrix 𝑋 ∈ 𝑩𝒎×𝒏 and a positive integer 𝐶,
the sparsity-constrained Boolean factorization problem
is to find 𝑘, 𝐵 = 𝑩𝒎×𝒌 and 𝑆 = 𝑩𝒌×𝒏 such that
𝑋 =𝑩∘𝑺
and
and
𝒊
𝒋 𝑺𝒊,𝒋
𝑺∙,𝒊
𝟏
≤ 𝑪, ∀𝒊
is maximized.
C=2
𝑋
9
𝐵
𝑆
Algorithm Idea
Runtime Verification 2012
• Observe that the data matrix X can be viewed as the adjacency
matrix for a bipartitie graph.
• Idea: factorization → biclique cover (biclique ↔ basis subtrace)
v
u
10
Algorithm Overview
Runtime Verification 2012
• Incrementally generate maximal bicliques
– Consensus-based algorithm
A
D
A
A
B
E
C
E
C
C
– Extend to a maximal biclique
A
D
A
C
D
E
E
C
• Keep track of closeness to sparsity constraint
• Heuristically optimize for basis sharing
11
D
Algorithm Overview
Runtime Verification 2012
•
•
•
•
•
Step 1: start with the set of v-rooted star bicliques
Step 2: Pick two stars and form a consensus
Step 3: Extend the consensus to a maximal biclique
Step 4: Add the biclique to the cover if possible
Step 5: update sparsity constraint at the covered nodes
A
D
B
A
D
A
D
C
E
…
E
C
A
C
F
A
D
B
E
C
F
X
Y
E
12
C
Z
G
An Arbiter Example
Runtime Verification 2012
A 2-input 2-output arbiter with round-robin scheme
Number of subtraces
p0
0
1
0
1
1
0
… …
p1
1
0
0
1
1
1
… …
q0
0
1
0
0
1
0
… …
q1
1
0
0
1
0
1
… …
0
1
1
0
Sample mined assertions (basis subtrace):
13
0
0
0
0
1
0
0
1
1
1
0
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
1
1
0
0
0
0
0
0
1
0
12
Error Localization and Explanation
Runtime Verification 2012
• Error localization and explanation based on reconstruction:
A subtrace has an error if it cannot be
reconstructed from the basis subtraces
0
1
0
1
1
0
…
…
1
0
0
1
1
1
…
…
Minimize 𝑋∙,𝑖 ⨁(𝐵 ∘ 𝑆∙,𝑖 )
0
1
0
0
1
0
…
…
𝑆∙,𝑖
1
0
0
1
0
1
…
…
Subject to
𝑋∙,1
≤ 𝐶
𝑋∙,2
• A subtrace is error-free if 𝑋∙,𝑖 ⨁(𝐵 ∘ 𝑆∙,𝑖 )
14
𝑆∙,𝑖
1
=0
• If not, a (minimum) error explanation is 𝑋∙,𝑖 ⨁(𝐵 ∘ 𝑆∙,𝑖 ), where
𝑆∙,𝑖 is the solution to the minimization problem above.
1
Example Illustration
Runtime Verification 2012
• Error localization and explanation (arbiter example):
1
0
0
0
1
0
0
1
0
0
0
1
1
0
1
0
0
1
0
0
1
0
1
0
0
0
1
0
0
1
0
0
0
0
1 0 0 0
Error trace
0 0 0
Error subtrace
Error
All subtraces
Correct
subtraces
15
Space
spanned by
the learned
basis
0 0 0
Error explanation
0
0
0
0
0
0
0
0
0
0 1 0
Alternative error
Explanation
Experimental Results
Runtime Verification 2012
• Chip Multiprocessor Router:
A CMP
Router
in a NoC
– Observe 14 control signals
– Subtrace width of 2 cycles
– Learn the basis from a single errorfree trace of 1000 cycles: 0.243
seconds to obtain 189 basis
subtraces from 93 distinct subtraces
• Error Localization:
– Inject a single bit flip at a random cycle for each of 99 error traces
– Localize the error to the subtrace (out of 999) where it was injected
• Comparisons:
16
– Baseline approach (1): hash all distinct subtraces – report error
even before an error is injected for the 99 traces
– Baseline approach (2): use unit basis – 0% localization
– Sparse Coding: 55.6% localization
Conclusion
Runtime Verification 2012
• A template-free assertion miner that can explore
embedded patterns in digital circuit traces
• Effective assertion-mining based error localization
and explanation
• Potential applications to other domains, e.g.
programs or distributed systems
THANK YOU
17