Advisor: Frank,Yeong-Sung Lin
Presented by Yu-Shun, Wang
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
2
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
3
Introduction
 The complexity and attack level of network systems grow with
each passing day.
 The attacked organization will get lots of lose no matter on
monetary or reputation.
 the most expensive incident on average was financial fraud,
with an average reported cost of $463,100. *
 followed by dealing with “bot” computers within the
organization’s network, reported to cost an average of $345,600
per respondent. *
 Dealing with loss of either proprietary information or loss of
customer and employee confidential data averaged at
approximately $241,000 and $268,000, respectively. *
*Robert R., CSI Director, “2008 CSI Computer Crime & Security Survey,” 2008.
2009 version will release on December 1, 2009 11:00 am PT/2:00 pm ET
2017/7/28
OPLab@IM, NTU
4
Introduction
 We define survivability as the capability of a system to
fulfill its mission, in a timely manner, in the presence of
attacks, failures, or accidents. We use the term system in
the broadest possible sense, including networks and
large-scale systems of systems. *
Survivability Status
Compromised
Safe
* R. J. Ellison, D. A. Fisher, R. C. Linger, H. F. Lipson, T. Longstaff, and N. R. Mead,
“Survivable Network Systems: An Emerging Discipline,” Technical Report CMU/SEI-97TR-013, November 1997.
2017/7/28
OPLab@IM, NTU
5
Introduction
Title
Author(s)
An Evaluation of Network Survivability When Defense Levels
Are Discounted by the Accumulated Experience of Attackers
F.Y.-S. Lin, P.-Y. Chen, and P.-H.
Tsang
Maximization of Network Survival Time in the Event of
Intelligent and Malicious Attacks
P.H. Tsang, F.Y.S. Lin, and C.W,
Chen
Near Optimal Attack Strategies for the Maximization of
Information Theft
F.Y.S. Lin, C.-L. Tseng and P.-H.
Tsang
Near Optimal Protection Strategies against Targeted Attacks on
the Core Node of a Network
F.Y.-S. Lin, P.-H. Tsang and Y.-L.
Lin
Evaluation of Network Robustness for Given Defense Resource
Allocation Strategies
F.Y.-S. Lin, P.-H. Tsang, C.-H.
Chen, C.-L. Tseng and Y.-L. Lin
Maximization of Network Robustness Considering the Effect of
Escalation and Accumulated Experience of Intelligent Attackers
F.Y.-S. Lin, P.-H. Tsang, P.-Y.
Chen, and H.-T. Chen
2017/7/28
OPLab@IM, NTU
6
Introduction
Previous research
My work
Complete information about topology
Only one hop information
Complete information about defense
resource allocation
Only next hop defense resource
information
Complete information about node
attribute
Partial information about node attribute
Single category of attacker
Multiple categories of attacker
Information is gathered before an attacker Information is gathered during attack
launches an attack
2017/7/28
OPLab@IM, NTU
7
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
8
Problem formulation
 For defense resource, we not only consider resource that
increase defense level but also another deception based
defense mechanism, honeypots.
 Acting as a false target to distract attackers. *
* http://honeypots.sourceforge.net/
2017/7/28
OPLab@IM, NTU
9
Problem formulation
 For attackers, we apply following criteria to classify:
 Budget

Three levels, using minimum attack cost as the benchmark.
 Capability

Three levels, it influences the probability attackers cheated by
honeypots.
 Next hop selection criteria



The highest defense level (for valuable information)
The lowest defense level (for stealth strategy *)
Random attack (for random strategy *)
* Fred Cohen, “Managing Network Security Attack and Defense Strategies”
2017/7/28
OPLab@IM, NTU
10
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
11
Mathematical formulation
 Assumptions
 There is only one single core node in the network.
 The defender has the perfect knowledge of network that is
attacked by several attackers with different budget,
capabilities, and next hop selection criteria.
 The attackers are not aware that there are honeypots
deployed by the defender in the network, i.e., the attackers
have the imperfect knowledge of network.
 There are two types of defense resources, the honeypot and
non-honeypot.
2017/7/28
OPLab@IM, NTU
12
Mathematical formulation
 Assumptions (cont.)
 A node is only subject to attack if a path exists from the
attacker’s position to that node, and all the intermediate
nodes on the path have been compromised.
 A node is compromised when attack resources allocated to it
is no less than the defense force incurred by defense
resources.
 Only malicious nodal attacks are considered
 The network is viewed at the AS level.
2017/7/28
OPLab@IM, NTU
13
Mathematical formulation
Given parameters
Notation
Description
M
The total evaluation frequency for all attacker categories
K
The total attacker categories
Pk
The portion of attacker type k in total attackers (where k  K)
Rk
Rounded evaluation frequency of each attacker type Frequency
D
All possible defense strategies
The strategy of an attacker, comprising his budget, capabilities,
Ak
and next hop selection criteria. (where k K) Attack & Defense
1 if the attacker j of the kth category can compromise the core
Skj( D, A )
node under D defense strategy, and 0 otherwise (where k  K)
k
B
The total budget of defender
Budget
th
Bk
The total budget of the k type of attacker, where k  K
F
The index set of honeypots to play the role of fake core nodes
I
The index set of all general nodes in the network
Index
2017/7/28
OPLab@IM, NTU
14
Mathematical formulation
Decision variables
Notation
Description
bi
The defense resource allocated to protect a node i, where i I
hf
The defense resource allocated to honeypot f as the fake core
Defense budget
node in the network, where f  F
a(bi)
The cost of compromising a general node i in the network,
Attack budget
where i  I
a(hf)
The cost of compromising a honeypot f in the network, where f
F
2017/7/28
OPLab@IM, NTU
15
Mathematical formulation
 Objective Function:
2017/7/28
OPLab@IM, NTU
16
Mathematical formulation
 Constraints
Defender budget constraints
Attacker budget constraints
2017/7/28
OPLab@IM, NTU
17
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
18
Solution Approach
 Evaluation Process
Initial state
Run evaluation with the 27
kinds of different attackers for
M times and get the core node
compromised frequency.
Adjust defense parameters by
policy enhancement
Let the frequency divided by M
to gather average core node
compromised probability.
Run another evaluation M times
using adjusted defense
parameters and get the
corresponding probability
Stop
criteria
Yes
No
Compare result
with the initial one
2017/7/28
OPLab@IM, NTU
19
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
20
Solution Approach
• Policy Enhancement
 The main concept of policy enhancement can be summarized
into the following parts.
Enhanced probability  Initial probability
Derivative
Reallocation defense resource
 This concept is using to measure the marginal effectiveness of
each defense resource allocation.
 Popularity Based Strategy
 This strategy is focuses on those nodes are frequently attacked.
Therefore, we let the cost attackers spent on each node divided
by total attack cost spend in the entire network as the metric in
the policy enhancement.

2017/7/28
OPLab@IM, NTU
21
Solution Approach
• Policy enhancement
We first take certain
amount of resources from
nodes in the network
Quantity of
resources is
too large?
Yes
Only remove resources
from nodes afforded
No
Change the quantity
of resources we
take from nodes
Total quantity of
resources is higher
than the threshold?
No
Yes
Yes
Choose the one with lowest
derivative to replace current
No
allocation scheme
2017/7/28
Whether there
is a better value
to test?
OPLab@IM, NTU
Calculate derivative
of every reallocation
scheme
22
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
23
Experimental result
 Important parameters
Parameter
Value
Total number of attacker profiles
27
Attacker budget levels
3
Attacker capability levels
3
Next hop selection criteria
3
Defender total budget
1,000
Total evaluation times for one round
2017/7/28
10,000,000
OPLab@IM, NTU
24
Experimental result
 Important parameters (cont.)
Types of attackers’ budget level
Value
High level
2 times of minimum attack cost
Medium level
1.5 times of minimum attack cost
Low level
1 time of minimum attack cost
Types of attackers’ capability level
Value
High level
30% distracted by false target honeypot
Medium level
50% distracted by false target honeypot
Low level
70% distracted by false target honeypot
2017/7/28
OPLab@IM, NTU
25
1
19
37
55
73
91
109
127
145
163
181
199
217
235
253
271
289
307
325
343
361
379
397
415
433
451
469
487
505
523
541
559
577
595
613
631
649
667
685
703
721
739
757
775
793
811
829
847
865
883
901
919
937
955
973
991
Experimental result
 Experiment on M
 1000 chunks
2017/7/28
AvgComFreq.
2010
2000
1990
1980
1970
1960
1950
1940
1930
OPLab@IM, NTU
26
1
177
353
529
705
881
1057
1233
1409
1585
1761
1937
2113
2289
2465
2641
2817
2993
3169
3345
3521
3697
3873
4049
4225
4401
4577
4753
4929
5105
5281
5457
5633
5809
5985
6161
6337
6513
6689
6865
7041
7217
7393
7569
7745
7921
8097
8273
8449
8625
8801
8977
9153
9329
9505
9681
9857
Experimental result
 Experiment on M (cont.)
AvgComFreq.
 10000 chunks
2000
1990
1980
1970
1960
1950
1940
1930
2017/7/28
OPLab@IM, NTU
27
Experimental result
 Initial allocation scheme
 We apply two metrics to allocate our defense resource:
The number of hops to the core node
 We believe nodes closer to the core node play more important role.
Therefore, we allocate more resources on nodes near the core node.
 Link degree of each node
 Since the link degree can also reflect importance of a node, we
allocate more resources on nodes with higher link degree.

 We combine these two metrics by giving different weight, for
example, 30% number of hops and 70% link degree, to
allocate resource.
2017/7/28
OPLab@IM, NTU
28
Experimental result
 Different values of weight will result in distinct initial
allocations.
 Once the initial allocation is changed, the value of
minimum attack cost also altered.
 Attackers’ budget is determined by multiple of minimum
attack cost.
 We need an uniform benchmark to compare performance.
 Consequently, the benchmark of deciding attackers’
budget is fixed at certain values in the following
experiments.
2017/7/28
OPLab@IM, NTU
29
Experimental result
 Performance comparison when benchmark is set at 443
(minimum attack cost of 20% hop and 80% link initial
allocation):
2017/7/28
OPLab@IM, NTU
30
Experimental result
 Performance comparison when benchmark is set at 480
(minimum attack cost of 50% hop and 50% link initial
allocation):
2017/7/28
OPLab@IM, NTU
31
Experimental result
 Performance comparison when benchmark is set at 515
(minimum attack cost of 80% hop and 20% link initial
allocation):
2017/7/28
OPLab@IM, NTU
32
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comment
2017/7/28
OPLab@IM, NTU
33
Conclusion
 In this paper, we relax the commonly made “perfect information
assumption for attackers” in previous research and propose a
mathematical model to evaluate network survivability.
 We consider a more realistic environment where multiple classes of
attackers may exist, and that attackers from different classes may be
of distinct attributes, behaviors and strategies.
 Our main contribution is that we combine mathematical
programming and simulation techniques and develop a novel
approach to solve problems with the imperfect knowledge property.
2017/7/28
OPLab@IM, NTU
34
Agenda
 Introduction
 Problem formulation
 Problem description
 Mathematical formulation
 Solution Approach
 Evaluation Process
 Policy Enhancement
 Experimental result
 Conclusion
 Reviewers’ comments
2017/7/28
OPLab@IM, NTU
35
Reviewers’ comments
 Reviewer 1:
 The authors describe a mathematical model that allows to asses the
survivability of a computer network and its core components. While the
model may be an interesting theoretical contribution, I see several
problems once the methodology is applied to a real world scenario.
 First, in real works it is almost impossible to estimate/fix the parameters
of the system. For example, how can one asses the "cost of
compromising a general node in the network" (value a(b_i))? How can I
compute the "cost" of a specific defense mechanism?
 Second, it remains completely unclear which "attacker categories" the
authors consider. They do tell on page 2 that there are in total 27 of
them, but they do not give any details.
 Third, I do not understand why their proposed algorithm is “near
optimal” as stated in the title. What does that mean? When is an
algorithm "optimal"?
2017/7/28
OPLab@IM, NTU
36
Reviewers’ comments
 Reviewer 2:
 You paper is well written and I was inclined to think that you had
stumbled across an area of growing interest when you referenced
it to several other pieces of work: "A number of previous works,
e.g., [2] [3] [4] [5] [6] [7]"
 However, on examination, you have only cited your own work
and thus are presenting minor changes to your own work.
 If the research question is a significant one, and it may be, then
you need to provide an in-depth literature review that proves
this. Otherwise I have to reject it since you have not really begun
to show your reader why this work is significant.
2017/7/28
OPLab@IM, NTU
37
Reviewers’ comments
 Reviewer 3:
 This paper studied the near optimal defense strategies to
minimize attacker's success probabilities in honeypot networks.
The presentation is clear and the paper is well organized. Given
the assumptions in the paper, the evaluation looks good.
 My concern about the paper is the strong assumptions in the
paper. In Section II, Problem Formulation, the authors over
simplified the attacker's knowledge and the procedure of
attacks. Given such strong assumption, the later calculations and
analysis are less challenging. I doubt how many attacks can fall
into the assumed situation. The strong assumption may seriously
limit the application of the proposed method and make the
contribution of the paper less significant.
 Moreover, the technical strength of the paper, especially the
analysis part, is a little bit weak.
2017/7/28
OPLab@IM, NTU
38
2017/7/28
OPLab@IM, NTU
39
Solution approach
 Evaluation Process
 Since our scenario and environment are very dynamic, it is hard
to solve the problem purely by mathematical programming.
 For each attacker category, although attackers in it belong to the
same type, there is still some randomness between each other.
 This is caused by honeypots. if an attacker compromises a false
target honeypot, there is a probability that he will believe the
core node is compromised and terminate this attack.
 Therefore, we can never guarantee the result of an attack is
successful or failed until the end of the evaluation.
2017/7/28
OPLab@IM, NTU
40
Solution Approach
 Evaluation Process
 Parameter setting


2017/7/28
M (Total evaluation frequency for one round)
 First, we make an initial value, for example, 10 million. Then, we let 10
thousands as a chunk to summary the result and draw a diagram
depicting the relationship between compromised frequency and number
of chunks.
 If the diagram shows a stable trend, it implies the value of M is an
ideal one.
Stop criteria
 N (Total rounds for policy enhancement)
• We set this value by resource constrained approach.
 If we cannot improve the quality of resource allocation scheme
anymore, we also terminate this process.
OPLab@IM, NTU
41
Solution Approach
 Policy enhancement
 The quantity of defense resource we take from node is
30+30/3=40
20+20/2=30
30-30/3=20
Initial value (20)
‧‧‧‧‧‧‧‧‧‧
determined by harmonic series.
 Further, we also determine direction of this quantity.
 When the quantity divided by iteration number is no more
than 2, we stop searching for better value.
10+10/3=13
20-20/2=10
10-10/3=7
2017/7/28
OPLab@IM, NTU
42
Topical on honeypot in Taiwan
2017/7/28
OPLab@IM, NTU
(i)
Topical on honeypot in Taiwan
2017/7/28
OPLab@IM, NTU
(ii)
Response to the comment
 It is worth to emphasis there is a great difference between
perfect knowledge and imperfect knowledge.
 For example, most of shortest path algorithms and minimum
cost spanning tree algorithms are based on the perfect
knowledge assumption.
 If nodes and links will dynamically appear during searching
for the shortest path or the minimum cost spanning tree, wellknown algorithms may not feasible anymore.
 Although there is no need to relax this assumption in those
algorithms, it is a necessary concern in our attack defense
scenario.
2017/7/28
OPLab@IM, NTU
(iii)