Automatic XACML requests generation
for policy testing
Antonia Bertolino, Said Daoudagh,
Francesca Lonetti, Eda Marchetti
Istituto di Scienza e Tecnologie
dell’Informazione “A. Faedo”
Consiglio Nazionale delle Ricerche
Agenda
»
»
»
»
»
Access control policies and XACML
language
Why a testing methodology?
An empirical evaluation
Conclusions and future works
X-CREATE Demo
Access Control Policies
Data and resources must be protected against
unauthorized, malicious or improper usage or
modification
Policies
specification
- -
Data
Resources
Testing the Policy Implementation
Policies
SUT
request
request
request
request
Test Suite
verdict
PDP
reply
Oracle
PDP (Policy Decision Point): evaluates the requests
against the access control policies
XACML Policy Structure
XACML (eXtensible
Access Control
Markup Language)
Esample of Policy
<Policy>
<Target>
<Subject>Mario Rossi</Subject>
The XACML Policy elements:
<Resource>personal id</Resource>

PolicySet
<Action>read</Action>

Policy
</Target>

Target
</Policy>

Subjects

Resources
Example of request

Actions
<Request>

Environments
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>

Rules
<Action>read</Action>
</Request>
X-CREATE Testing Framework
XaCml REquests derivAtion for TEsting
Implements several testing strategy:
• Preliminary XPT (XML Partition Testing)
• Incremental XPT
• Simple Combinatorial
• Hierarchical Simple
• Hierarchical Incremental
Policies
specification
Instantiated
Request
Request
structure
Preliminary XPT Main Idea
Deriving once and for all a universally valid generic test suite
of conforming requests by applying:
» A variant of the Category Partition methodology
» The Boundary Conditions methodology
XACML
Context
Schema
Request
structure
Conforming
test suite
E.G. Fixing ∞ to 3
X
{1,2,3}
X
{1,2,3}
An Example
Example of request structure
<Request>
<Subject> </Subject>
<Subject> </Subject>
<Resource> </Resource>
<Action> </Action>
</Request>
Issue: The maximum number of structurally
different intermediate requests is of
310 * 21 = 118098
118098!!!!
Too Much!!!
Testing objectives:
» New methodology for request structures
generation (Incremental XPT)
» New stopping criterion for test requests
execution
» New specific test strategy satisfying the
stopping criterion (Simple Combinatorial)
Incremental XPT
36 = 729 request
stuctures:
» one value for the
<AttributeValue>
» zero to minOccurs and
maxOccurs of the
ResourceContent
element and those of
the contained <Any>
element because not
used in test values
generation
Filling request structures with
values
Take values from the policy under test for elements
and attributes.
»
»
»
»
»
SubjectSet
ResourceSet
ActionSet
EnvironmentSet
An entity is a combination of 4 values taken from
these sets (n-wise approach is used)
Toy Example
<Policy>
<Target>
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>
<Action>read</Action>
</Target>
AttributeId
Data Type
</Policy>
SubjectSet
Subjectid
string
Attribute Value
Mario Rossi
ResourceSet
Resourceid
string
personal id
ActionSet
Actionid
string
EnvironmentSet
read
Complete Table
For robustness and negative testing random values for elements
and attributes are added
AttributeId
Data Type
Attribute Value
SubjectSet
Subjectid
string
Mario Rossi
S1
string
s2
ResourceSet
Resourceid
string
personal id
R1
string
r2
ActionSet
Actionid
string
read
A1
string
a2
EnvironmentSet
E1
string
e2
How many entities?
»
»
»
»
Avoiding duplication derive all combinations of
subject entities, resource entities, action
entities and environment entities by applying:
the pair-wise combination (PW)
the three-wise combination (TW)
apply the four-wise combination (FW)
Note: The number of combinations is limited and
strictly depends on the policy considered
Examples
Example of request
Example of request
<Request>
<Request>
<Subject>Mario Rossi</Subject>
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>
<Subject>s2</Subject>
<Action>read</Action>
<Resource>p2</Resource>
</Request>
<Action>read</Action>
<Enviroment>e2</Enviroment>
</Request>
Example of request
<Request>
<Subject>s2</Subject>
<Resource>personal id</Resource>
<Action>a2</Action>
</Request>
Simple Combinatorial
Idea: derive as many requests as the
possible combinations of the values of the
subjects, resources, actions and
environment of the XACML policy.
The number of combinations could be also
be used as a stopping criterion for the test
case generation in XPT
Incremental XPT vs. Simple Combinatorial
Research questions:
1st Match TSEff: adopting the proposed stopping criterion, is
the fault detection of the Simple Combinatorial strategy
similar to that of the Incremental XPT one?
2nd MatchTSDecr: is it possible to reduce the test suites
maintaining the same level of fault detection?
3rd MatchTSIncr: is it possible to increase the Incremental
XPT fault detection?
Rules of comparison
Evaluation of the test strategies effectiveness:
» Define a set of XACML policies
» Apply mutation to each policy to introduce
faults
» Execute each set of test cases on the policy
and its mutants
» Establish the winner according in each match
Simple cobinatorial
Incremental XPT
XPT v.s. Simple Combinatorial
1st Match TSEff:
»
»
»
The same number of requests for each policy
the effectiveness of the Incremental XPT is generally higher
than that of the Simple Combinatorial strategy
In two cases the fault detection of the Simple Combinatorial
is higher than that of Incremental XPT
Deep Analysis
»
»
Incremental XPT is the winner when the
access decision of the policy rules depends
concurrently on the values of more than one
subject or resource or action or environment
entity
Simple Combinatorial is the winner when the
policies are very simple and the satisfiability of
the policy rules depends on the combinations
of a single subject, resource, action and
environment entity
2nd Match TSDecr: from the first request ahead till we reached
the maximum reachable percentage of fault detection
Simple Combinatorial is the winner
Note: For XPT usually the maximum reachable percentage is
reached with almost half of the available requests =>the
stopping criterion is a good upper bound
XPT v.s. Simple Combinatorial
3rd Match TSIncr: the loss in the fault detection effectiveness due
to the stopping criterion
1.
Execute the full pull of available requests
»
percentage of mutants killed could be increased a lot
Calculate the minimum # requests for the maximum fault
detection effectiveness
2.
in most of the cases the loss of fault detection effectiveness is
around 15%.
»
Preliminary Conclusions and Future
Works
A good fault detection percentage of the Incremental XPT testing
strategy due to the variability of the structures of the generated
requests
It is possible to reduce the number of requests
The high variability of the Incremental XPT strategy can limit its
performance when policies are very simple
Homework:
• Generalize the results
• Consider further mutation operators
• Conceive new test strategy generating requests containing all the
possible combinations of more than one subject, resource, action
and environment entity.
»
Thank you!