Key Research Issues in Scientific
Workflow Temporal Verification
Xiao Liu
CS3 -- Centre for Complex Software Systems and Services
Swinburne University of Technology, Australia
[email protected]
Outline

Scientific Workflows

Key Research Issues in temporal verification
 Temporal Verification
 A motivating example
 Constraint Setting
 Checkpoint Selection
 Temporal Verification
 Temporal Adjustment

Temporal Verification Framework

SwinDeW-V Project
2
Scientific Workflows

Scientific Workflow Management System
 A type of workflow management system aiming at supporting
complex scientific processes in many e-science applications
such as climate modelling, astronomy data processing. It
may be built upon grid, cluster, P2P, Cloud infrastructure.
3
E-Science and E-Business
Utility computing
High-performance computing
Collaborative design
Financial modeling
E-Business
High-energy physics
Drug discovery
Data center automation
Life sciences
E-Science
Natural language
processing & Data Mining Collaborative data-sharing
From www.gridbus.org
4
Outline

Scientific Workflows

Key Research Issues in temporal verification
 Temporal Verification
 A motivating example
 Constraint Setting
 Checkpoint Selection
 Temporal Verification
 Temporal Adjustment

Temporal Verification Framework

SwinDeW-V Project
5
Introduction: Temporal Verification

Scientific workflow verification: Structure, Performance,
Resource, Authorisation, Cost and Time.

Temporal verification is to check the temporal consistency states
so as to identify and handle temporal violations.

In reality, complex scientific and business processes are normally
time constrained. Hence:
 Time constraints are often set when they are modelled as
scientific workflow specifications.
 Temporal consistency states, i.e. the tendency of temporal
violations from consistency to inconsistency, need to be
verified and treated proactively and accordingly.
6
Definition: Temporal Consistency
7
Outline

Scientific Workflows

Key Research Issues in temporal verification
 Temporal Verification
 A motivating example
 Constraint Setting
 Checkpoint Selection
 Temporal Verification
 Temporal Adjustment

Temporal Verification Framework

SwinDeW-V Project
8
A Motivating Example
Question 1: Where and how much should we set
temporal constraints?
9
Constraint Setting – A Solution

Two basic requirements:
 Temporal constraints should facilitate both overall coarse-grained
control and local fine-grained control.
 Coarse-grained constraints refer to those assigned to the entire
workflow or workflow segments.
 Fine-grained constraints refer to those assigned to individual
activities.
 Temporal constraints should be well balanced between user
requirements and system performance.

A probabilistic setting strategy (X. Liu, BPM08)
 Aggregation: Setting coarse-grained constraints
 Propagation: Setting fine-grained constraints
10
Constraint Setting – A Challenge

Where?
 Currently, the locations of temporal constraints are normally
assumed to be pre-defined. It is evident that the locations of
temporal constraints have great impact on the efficiency
control of workflow executions.
End
Activity
Decision
Point
Critical
Path
11
A Motivating Example cont.
Question 2: Where should we check the current
temporal consistency state?
12
Checkpoint Selection – A Solution

Two basic requirements:
 Necessity: only those activity points where real
temporal inconsistency states take place are
selected
 Sufficiency: there are no any omitted points.

A minimum time redundancy based checkpoint
selection strategy (J. Chen, ACM-TASS2007)
13
Checkpoint Selection – A Challenge

Efficiency
 The criteria of necessity and sufficiency have significantly
reduced the cost over the previous strategies, it is still
huge especially in a scientific workflow of thousands of
activities.
14
A Motivating Example cont.
Question 3: What is the current temporal
consistency state?
Qualitative : {strong consistency/inconsistency,
weak consistency/inconsistency }
Quantitative : {80% probability of consistency, 20%
probability of inconsistency}
15
Temporal Verification – A Solution
Multi-States based temporal consistency (J. Chen, CCPE2007)
Temporal Dependency based Checkpoint Selection (J.
Chen, Y. Yang, ICSE2008)
16
Temporal Verification – A Challenge

Efficiency
 The efficiency of temporal verification strongly related to
checkpoint selection since they are always performed
together.
 The relationship between different temporal consistency can
be helped to improve the efficiency.
17
A Motivating Example cont.
Question 4: What should we do if there are
temporal violations?
18
Temporal Adjustment – A Solution

Time deficit allocation (J. Chen CCPE2007)

Time deficit allocation strategy (TDA) compensates
current time deficit by utilising the expected time
redundancies of subsequent activities.
 Based on expected time redundancies.
 Only delay the violations of local constraints.
 No effective on overall constraints.
19
Temporal Adjustment – A Challenge

No effective solutions have been proposed yet.

Different from conventional exception handling:
 on the fault tolerance of functional failures; on non-functional QoS
violations
 triggered when true violations happened; triggered when expected
violations detected

Possible solution:
 Recruiting additional resources
 Workflow scheduling
 Negotiation—amendment of temporal constraints
 ?...
20
Outline

Scientific Workflows

Key Research Issues in temporal verification
 Temporal Verification
 A motivating example
 Constraint Setting
 Checkpoint Selection
 Temporal Verification
 Temporal Adjustment

Temporal Verification Framework

SwinDeW-V Project
21
A Temporal Verification Framework

Constraint Setting
 Setting temporal constraints according to temporal QOS
specifications.

Checkpoint Selection
 Selecting necessary and sufficient checkpoints to conduct
temporal verification.

Temporal Verification
 Verifying the consistency states at selected checkpoints.

Temporal Adjustment
 Handling different temporal violations.
22
Outline

Scientific Workflows

Key Research Issues in temporal verification
 Temporal Verification
 A motivating example
 Constraint Setting
 Checkpoint Selection
 Temporal Verification
 Temporal Adjustment

Temporal Verification Framework

SwinDeW-V Project
23
SwinDeW-V

SwinDeW-V is an ongoing research project which focuses on
temporal verification and serves as one of the key
functionalities in our SwinDeW-G, a peer to peer based
scientific grid workflow system.
Scientific Workflow Execution
VPAC
Swinburne
CS3
Astrophysics
Supercomputer
Beihang
CROWN
· SwinDeW-G
· GT4
· SuSE Linux
· SwinDeW-G
· CROWN
· Linux
Swinburne
ESR
PfC
UK
· SwinDeW-G
· GT4
· CentOS Linux
· SwinDeW-G
· GT4
· CentOS Linux
Hong
Kong
SwinDeW-G Peer
Grid Node
Network Connection
24
Current States and Future Work

Currently, as an important reinforcement for the overall
workflow QoS, temporal verification is being
implemented in SwinDeW-G. It currently supports
dynamic checkpoint selection and temporal verification
at run-time.

In the future, SwinDeW-V will explore more on the two
tasks of constraint setting and temporal adjustment. Our
main objective is that SwinDeW-V can be developed as
an independent software component which can be
easily adopted by any workflow systems to facilitate the
functionalities of temporal verification.
25
Conclusion

Temporal verification is important in scientific workflows

Key research issues and challenges
 Constraint Setting: the location of temporal constraints
 Checkpoint Selection: efficiency, computation cost
 Temporal Verification: efficiency, different consistency
 Temporal Adjustment: how to compensate time deficit

The research on scientific workflow temporal verification
is still in its infancy and requires more efforts.
26
The End
 Thanks for
your attention!
27