Gridbus Resource Broker for Application Service
Costs-based Scheduling on Global Grids: A
Case Study in Brain Activity Analysis
GRIDS
Srikumar
Venugopal1,
Rajkumar
Buyya1,
Susumu
Date2
1.Grid computing & Distributed Systems (GRIDS) Lab.
The University of Melbourne
Melbourne, Australia
www.gridbus.org/
2. Cybermedia Center, Osaka University
What does a Resource Broker do?





Gets user/application
requirements
Discovers resources like
computational nodes, data
sources, etc.
Establishes costs, user
credit, etc.
Makes decisions about
the optimal schedule for
jobs
Dispatches jobs
Application
Accounting
Services
Information
Services
Resource
Broker
Cataloguing
Services
…
Grid Nodes
Architecture of Gridbus Scheduler
Application
Visual Parametric Tool
Data Catalogue
Gridbus
Scheduler
Grid Info Service
ASP Catalogue
Access Service
(Globus)
Grid Market Directory
Bill
Globus
Globus
CPU
or
PE
Grid Node
(e.g., ANL)
Cluster Scheduler
PE
Grid Node
(e.g., UofM)
PE
GTS
Grid Node
(e.g., VPAC)
GridBank
Grid Node
Gridbus Scheduler

Interfaces with:





Application Development - Visual Parametric Tool
Information Services - Grid Market Directory (Cost), GRIS ,etc.
Accounting Services - Grid Trading Service, GridBank
Cataloguing Services - Application Catalog, Replica Catalog
Job Dispatcher



Nimrod-G (for parametric jobs)
Gridbus Dispatcher (for data intensive, reservation, P-GRADE
support, etc.) – work in progress
Supports:




User-specified QoS parameters such as Deadline, budget, etc.
Application Cost or Hardware Cost (CPU, etc)
Cost from Grid Market Directory or Flat File
Cost, Time or Cost-Time Optimization.
Application Service Costs?

Present Approach to Processing Cost 


Timeshare or CPU cycles used
Users – more interested in the cost of getting job
done than amount of processing power consumed
New Approach to Cost 


Application Service Costs – charge for using the
application once.
Different costs for different applications – depends
on provider
Broker finds Cost through Grid Market Directory.
Scheduling Algorithms
Gridbus Scheduler implements
 Cost Optimization


Time Optimization


Minimize computational cost (within
deadline)
Minimize execution time (within budget)
Cost-time Optimization


Similar to cost-optimization
Implemented for first time.
Scheduling (contd..)



Uses past performance to forecast each machine’s
capacity
The rate of completion is averaged to compensate for
any spikes or troughs
Cost Optimization


Time Optimization


Gives maximum jobs to the cheapest machine
Gives jobs to machines based on consumption rate but limited
by budget per job
Cost-Time Optimization

Distributes jobs among the machines of consumption sorted by
their consumption rate
Cost Optimization: No. of Jobs Done vs time
160
140
120
100
80
60
40
20
6.
33
9.
50
12
.8
3
16
.0
0
19
.1
7
22
.3
3
25
.5
0
29
.8
3
33
.0
0
36
.1
7
39
.5
0
42
.6
7
45
.8
3
49
.0
0
52
.1
7
54
.3
3
57
.5
0
60
.8
3
64
.0
0
67
.1
7
70
.3
3
73
.5
0
76
.8
3
80
.0
0
83
.1
7
3.
17
0
hpc76.ai.iit.nrc.ca
gnet01.hpc.unimelb.edu.au
herschel.amtp.cam.ac.uk/jobmanager-pbs
cabibbo.physics.usyd.edu.au
lem.ph.unimelb.edu.au
lem.ph.unimelb.edu.au/jobmanager-pbs
date1.ics.es.osaka-u.ac.jp
node1001.gridcenter.or.kr/jobmanager-pbs
node2001.gridcenter.or.kr
node2001.gridcenter.or.kr/jobmanager-pbs
fs0.das2.cs.vu.nl/jobmanager-pbs
fs0.das2.cs.vu.nl/jobmanager-pbs
bg01n000e00.hpc.cmc.osaka-u.ac.jp
bg01n000e00.hpc.cmc.osaka-u.ac.jp
Cost-Time Optimization: No. of Jobs Done vs Time
50
45
40
No.of Jobs done
35
30
25
20
15
10
5
0
0.00 1.17 2.17 3.33 4.33 5.33 6.50 7.50 8.67 9.67 10.67 11.83 12.83 14.00 15.00 16.00 17.17 18.17 19.33
Time (in min.)
hpc76.ai.iit.nrc.ca
gnet01.hpc.unimelb.edu.au
herschel.amtp.cam.ac.uk/jobmanager-pbs
cabibbo.physics.usyd.edu.au
lem.ph.unimelb.edu.au
lem.ph.unimelb.edu.au/jobmanager-pbs
date1.ics.es.osaka-u.ac.jp
node1001.gridcenter.or.kr/jobmanager-pbs
node2001.gridcenter.or.kr
node2001.gridcenter.or.kr/jobmanager-pbs
fs0.das2.cs.vu.nl
fs0.das2.cs.vu.nl/jobmanager-pbs
bg01n000e00.hpc.cmc.osaka-u.ac.jp
bg01n000e00.hpc.cmc.osaka-u.ac.jp
Time Optimization: No. of Jobs Done vs Time
40
35
30
25
20
15
10
5
0.
00
1.
00
2.
17
4.
17
5.
33
6.
33
7.
50
8.
50
9.
50
10
.6
7
11
.6
7
12
.8
3
13
.8
3
14
.8
3
16
.0
0
17
.0
0
18
.1
7
19
.1
7
20
.1
7
21
.3
3
22
.3
3
23
.5
0
23
.5
0
24
.5
0
25
.5
0
26
.6
7
0
hpc76.ai.iit.nrc.ca
gnet01.hpc.unimelb.edu.au
herschel.amtp.cam.ac.uk/jobmanager-pbs
cabibbo.physics.usyd.edu.au
lem.ph.unimelb.edu.au
lem.ph.unimelb.edu.au/jobmanager-pbs
date1.ics.es.osaka-u.ac.jp
node1001.gridcenter.or.kr/jobmanager-pbs
node2001.gridcenter.or.kr
node2001.gridcenter.or.kr/jobmanager-pbs
fs0.das2.cs.vu.nl
fs0.das2.cs.vu.nl/jobmanager-pbs
bg01n000e00.hpc.cmc.osaka-u.ac.jp
bg01n000e00.hpc.cmc.osaka-u.ac.jp
Comparison of Scheduling Algorithms

All experiments were started with



No of Jobs = 200
Deadline = 2hrs
Budget = 600 Grid $
Start Time
Completion Time
Budget
Consumed
(Grid $)
Cost
10:00 a.m.
11:27 a.m.
188
Cost-Time
11:40 a.m.
12:08 p.m.
277
Time
12:30 p.m.
12:59 p.m.
274
Case Study: Brain Activity Analysis


In Collaboration with Osaka University,
Japan
Computationally and data intensive
MEG Data/Brain Activity Analysis

MEG (Magnetoencephalography)



Achieve both non-invasiveness and high degree of
measurement accuracy
cf. EEG (Electroencephalography), ECoG
(Electrocorticography)
Measure functional data on multiple points around the head
Promising among medical doctors and brain scientists.
A
B
A:
B:
http://www.ctf.com
MEG data analysis
Osaka Univ.
Osaka Univ. Hospital
DV transfer
Analysis Results
Data Generation
Data Analysis
Analysis Results
Cybermedia Center
Life-electronics laboratory,
AIST
•Provision of MEG
•Provision of expertise in
the analysis of brain function
MEG data analysis
Osaka Univ.
Osaka Univ. Hospital
Cybermedia Center
DV transfer
Analysis Virtual
Results
Data Generation
Laboratory
for medicine and brain science
•Knowledge sharing
Data Analysis
•MEG sharing?
Analysis Results
•Data Sharing
Life-electronics laboratory,
AIST
•Provision of MEG
•Provision of expertise in
the analysis of brain function
Requirements



Computational and data intensive problem
The number of MEG instruments available is
small.
Knowledge of scientists is distributed.



No database?
Different group uses different analysis methods for
different data..
Many medical institutions and hospitals have
no computers and that can satisfy doctors’
analysis demand.
Wavelet cross-correlation analysis
Sensor B
Sensor A
Raw MEG Data
f
f
t
t
f
t
f’
At 1st Phase,
wavelet transform
At 2nd Phase,
Wavelet cross-correlation
This image indicates that a brain signal with frequency f’ was detected
earlier in Sensor B than in Sensor A.
This analysis procedure needs to be performed for each
pair of MEG sensors. E.g. 64ch -> 2016
New Approach: Users QoS Requirements driven MEG
Data Analysis on the Grid
64 sensors MEG
Analysis All pairs (64x64) of MEG data by
shifting the temporal region of MEG data
over time: 0 to 29750: 64x64x29750 jobs
2
Data Generation
3
1
5
Results
Data Analysis
Grid Resource Broker
(Nimrod-G+Gridbus)
4
Life-electronics laboratory,
AIST
•Provision of MEG analysis
[deadline, budget, optimization preference]
World-Wide Grid
Grid Enabling MEG data analysis

Nature



Data Sets on Source Node





High Latency for small jobs
Lower Efficiency
Hence, data sets were replicated on each node
Application changed to access local datasets


fine-grained jobs
small data sets
./metameg-datapath time_offset time_offset_step
meg_sensors_count Meg_data_path
Output is collated at the source node and then
visualized
Grid Enabled in very short time ~ 1 week
Conclusion




Introduced Gridbus Resource Broker using
Application Service Cost
Described the Scheduling Algorithms followed
Presented Case Study of Brain Activity Analysis
using our Resource Broker
Future Work:


Integration with Accounting Mechanisms such as
GridBank
Support for Group Scheduling and Economic-based
Advance Reservation of Resources