Scaling and Validation Programme
David Groep & vle-pfour-team
VL-e SP Meeting 2004.12.07
NIKHEF
SARA
LogicaCMG
IBM
Aims of the Scaling & Validation (S&V)
• Build environments for experimentation
 different scales, with a variety of resources
 stable quasi-production Proof-of-Concept (PoC)
• Support applications
 stable, but also extend (and improve) continuously
 deploy VL software and components from elsewhere
• Disseminate knowledge about them
 tutorials, events, helpdesk
• Learn ourselves from the deployment
 close the cycle, improve with developers and users
 S&V needs applications for a realistic workload
Positioning of P4 in the Project
Medical
Diagnosis &
Imaging
Characteristics
Usage
Initial compute
platform
Environments
BioDiversit
y
BioInformatics
Stable, reliable, tested
Cert. releases Grid MW & VLsoftware
Data
Intensive
Science/
Food
Informatics
Flexible, test environment
Dutch
Telescience
Flexible, ‘unstable’
Application development
Test & Cert.
Grid MW & VL-software
Compatibility
Virtual Lab.
rapid prototyping
(interactive simulation)
NL-Grid production cluster
Central mass-storage facilities
+SURFnet
NL-Grid Fabric
Research Cluster
DAS-2, local resources
VL-e Proof of Concept Environment
VL-e Certification
Environment
VL-e Rapid
Prototyping
Environment
external
developments
Three tracks
• Software engineering
 Rapid Prototyping  Certification  PoC
• Deploy a national-scale infrastructure
 central facilities (clusters, storage,DBs)
 access nodes on-site
 software installations
• Knowledge dissemination and import
 tutorials (and support for ~)
 pilots (think: SRB)
 developer & admin events (think: GANG)
PoC Software Release Methodology
PoC Release n
Characteristics
Stable, reliable, tested
Cert. releases Grid MW & VLsoftware
Application development
Usage
Initial compute
platform
Environments
Release Candidate
n+1
Developers
Heaven/Haven
Flexible, test environment
Flexible, ‘unstable’
Integration tests
Functionality tests
Test & Cert.
Adventurous
Grid MW & VL-software
application
people
Compatibility
NL-Grid production cluster
Central mass-storage facilities
+SURFnet
NL-Grid Fabric
Research Cluster
VL-e Proof of Concept Environment
VL-e Certification
Environment
GT2
Virtual Lab.
rapid prototyping
(interactive simulation)
DAS-2, local resources
GT3.2
GT2
VL-e Rapid
Prototyping
Environment
Tagged
Release Candidates
Download Repository
PoC Installer
Cluster Tools
Developer CVS
Nightly builds
Unit tests
stable, tested
releases
Software engineering process and tools
• Source code repository (CVS)
 also for general VL-e use
• For generally deployed VL-e components
 software engineering guidelines
 unit and integration test framework
 automatic builds
• Software download repository
Example: CruiseControl
Integrated results of
unit tests
No tests yet
Artifacts
produced for
deployment
Deployment: VL-e Proof-of-Concept
• Today:
 Basis of the PoC is an enhanced
Globus Toolkit™ 2.x serviced by VDT
 VLAM-G installed **
o
o
Matisse database for module definitions
s/w installed when cvs restructuring complete**
 Tools from EDG and CrossGrid
 SRB (in pilot phase)
• your requirements feed into upgrades
Continuous updates
• Requirements from applications
 gathered in (bilateral) meetings
• Based on your use cases we will
modify, upgrade, and update the PoC




other workflow tool…
other database server…
…
‘time-boxed’ releases
• PoC should be unified generic infrastructure
Resources and systems
• Central resource sites today




SARA: NCF NL-Grid Matrix cluster
SARA: Storage (Teras), SRB Matrix
NIKHEF: NDPF LCG2PROD + NCF GFRC
all: services
• Access node install via PoC Installer
 ‘keep pressing enter and you get there…’
• Supported OS’s
 RedHat Enterprise 3 + clones (CentOS, SL)
(works on Fedora Core *)
 RedHat 7.3
PoC Installer
Ease installation of GT2 and VO support:
• a User Interface & job submission i/f
• GridFTP server
• Authentication and VO-LDAP
• Globus 2 Development environment
Dissemination & Support
• Core Grid and GT2 tutorials
• Hands-on web services event
the Grid Admin Nerd Group (GANG)
• User help-desk for the PoC
(020) 592 8008, [email protected]
• Documentation
• Technology pilots: SRB, …
Example: the GANG
organized by LogicaCMG, IBM, NIKHEF, SARA together with GridForum Nederland
• expose people to the use of web/grid services
 December event based on GT3.2
 future will use GT4, Axis+Apollo, …
• this is the likely future for new software
 leverage industry support and standards
 Web Service Resource Framework, Notification, … now
in Apache Incubator
 OGSA to describe service architecture
• event will be repeated with new technology
Composition of P4 team
• NIKHEF, SARA, LogicaCMG, IBM
• will grow to ~8 FTE effective in 2005
 still one position to be filled @NIKHEF
Summary
• Infrastructure: the PoC is there for you
 hardware, Grid software, DBs, VLAM-G s/w
• Software engineering support
• Central services are available
 SourceForge look-alike
 software distribution, web site, mailing lists
• Support and training
• Think about your requirements and wishes –
and YOU tell us what you want!
 Type of support, tools, …
 We could take the PoC to your site (if you want)
 and we will visit you again…
Contacts
• Contact addresses




[email protected] – services, PoC, web, PoC-installer
[email protected] – certification, testing, tooling, …
[email protected] – PoC@SARA, SRB pilot, …
grid.support@{sara,nikhef}.nl – user questions
Or reach all of us at
[email protected]