QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
PetaShare All Hands
Meeting
State of the Union
Tevfik Kosar
Center for Computation & Technology
Louisiana State University
DOSAR Workshop
April 18, 2008
LOUISIANA
STATE UNIVERSITY
CCT: Center for Computation &AT
Technology
@ LSU
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Goal: enable domain scientists to focus on their primary
research problem, assured that the underlying
infrastructure will manage the low-level data handling
issues.
• Novel approach: treat data storage resources and the
tasks related to data access as first class entities just
like computational resources and compute tasks.
• Key technologies being developed: data-aware storage
systems, data-aware schedulers (i.e. Stork), and cross-domain
meta-data scheme.
• Provides and additional 250TB disk, and 400TB tape storage
(and access to national storage facilities)
CCT: Center for Computation & Technology @ LSU
Participating institutions in the PetaShare project, connected through
LONI. Sample research of the participating researchers pictured (i.e.
biomechanics by Kodiyalam & Wischusen, tangible interaction by
Ullmer, coastal studies by Walker, and molecular biology by Bishop).
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Sensor Networking
LaTech
High Energy Physics
Biomedical Data Mining
LSU
UNO
Coastal Modeling
Petroleum Engineering
Computational Fluid Dynamics
Computational Biology
Numerical Relativity
Biophysics
Tulane
ULL
Geology
Petroleum Engineering
Sensor Networking
Molecular Biology
Computational Electrophysiology
Infrastructure Overview
LaTech
ULL
LSU
UNO
Tulane
~ 100 TFLOPS
~8 TB RAM
250 TB Disk
400 TB Tape
Managed
Data
Movement
SDSC
50 TB
CCT: Center for Computation & Technology @ LSU
Replica & Meta
Data Management
Caching/
Prefetching
Data
Migration
HSM
Scheduled Data
Movement
CCT: Center for Computation & Technology @ LSU
PetaShare Core
POSIX interface:
- NO need to change code
- NO relinking
- NO recompiling
- NO privileged access
ULL
UNO
LSU
Tulane
petashell
SDSC
Web interface:
PetaSearch
LaTech
petashell
• a POSIX compatible shell interface to
PetaShare
$ petashell
psh% cp /tmp/foo.txt /petashare/tulane/tmp/foo.txt
psh% vi /petashare/tulane/tmp/foo.txt
psh% cp /tmp/foo2.dat /petashare/anysite/tmp/foo2.dat
psh% genome_analysis genome_data -->
psh% genome_analysis /petashare/uno/genome_data
psh% exit
$
CCT: Center for Computation & Technology @ LSU
Cross-Domain Metadata Search
• SCOOP – Coastal Modeling
– Model Type, Model Name, Institution Name, Model Init Time, Model
Finish Type, File Type, Misc information …
• UCoMS – Petroleum Engineering
– Simulator Name, Model Name , Number of realizations, Well number,
output, scale, grid resolution …
• DMA – Scientific Visualization
– Media Type, Media resolution, File Size, Media subject, Media Author,
Intellectual property information, Camera Name …
• NumRel - Astrophysics
– Run Name, Machine name, User Name, Parameter File Name, Thorn
List, Thorn parameters …
CCT: Center for Computation & Technology @ LSU
PetaSearch
Search Keywords: Katrina
In Archives:
SCOOP
UCoMS
NumRel
Digital Media
....
All Archives
CCT: Center for Computation & Technology @ LSU
Latest Developments
• The PetaShare services are installed on two sites so far
(LSU and UNO) and ready to use.
• We have developed first prototype of a tool called “petafs”
which provides a user-level virtual file system across all
PetaShare sites.
• LBIRN (Louisiana Bioinformatics Research Network)
decided to join PetaShare Storage Network. This
increases the number of PetaShare sites from 5 to 9, and
the total disk storage from 240TB to 300TB.
CCT: Center for Computation & Technology @ LSU
A system driven by the local needs (in LA), but
has potential to be a generic solution for the
broader community!
Hmm..
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
For more information on PetaShare:
http://www.petashare.org
CCT: Center for Computation & Technology @ LSU