TheScienceDMZ:
RecentDevelopments
EliDart,NetworkEngineer
ESnetScienceEngagement
LawrenceBerkeleyNa@onalLaboratory
WRNP17
Belém,Brazil
May16,2017
©2017,EnergySciencesNetwork
Overview
•  ScienceDMZAsPlaMorm
•  ModernResearchDataPortal
•  PacificResearchPlaMorm
– PRP
– NRP
•  Note:ThistalkassumesyoualreadyunderstandtheScienceDMZ
–  Ifyouhaven’tencounteredtheScienceDMZ,severalfolksinRNPcan
helpyou,includingLeandroCiuffoandAlexMoura
–  Orcheckoutthefasterdataknowledgebase:
•  hXp://fasterdata.es.net/science-dmz/
2 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ScienceDMZAsAPla3orm
•  OncetherearemanyScienceDMZsinyournetwork,more
thingsbecomepossible
•  Easyfiletransferisgood,butwhatelsecanwedo?
–  Updatethearchitectureofdataportals
–  Buildservicesbetweenins@tu@ons
–  Interconnectfacili@es
•  Severaleffortsunderwaytodothesethings
3 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ScienceDataPortals
•  Largerepositoriesofscien@ficdata
–  Climatedata
–  Skysurveys(astronomy,cosmology)
–  Manyothers
–  Datasearch,browsing,access
•  Manyscien@ficdataportalsweredesigned15+yearsago
–  Single-web-serverdesign
–  Databrowse/search,dataaccess,userawarenessallinasinglesystem
–  Allthedatagoesthroughtheportalserver
•  Inmanycasesbydesign
•  E.g.embargobeforepublica@on(enforceaccesscontrol)
4 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
LegacyPortalDesign
Border Router
perfSONAR
Firewall
WAN
perfSONAR
•  Verydifficulttoimproveperformance
withoutarchitecturalchange
–  Sodwarecomponentsalltangled
together
–  Difficulttoputthewholeportalina
ScienceDMZbecauseofsecurity
–  EvenifyoucouldputitinaDMZ,many
componentsaren’tscalable
Enterprise
Browsing path
Query path
Data path
10GE
Portal server applications:
· web server
· search
· database
· authentication
· data service
Portal
Server
10GE
Filesystem
(data store)
•  Whatdoesarchitecturalchangemean?
5 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ExampleofArchitecturalChange–CDN
•  Let’slookatwhatContentDeliveryNetworksdidforwebapplica@ons
•  CDNsareawell-deployeddesignpaXern
–  Akamaiandfriends
–  En@reindustryinCDNs
–  Assumedpartoftoday’sInternetarchitecture
•  WhatdoesaCDNdo?
–  Storesta@ccontentinaseparateloca@onfromdynamiccontent
•  Complexityisn’tinthesta@ccontent–it’sintheapplica@ondynamics
•  Webapplica@onsarecomplex,full-featured,andslow
–  Databases,userawareness,etc.
–  Lotsofintegratedpieces
•  Dataserviceforsta@ccontentissimplebycomparison
–  Separa@onofapplica@onanddataserviceallowseachtobeop@mized
6 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ClassicalWebServerModel
•  Webbrowserfetchespagesfromwebserver
–  Allcontentstoredonthewebserver
–  Webapplica@onsrunonthewebserver
•  Webservermaycallouttolocaldatabase
•  Fundamentallyallprocessingislocaltothewebserver
–  Webserversendsdatatoclientbrowseroverthenetwork
•  Perceivedclientperformancechangeswithnetworkcondi@ons
–  Severalproblemsinthegeneralcase
–  Latencyincreases@metopagerender
–  Packetloss+latencycauseproblemsforlargesta@cobjects
Web Server
WEB
Hosting
Provider
Transit
Network
Residential
Broadband
Browser
Long Distance / High Latency
7 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
SoluFon:PlaceLargeStaFcObjectsNearClient
•  CDNprovidessta@ccontent“close”toclient
–  Latencygoesdown
•  Timetopagerendergoesdown
•  Sta@ccontentperformancegoesup
–  Loadonwebservergoesdown(noneedtoserve
sta@ccontent)
–  Webservers@llmanagescomplexbehavior
•  Localreasoning/fastchangesforapplica@onowner
CDN Data
Server
DATA
CDN
•  Significantwinforwebapplica@onperformance
Short Distance /
Low Latency
Web Server
WEB
Hosting
Provider
Transit
Network
Residential
Broadband
Browser
Long Distance / High Latency
8 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ClientSimplySeesIncreasedPerformance
•  Clientdoesn’tseetheCDNasaseparatething
–  Webcontentisalls@llviewedinabrowser
•  Browserfetcheswhatthepagetellsittofetch
•  Differentcontentcomesfromdifferentplaces
•  Userdoesn’tknow/care
•  CDNsprovideanarchitecturalsolu@ontoa
performanceproblem
–  Notbrute-force
–  Worksmarter,notharder
CDN Data
Server
DATA
Simple,
Fast
Web Server
WEB
Web Server
The
‘Net
WEB
The
‘Net
Browser
Browser
Rich, Slow
9 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
ArchitecturalExaminaFonofDataPortals
•  Commondataportalfunc@ons(mostportalshavethese)
–  Search/query/discovery
–  Datadownloadmethodfordataaccess
–  GUIforbrowsingbyhumans
–  APIformachineaccess–ideallyincorporatessearch/query+download
•  Performancepainisprimarilyinthedatahandlingpiece
–  Rapidincreaseindatascaleeclipsedlegacysodwarestackcapabili@es
–  Portalserversodenstuckinenterprisenetwork
•  Canwe“disassemble”theportalandputthepiecesbacktogetherbeXer?
–  UseScienceDMZasaplaMormforthedatapiece
–  AvoidplacingcomplexsodwareintheScienceDMZ
10 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
LegacyPortalDesign
Border Router
perfSONAR
Firewall
WAN
perfSONAR
Enterprise
Browsing path
Query path
Data path
10GE
Portal server applications:
· web server
· search
· database
· authentication
· data service
Portal
Server
10GE
Filesystem
(data store)
11 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
Next-GeneraFonPortalLeveragesScienceDMZ
perfSONAR
Border Router
Firewall
WAN
Enterprise
Data Path
Browsing path
Query path
10GE
perfSONAR
perfSONAR
Portal server applications:
· web server
· search
· database
· authentication
Science DMZ
Switch/Router
Portal
Server
10GE
10GE
Data Transfer Path
10GE
10GE
10GE
10GE
DTN
10GE
Portal Query/Browse Path
DTN
DTN
12 – ESnet Science Engagement ([email protected]) - 5/15/17
DTN
Filesystem
(data store)
10GE
API DTNs
(data access governed
by portal)
©2017,EnergySciencesNetwork
PutTheDataOnDedicatedInfrastructure
•  Wehaveseparatedthedatahandlingfromtheportallogic
•  Portaliss@llitsnormalself,butenhanced
–  PortalGUI,database,search,etc.allfunc@onastheydidbefore
–  QueryreturnspointerstodataobjectsintheScienceDMZ
–  Portalisnowfreedfrom@estothedataservers(runitonAmazonifyou
want!)
•  Datahandlingisseparate,andscalable
–  High-performanceDTNsintheScienceDMZ
–  Scaleasmuchasyouneedtowithoutmodifyingtheportalsodware
•  Outsourcedatahandlingtocompu@ngcentersorcampuscentralstorage
–  Compu@ngcentersaresetupforlarge-scaledata
–  Letthemhandlethelarge-scaledata,andlettheportaldotheorchestra@on
ofdataplacement
13 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
The Pacific Research Platform Creates a Regional End-to-End
Science-Driven “Big Data Freeway System”
NSF CC*DNI Grant
$5M 10/2015-10/2020
•  PI: Larry Smarr, UC San
Diego Calit2
•  Co-PIs:
-  Camille Crittenden, UC
Berkeley CITRIS,
-  Tom DeFanti, UC San Diego
Calit2,
-  Philip Papadopoulos, UC
San Diego SDSC,
-  Frank Wuerthwein, UC San
Diego Physics and SDSC
PRPProvidesInteroperability
•  ScienceDMZsatpar@cipa@ngsitesensureinteroperability
•  PRPengineersworktoensuretheyinteroperate
–  GlobusdatatransferbetweenDTNs
–  perfSONAR
•  Somevaria@oninDTNs
–  SomehaveFIONADTNs
•  FIONA==FlashI/ONetworkAppliance
•  DesignedbyPRPengineersatUCSanDiego
•  hXps://fasterdata.es.net/science-dmz/DTN/fiona-flash-i-o-network-appliance/
–  SomehaveDTNsconnectedtoHPCstorage
•  Key–theyallinteroperate,removingintegra@onburdenfromscien@sts
15 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
PRPScienceDrivers
•  Mul@plescienceareas
–  Astronomyandastrophysics
–  Biomedicalapplica@ons
–  Lifesciences
–  Par@clephysics
–  Virtualrealityanddatavisualiza@on
•  hXp://prp.ucsd.edu/
16
5/15/17
NaFonalResearchPla3orm(NRP)
•  ReplicatethePRPonana@onalscale
•  Interoperable,high-performancecyberinfrastructure
–  Builttoservedomainscience
–  Scaleupto~200ins@tu@ons
•  Firstworkshoptobeheldthissummer
–  Domainscienceinput
–  Policyques@ons
–  Architecture,scalability
–  IncludecampusIT,regionalnetworks,na@onalnetworks,funding
agencies,etc.inacommonconversa@on.
17 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
PetascaleDTNProject
•  AnotherexampleofbuildingontheScienceDMZ
•  Supportsalldata-intensiveapplica@onswhichrequirelarge-scaledata
placement
•  Collabora@onbetweenHPCfacili@es
–  ALCF,NCSA,NERSC,OLCF
•  Goal:per-Globus-jobperformanceat1PB/weeklevel
–  15gigabitspersecond
–  Withchecksumsturnedon,etc.
–  Nospecialshortcuts,noarcaneop@ons
•  Referencedatasetis4.4TBofastrophysicsmodeloutput
–  Mixoffilesizes
–  Manydirectories
–  Realdata!
18 – ESnet Science Engagement ([email protected]) - 5/15/17
©2017,EnergySciencesNetwork
PetascaleDTNProject
alcf#dtn_mira
ALCF
March 2017
L380 Data Set
DTN
16.3 Gbps
18.3 Gbps
nersc#dtn
NERSC
24.1 Gbps
32.4 Gbps
17.0 Gbps
17.4 Gbps
DTN
25.3 Gbps
DTN
olcf#dtn_atlas
OLCF
19.3 Gbps
17.6 Gbps
24.0 Gbps
10.0 Gbps
DTN
ncsa#BlueWaters
NCSA
19 – ESnet Science Engagement ([email protected]) - 5/15/17
14.8 Gbps
Data set: L380
Files: 19260
Directories: 211
Other files: 0
Total bytes: 4442781786482 (4.4T bytes)
Smallest file: 0 bytes (0 bytes)
Largest file: 11313896248 bytes (11G bytes)
Size distribution:
1 - 10 bytes: 7 files
10 - 100 bytes: 1 files
100 - 1K bytes: 59 files
1K - 10K bytes: 3170 files
10K - 100K bytes: 1560 files
100K - 1M bytes: 2817 files
1M - 10M bytes: 3901 files
10M - 100M bytes: 3800 files
100M - 1G bytes: 2295 files
1G - 10G bytes: 1647 files
10G - 100G bytes: 3 files
©2017,EnergySciencesNetwork
Thanks!
[email protected]
EnergySciencesNetwork(ESnet)
LawrenceBerkeleyNa@onalLaboratory
hXp://fasterdata.es.net/
hXp://my.es.net/
hXp://www.es.net/
ExtraSlides
21
5/15/17
WhatIsScienceEngagement?
•  Technologypeopleworkingwithscien@ststohelpsolveproblems
–  Improvedatatransferperformance
–  Improvedataworkflows(e.g.torequirelesshumaneffort)
–  Improveexperimentopera@ons
–  …andmore…
•  Usingexperiencegainedfromhelpingscien@ststoimprove
cyberinfrastructure
–  Networkdesign
–  Tooldesign
–  Systemdesign
22
5/15/17
EngagementIsImportant:OldModel
•  Scien@stasintegrator
–  Requiresscien@ststodiscovernewtechnologies
–  Requiresscien@ststobecomeexpertinnewtechnologies
–  Requiresscien@ststoassembledis@ncttechnologiesintoan
integratedsolu@onthatworksforthem
–  Somescien@stsdothisbrilliantly–mostdonot
23
5/15/17
EngagementIsImportant:NewModel
•  Scien@stascollaborator
–  Technologistsunderstandtechnology
–  Technologistsunderstandenoughofthesciencetoseehow
technologyfits
–  Technologistshelpscien@stsadoptausefulsolu@on
–  Thisismuchmoreproduc@ve,andrequiresscience
engagement
24
5/15/17