Genentech`s IT Infrastructure

Genentech’s IT Infrastructure:
Evolving to support the Revolution
John “Scooter” Morris, Ph.D.
Distinguished Systems Architect
Computing Technologies
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 1
Genentech, Inc.
“Genentech is a leading biotechnology company that
discovers, develops, manufactures and commercializes
biotherapeutics for significant unmet medical needs.”
Statistics
• ~5,000 Employees
• ~$2.2B in Revenue (2001)
• 10 products
- Protropin®, Nutropin®, NutropinAQ®, NutropinDepot®,
Cathflo™ Activase®, Activase®, TNKase™, Pulmozyme®,
Herceptin®, Rituxan®
• 1 product awaiting FDA approval
- Xolair™
• Three major sites
-
Scooter Morris, CIT
([email protected])
South San Francisco, California
Vacaville, California
Porriño, Spain
Several U.S. Sales offices
Genentech IT Infrastructure
June 12, 2002 page 2
Genentech, Inc.
Founded in 1976
• Herb Boyer – UCSF Professor
• Bob Swanson – Entrepreneur, Venture Capitalist
Focus areas:
• Oncology
- 7 drugs or new indications in pipeline
- 4 in phase III
• Immunology
- 5 drugs or new indications in pipeline
- 2 in phase III, 1 awaiting approval
• Opportunistic
- 3 drugs or new indications in pipeline
- 1 in phase III
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 3
Clinical Development of Drugs
Discovery
Marketing and
Line Expansion
Development
Post marketing studies
Synthesis and testing
New clinical
indications pursued
IND plan established
and initiated
Chemical lead found
New dosage forms and
formulations developed
IND filed
Phases I, II, III
Additional compounds
Clinical studies initiated
are made
NDA prepared
Candidate compound and submitted
chosen and additional
tests run
NDA approved
Phase IV
Idea for new chemical Compound elevated
to project status
Safety surveillance
Drug launched
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 4
Outline
Current Infrastructure
• A bunch of details which I will skip
Research Computing
Revolution vs. Evolution
Evolution of Computing for Research
• Evolutionary tree
Future Directions
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 5
Current IT Infrastructure
Highly heterogeneous
• Servers: SGI, HP Alpha, HP PA-RISC, HP Intel, Sun
• Desktops: Mac, PC, some SGI
Primarily IP-based network
• AppleTalk also supported
• Routers and switches: Cisco
Security based on M&M principle
• Hard outer shell, soft inside
• Some “softness” appearing to support collaboration
• Important to maintain open environment
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 6
Current IT Infrastructure
Building
Building
Switch
200
Switch
bps
200 M
200
Mbp
s
Cisco
Router
2G
bps
200 Mb
ps
Cisco
Router
2 Gbps
Cisco
Router
Cisco
Router
Cisco
Router
Cisco
Router
2
2G
bp
s
Vacaville
200 Mbps
G
Switch
bp
s
Switch
Switch
Cisco
Router
Switch
Switch
2 Gbps
Cisco
Router
s
Mbp
Cisco
Router
2 Gbps
Switch
Building 4 Computer Room
Scooter Morris, CIT
([email protected])
Building 5 Computer Room
Genentech IT Infrastructure
June 12, 2002 page 7
Current IT Infrastructure
Key
Firewalls & proxies
Web & file sharing
VPN
DNA
DNA
Cisco
Router
Switch
Cisco
Router
Cisco
Router
genie
gnome
efreet
djinn
VPN concentrators
Switch
Switch
Cisco
Router
Cisco
Router
Cisco
Router
UUNET Genuity
wallace-ltd
outcast2
www-secure
outcast
Cisco
3002
Partners
Partners
(frame)
(frame)
Limited Net
Scooter Morris, CIT
([email protected])
Sales
Offices
Genentech
Spain
Cisco
3002
Internet
Internet
Partners
Partners
(IPSEC)
(IPSEC)
Open Net
Genentech IT Infrastructure
June 12, 2002 page 8
Details
This starts the part I’m going to leave
out….
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 9
Details - Network
Backbone: Switched Gigabit Ethernet
Vacaville link: 200 Mbps SONET Ring
Desktop: Switched 10/100 Ethernet
Routers: CISCO
Addressing: DHCP preferred
Naming: DNS (Bind 9.2.1), WINS, Active Directory, LDAP
Firewall: SOCKS5 (Aventail)
Monitoring: Big Brother, HP Open View
VPN: Cisco 3000, IPSEC
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 10
Details - Desktop
Compaq (now HP)
• Migrating to Windows 2000
Apple
• Migrating to Mac OS X
Primary Applications
• Microsoft Office 2000/2001
• Netscape Communicator (Browser, Mail)
- Migrating to Mozilla/Netscape 7
• Steltor CorporateTime
• Norton Antivirus
• FileMaker Pro
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 11
Details - Server
HP Tru64 Unix
• Web, E-Mail (IMAP), Bioinformatics, Infrastructure (DNS, Firewall,
DHCP, backup/restore, LDAP), General computing, Oracle
- 5.1A (TruCluster Server 5.1A)
HP/UX
• Manufacturing, Commercial Computing (Lawson, PeopleSoft)
• 10.20, 11.0
Solaris
• Medical Affairs, Infrastructure (Calendar, Remedy, Web Proxy),
Research
SGI
• Molecular Modeling, Computational Chemistry
Linux
• Computational Chemistry (pilot)
NT
• Workgroup Computing, Specific Applications
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 12
Details - Database
Oracle 8.1.7
• Exploring Oracle 9iRAC
n-tier approach
•
•
•
•
Scooter Morris, CIT
([email protected])
Web Browser for presentation
Web server for static pages
Application servers for business logic
Database server for data store
Genentech IT Infrastructure
June 12, 2002 page 13
Details - Web
Server: Netscape Enterprise Server 4.6
• Migrating to Apache
Programming: Perl/CGI, JSP, Javascript
Application Servers:
• WebLogic
Distributed Computing:
• Direction is towards Enterprise Java Beans
- WebLogic
- Tuxedo in use for Manufacturing applications
Development Tools:
• Dreamweaver, JBuilder, Visual Age
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 14
Details - Security
Security is based on Kerberos V5
Provides authentication for Unix and Windows 2000
Oracle accounts often use Unix username, but also
lots of application-specific accounts
• Exploring use of Kerberos for Oracle accounts
VPN and dial-in access through SecurID tokens
LDAP is used for Directory services
• Netscape Directory Server
Serious regulatory restrictions (21CFR Part 11)
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 15
Details - Internet/Extranet
Firewall is based on SOCKS5 (RFC1928)
Totally Proxy-based (very secure)
Firewall has three parts:
• Internal
• Internet
• Limited net
Internet link is redundant
• 42 Mbps link with Genuity
• T1 (1.54 Mbps) link with UUNET
- Migrating to a redundant 42 Mbps link
• Uses OSPF for dynamic fail-over
• Using IPSEC tunnels over the Internet for secure communications with
partners
Cisco 3000 VPN concentrators
• Employee access only
Reverse-web Proxy
• Allows external partners access to selected internal web sites
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 16
Details
Any questions on the details?
• I didn’t think so….
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 17
The Revolution
(a parochial view)
Biochemical Cycle
Systemics
Protein
Boyer & Cohen
1973
Itakura & Riggs
1977
Proteomics
Human Genome Project
1990
PCR
1983
ESTs
1990
Gene
1978
Insulin Cloned
ABI Sequencers
1994
Human Genome Draft
2001
Genomics
1985
Human Growth Hormone Approved
1998
First “humanized”
MAB-based Pharmaceutical
1976
Genentech Founded
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 18
The Revolution
More than changes in scale
Fundamental changes in scope and
complexity
• GeneÆGenomeÆProteomeÆSystems
Dependent on computation
• Dramatic increases:
- Performance
- Storage capabilities
- Communication capabilities
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 19
Evolution of computing at DNA
In the beginning was the……
Apple II
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 20
Computing “Revolution”
(a parochial view)
DEC PDP/11
1970
ARPANet
1969
Alpha
1992
Internet
1983
Ethernet
1973
Altair
1975
Apple II
1977
Beowulf
1994
RISC
1986
DEC VAX 11/780
1977
IBM PC
1981
Apple IIe
1983
genie
VAX 11/780
1980
Genentech’s First Computer
Apple II
1978
genie
VAX 8650
1987
56 KB Leased
Line to UCSF
(TCP/IP)
1986
ruby
Alpha 8400
1995
tinman
DECSystem 5900
1991
geneland
TruCluster
2000
BARRNet Affiliate
1990
1 MIPS
90,000 MIPS
MIPS = Meaningless Indicator of Processor Speed
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 21
Computing “Revolution”
Changes in scale
• System performance
- 1 MIPS Æ 90,000 MIPS (clustered)
• Memory capacity/system
- 4MB Æ 88GB (~4 orders of magnitude)
• Global connectivity
- 9.6 Kbps Æ 42 Mbps
Changes in complexity
• Multiprocessing (threaded code)
• Clustering (distributed code)
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 22
Strategies for IT Change
Change is required to:
• support changing research needs
• take advantage of changing technology
Strategies
• Revolutionary
- Rip and replace
- Takes maximum advantage of changing technology
• Evolutionary
- Incremental changes
- Minimizes impact to users
- Might result in slower adoption of technology
• Genentech has taken an evolutionary approach
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 23
Setting - Research
Goal
• Basic research
• Human pharmaceuticals
Organization (roughly)
• Discovery
- e.g. Molecular Oncology, Immunology
• Technology
- e.g. Bioinformatics, Protein Engineering, Bioorganic Chemistry
Academic culture
• Open, fast-paced environment
• Need to provide tools as much as solutions
Computational needs are high
• Bioinformatics
• Molecular modeling
• Computational Chemistry
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 24
Bioinformatics Computing Evolution
Disk Subsystem
HSG80
HSG80
HSG80
HSG80
FC Switch
ES40
GS160
8400
12-processors
10-processors
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ES40
ruby
bioinfoweb
MC II Hub
Scooter Morris, CIT
([email protected])
ES40
geneland
Genentech IT Infrastructure
June 12, 2002 page 25
Bioinformatics Cluster
Disk Subsystem
HSG80
HSG80
HSG80
HSG80
FC Switch
trp
cytosine
guanine thymine
cys
leu
ala
met
ruby
bioinfoweb
MC II Hub
geneland
adenine
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 26
Protein Engineering / Bioorganic
Linux Cluster
SGI Origin 2000
(16 processors)
SGI Origin 3000
(24 processors)
SGI Origin 2000
(12 processors)
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 27
Evolution of computing at DNA
BARRNet
UCSF
1980: VAX 11/780
8600
digitaldigital
VAXVAX
11/785
digital VAX 8600
• BSD Unix
• UUCP Dialup Connection to UCSF
1984: VAX 11/785
• BSD Unix
• “Corporate” network
1986: Connection upgrade
• 56Kb Leased line (TCP/IP)
1987: VAX 8600
• BSD Unix
• gene.com registered
• BARRNet affiliate (1990)
digital VAX 8600
SGI 4D/220S
1989: VAX 8650
• BSD Unix
• Total of 3 shared systems
1989: SGI Server
• IRIX
• NFS Services
• Molecular modeling
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 28
Evolution of computing
GS
ph
Al
X
VA
86
DE
50
s te
Sy
C
DE
m
5
ys
CS
t em
70
aS
e
er
rv
84
16
E
0/
-6 4
IA
V7
00
00
0
90
M
IB
SGI 4D/220
ux
Lin
Fa
rm
SGI Origin 3000
X-Ray Crystallography
X
VA
11
/ 78
0
1980
1990
VAXocene
Scooter Morris, CIT
([email protected])
MIPSocene
2000
Alphacene
IPFocene?
Genentech IT Infrastructure
June 12, 2002 page 29
Evolution of computing –
Punctuated Equilibrium?
We have been in a period of relative stability
Episodes of change are “normal”
•
•
•
•
Hardware changes
OS changes
Technological changes (Java, J2EE, etc.)
Vendor changes
Our job:
• Making the change less intrusive
- Attractiveness of Java
- Vendor stability
- Planning
• Avoid revolution
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 30
Futures
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 31
Linux Pilot
Purchased a 24-node Linux cluster
• 2 800MHz PIIIs in each node
• Myrinet
• Computational Chemistry Applications
- Amber, Gaussian
Results
• Memory bandwidth was a problem
• PIIIs already out-of-date
• Myrinet capabilities not heavily used
Hypothesis
• Could build a cluster with different types of nodes
• Submit jobs to appropriate node depending on
computational needs
• May not need expensive cluster interconnect
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 32
Heterogeneous Compute Farm
Disk Subsystem
HSG80
HSG80
MC II Hub
HSG80
HSG80
FC Switch
MC II Hub
ES45
Gigabit Ethernet
P4 Linux
IPF Linux/HP-UX
Scooter Morris, CIT
([email protected])
Apple Xserve MacOS X
Alpha Tru64
Genentech IT Infrastructure
June 12, 2002 page 33
Research Computing Environment – Future?
Disk Subsystem
HSG80
HSG80
HSG80
HSG80
FC Switch
trp
cytosine
thymine guanine
leu
cys
ala
met
MC II Hub
Gigabit Ethernet
SGI
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 34
Questions?
Good, I’ve got some questions:
• How many are Tru64 customers?
• How many are concerned about Itanium transition?
- What are your concerns?
• How many are concerned about HP/UX transition?
- What are your concerns?
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 35
Thank you!
Acknowledgements:
Colin Watanabe
Carol Morita
Roy Takai
Pam Fitch
Scooter Morris, CIT
([email protected])
Genentech IT Infrastructure
June 12, 2002 page 36