Publishing Web Services – a provider’s
point of view
Rodrigo Lopez
External Services - EMBL-EBI
EMBRACE Workshop DTU June 2010
EBI is an Outstation of the European Molecular Biology Laboratory.
Outline
• 10 TIPS
• BioCatalogue
• Anatomy of a service in the BioCatalogue
• Acknowledgements
2
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 1
• Plan
•
•
•
•
3
29/07/2017
Identify and document the science
Identify the users
Search and document equivalent or similar services
How much will it cost?
• Breakdown IT and HR resources
EMBRACE Workshop DTU June 2-4 2010
Tip 2
• Characterise IT capabilities
• Select sustainable infrastructure
• Never go ‘alone’
• Get your IT team involved
• Select up-to-date technology
• Your IT team should provide SANE/LATEST
• Consider Virtualisation
• Some fun: If the service is discreet, consider deploying in the
Cloud (EC2)?
• What about Google, Azure, Soho, Yahoo, Salesforce,
Rackspace and the other clouds?
4
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Internet
Web node
5
29/07/2017
Web node
Zeus LB1
Zeus LB2
Web node
Web node
EMBRACE Workshop DTU June 2-4 2010
Web node
Web node
Internet
PG
OY
HX
6
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 3
• Identify HR capabilities
• Your programmer’s capability
• Qualifications
• Discipline (adopt agile project management?)
• Academic vs professional aptitudes
• Is (s)he a good scientist?
• Don’t turn a good scientist into a web master!
• (or into a nerd)
• (or into a geek)
• Programming languages
• Spring, Apache Axis2, CFX...
• PERL, PYTHON, PHP, Java, C/C++/C#...
• Make choices according to the footprint of your IT/HR.
7
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 4
• Define the service architecture constraints
• Modular or integrated architecture (monolithic or diversified)
• Small components can play many roles
• Many services have dependencies!
• Integrated
• Usually few people understand the complexity of the
application.
• SOA definitions that apply to your service
• Specifications and Organisations
• Read/Adhere to the EMBRACE recommendations
• BioXSD (don’t invent unless you have to!)
• Get your XML background, specifications and vocabularies
(EDAM), right.
8
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 5
• Define application server tier
• Security & LoS
• Make sure you are safe
• Don’t advertise security, unless you have to.
• Do you have a LoS (e.g. 98%, 24/7, etc.)?
• State maintenance
• Is your application stateless or stateful?
• Data access
• Is your back-end up to scratch
• Optimise connection pools
• Persistence
• Do you need it?
• Monitor the service
• ...and profile it regularly (at each data update?).
9
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 6
• Service Redundancy
• 2 nodes (physical/virtual)
• Round Robin, Simple Fail Over, Dynamic Round Robin,
Dynamic Fail Over, Load Balanced?
• Service failover
• Local
• Remote (federated?)
• Recovery policies?
• Maintenance policies?
10
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 7
• Adhere to existing data delivery standards
• Formats, styles...
• Adhere to existing data consumption standards
• Results integration (BioXSD)
• Ensure integration into existing workflows or pipe-lines
11
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 8
• Ensure your services are easy to understand and
consume
• User support
• 1st-3rd level support
• Clarify Responsibilities
• Documentation
• Programmer
• End-user
• Build and publish example clients
• Test in as many programming languages as possible!
• Test with as many WS Enactors as possible!
• Test, test, test...
12
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 9
• Justification
•
•
•
•
Keep logs
Keep user communications
Document costs
Usage trend analysis
• Know your three A’s
• Accessibility
• Accountability
• Awareness
13
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Tip 10
•Publish your service!
14
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Conclusions 1/2
• Identify your services life-cycle
15
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Conclusions 2/2
• Implement (if you have not already done so) Business
Intelligence into your services operation
• Make a INVENTORY of all your services’ components
• Don’t forget to include all variables!
• Keep it up-to-date (or get someone to do this for you)
• So, you have an INVENTORY (and you huff and you
puff...)
• Use it to market your services...
16
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
BioCatalogue
•
•
•
•
17
A means to pool metadata about services in the wild
A means to discover and reuse those services
A means to curate services
A platform for service monitoring and analytics
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
BioCatalogue Annotation ‘guidelines’
•
•
•
•
•
Where is the documentation?
What is it about?
Can anyone use it?
Who to contact?
What does the operation do or what are the
resources?
• What is the input?
• What is the output?
18
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
BioCatalogue
• Anatomy of a service in the
BioCatalogue
• http://www.biocatalogue.org/
19
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
20
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Looking for a rigorous sequence search
service...
21
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
22
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
23
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Ownership/Responsability
24
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Methods
25
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Monitoring
26
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
27
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
28
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
BioCatalogue - Usage: Wider Context
• Partnerships:
• Collaborations:
29
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
How to contact us:
• Please use:
• http://www.biocatalogue.org/contact
• http://www.ebi.ac.uk/support/
30
29/07/2017
EMBRACE Workshop DTU June 2-4 2010
Acknowledgements
•
•
•
•
•
•
31
EMBL-EBI/ES
EMBRACE
BBSRC
EMBL
NIH
EPO
29/07/2017
EMBRACE Workshop DTU June 2-4 2010