SESSION 201
Wednesday, November 2, 11:30 AM - 12:30 PM
Track: DevOps and Agile
10 Things I Hate About DevOps
Dave Cliffe
Group Product Manager,PagerDuty
[email protected]
Session Description
If there’s anyone who understands the complex relationship many of us have with the term DevOps,it’s
Shakespeare. Through the lens of the five wits referenced in his Sonnet 141, this session will unpack the
different ways we’ve seen DevOps successfully and unsuccessfully applied to a variety of organizations,
with a particular focus on operability and incident management. What impact does shared operational
responsibility have on incident response? How is uptime affected by having "more cooks in the kitchen"?
Under what circumstances does this kind of operational model even make sense? Together, we’ll answer
the most fundamental question: O DevOps, DevOps! Wherefore art thou, DevOps?
(Experience Level: Intermediate)
Speaker Background
Dave Cliffe is a software engineer who adopted a more peaceful role on the product team at PagerDuty,
a company whose sole goal is to make the lives of Dev and Ops engineers everywhere calmer and saner.
Before PagerDuty, Dave worked in cloud computing at Microsoft (Windows Azure) and at Amazon.com.
He often wonders which is scarier: being on-call during an outage or being a parent. The debate
continues.
10 Things I HATE
AboutI DevOps
10 Things
HATE About
DevOps
Dave Cliffe – PagerDuty
Dave Cliffe
@CliffeHangers
@CliffeHangers
10/21/2016
@CliffeHangers
Dearest DevOps:
“In faith, I do not love thee with mine eyes,
For they in thee a thousand errors note,
But ‘tis my heart that loves what they
despise,
Who in despite of view is pleased to dote.
…
But my five wits nor my five senses can
Dissuade one foolish heart from serving
10/21/2016
thee.”
@CliffeHangers
- Shakespeare, Sonnet 141
Don’t
be fooled:
Don’t
be fooled:
I am
not anot
poet a poet
I am
• Software Engineer @
Amazon.com
• Program Manager @
Microsoft .NET, Azure
• Group Product Mgr @
PagerDuty
• Also a 3x Co-organizer @
devopsdays Toronto
10/21/2016
@CliffeHangers
Tools
Process
People
Deployment Tools
Monitoring Tools
MICROSERVICES
APPS & SERVICES
CONTAINERS
Collaboration/Resolution
Developer
On-Call
Scheduling
EVENTS
APP
CLOUD
SYSTEM
NETWORK
LOG
ALERT 1
ALERT 2
ALERT 3
NOC
Automatic
Escalations
DATABASE
SERVERS
WEB
MOBILE APP
Ticketing Tools
Correlate, Cluster
and Manage
Helpdesk
IT Ops
System and User
Efficiency
Your Fastest Path to Incident Resolution
8000+ customers globally, 50 of the Fortune 100, proud supporters of DevOps
10/21/2016
@CliffeHangers
Agenda
•10 Things I HATE
About DevOps
• From the Business
• For Developers
• For Ops
• The Five Wits of
Incident Response w/
DevOps
• Common Sense
• Imagination
• Fantasy
• Estimation
• Memory
10/21/2016
@CliffeHangers
1. You say Potato, I say
DevOps
(Are we even talking the same language?)
10/21/2016
@CliffeHangers
What do we mean by DevOps?
https://blog.chef.io/2010/07/16/what-devops-means-to-me/
C ulture
A utomation
M easurement
S haring
Damon Edwards and John Willis
devopsdays.org
10/21/2016
@CliffeHangers
$$$
2. DevOps is a tech fad,
irrelevant to the business
10/21/2016
@CliffeHangers
“By 2016, DevOps will evolve from a
niche to a mainstream strategy
employed by 25% of global 2000
organizations”
- Gartner
“… high-performing
IT organizations …”
https://puppet.com/resources/white-paper/2016-state-of-devops-report
10/21/2016
@CliffeHangers
$$$
3. DevOps means more
changes and change is
painful
10/21/2016
@CliffeHangers
$$$
4. DevOps seems to break
things more frequently
10/21/2016
@CliffeHangers
Move Fast:
Break, Fail, Learn
http://www.ca.com/us/rewrite/articles
/devops/face-to-face-devops-to-protectand-serve-.html
10/21/2016
@CliffeHangers
Dev
5. I shouldn’t have to care
about the customer
10/21/2016
@CliffeHangers
10/21/2016
@CliffeHangers
Dev
6. I hate being accountable
for what I build
10/21/2016
@CliffeHangers
10/21/2016
@CliffeHangers
Dev
7. It’s not my fault
(a.k.a. it works on my machine)
(a.k.a. that’s not a bug, it was spec’d that way)
10/21/2016
@CliffeHangers
Ops
8. DevOps isn’t ___
compliant
10/21/2016
@CliffeHangers
10/21/2016
@CliffeHangers
Ops
9. But I’m the expert
10/21/2016
@CliffeHangers
http://futurice.com/blog/from-t-to-pi-design-skill-expectations-in-change
10/21/2016
@CliffeHangers
Ops
10. DevOps is automating
me out of a job
10/21/2016
@CliffeHangers
https://twitter.com/wattersjames/status/770375207219449858
10/21/2016
@CliffeHangers
!
Call to Action:
Stop drinking
the haterade
10/21/2016
@CliffeHangers
The Five Wits: Incident Response w/ DevOps
1. Common Sense
2. Imagination
Realize
• Realize what the mind is being presented
3. Estimation
Retain
• Decide on matters of time, space, etc.
4. Fantasy
Decide
• Act in accordance with judgment
5. Memory
• Retain the recollection of what is imparted
Act
10/21/2016
@CliffeHangers
Common Sense: DevOps
https://blog.chef.io/2010/07/16/what-devops-means-to-me/
C ulture
A utomation
M easurement
S haring
Damon Edwards and John Willis
devopsdays.org
10/21/2016
@CliffeHangers
Culture
“Giving developers operational
responsibilities
has greatly enhanced the quality of the
services, both from a customer and a
technology point of view.
… You build it, you run it.”
- Dr. Werner Vogels, CTO Amazon
10/21/2016
@CliffeHangers
Realize
Realize a.k.a. Monitor
Retain
• Automation 
• Build the platform for effective monitoring
• Measurement 
• Know BEFORE your customer
• Sharing 
• Aggregate across tools, alert the right people
10/21/2016
@CliffeHangers
Decide
Act
Realize
Decide a.k.a. Triage
Retain
Decide
• Automation 
Act
• Find the signal in the noise
• Measurement 
• Business Metrics, not just Technical Metrics
• Sharing 
• “Are you dealing with something? We’re seeing a P1”
10/21/2016
@CliffeHangers
Realize
Act a.k.a. Resolve
Retain
• Automation 
• ChatOps – more effective resolution
• Measurement 
• Test your hypotheses – differential diagnosis
• Sharing 
• Proactive communication with stakeholders
10/21/2016
@CliffeHangers
Decide
Act
Realize
Retain a.k.a. Learn
Retain
• Automation 
Act
• Build the timeline of events
• Measurement 
• How effective was our response?
• Sharing 
• Blameless Post-Mortem
10/21/2016
@CliffeHangers
Recap: Incident Response w/ DevOps
Monitor
S haring
M easurement
Learn
Triage
A utomation
C ulture
Resolve
Lay the SMAC down on Incidents!
10/21/2016
@CliffeHangers
Decide
Thank you for attending this session.
Please don’t forget to complete an evaluation for this session!
Evaluation forms can be completed electronically on the
FUSION 16 Conference App.
10/21/2016
@CliffeHangers
Thank You!
Dave Cliffe - @CliffeHangers
www.pagerduty.com
10/21/2016
@CliffeHangers
Your Fastest Path to Incident Resolution
10/21/2016
@CliffeHangers
8,000+ Customers
150,000+ Users
80 Countries
50 of the Fortune 100
175+ Native Integrations
10/21/2016
@CliffeHangers
10/21/2016
@CliffeHangers
PagerDuty
The Fastest Path to Incident Resolution
Bringing together tools, process, and people
Tools
Process
People
Leverage existing tools to
deliver modern ops for a
unified view across all apps and
services
To consolidate and cluster alerts
to surface the real problems
and notify the right people
One platform for NOC,
Helpdesk, & Responders that
brings teams together
10/21/2016
@CliffeHangers
“With PagerDuty, we spend less time
worrying about on-call, and more time
creating great products to impact lives.”
“If we didn't get PagerDuty, we
wouldn't have a business.”
- Martin Rhoads, Site Reliability Engineer
- Bryant Eastham, Chief Architect
“Centralized alerts allow us to streamline
communication, decreasing MTTR and
increasing incident efficiency.”
- Shaun Boucher, Manager, Network Operations
10/21/2016
@CliffeHangers
“I’m calling it — PagerDuty is a musthave WebOps service.”
- Jesse Robbins, Co-founder