WHITE PA PER
5 Steps to Eliminating
Database Change Risk
5 Steps to Eliminating
Database Change Risk
In new research conducted by Simon Management Group (SMG) among Fortune 1000
companies, we’ve learned that two-thirds of change requests require a schema update.1
Updating the schema carries a 78% probability that your application will “break”
somewhere along the release path to production.
If you are currently managing your schema changes manually, you are setting
yourself up for unnecessary risk in your application lifecycle.
Enterprise Risk Management has gained a lot of attention
recently, especially since the financial crisis of 2008. Groups
like the Committee of Sponsoring Organizations (COSO), a joint
initiative between accounting organizations, and the International
Organization for Standardization, ISO, have published frameworks
for managing risk in the enterprise. These frameworks bear
many similarities to the U.S. Army’s process for managing risk—
Composite Risk Management.2
Intuitively this makes sense—why expend the effort to analyze the
nature of risk as it applies to the enterprise, when one can adapt
a comprehensive methodology that combat leaders have successfully
employed for centuries?
Updating the schema
carries a 78%
probability that your
application will “break”
somewhere along the
release path to
production.
This white paper presents an overview of Composite Risk Management (CRM), and
discusses the process of updating your database schema within the application release
process as a practical example.
1 Simon Management Group (2012). Datical Market Validation Study. Austin, TX.
2 United States Army (2006). FM 5-19, Composite Risk Management. Washington D.C. Headquarters,
Department of the Army.
5 Steps to Eliminating Database Change Risk
2
Overview: Composite Risk Management
There are several guiding principles for applying the framework—how do they apply to your
application lifecycle?
•
Integrate CRM into all phases of missions and operations – Every phase of your
application lifecycle has unique risks associated with it.
•
Make risk decisions at the appropriate level – Manage risk at the appropriate level,
and know when an issue must be elevated.
•
Accept no unnecessary risk – Accept no level of risk unless the potential gain or
benefit outweighs the potential loss.
•
Apply the process cyclically and continuously – The process of managing risk is
never complete.
•
Do not be risk averse – At the heart of risk lies uncertainty, which also carries an equal
chance for opportunity.
The CRM framework consists of five steps:
1.
2.
3.
4.
5.
Identify hazards
Assess hazards to determine risk
Develop controls and make risk decisions
Implement controls
Supervise and evaluate
Identify the Hazards
Some common hazards that organizations typically face when
updating their database include:
Risk occurs when the
state of the database
is unknown. Will
the changes that
“worked” in your dev
environment also
work in QA, staging or
production?
HAZARD: Misalignment of version numbers between application and database code
• Risk occurs when only certain changes or feature requests will be deployed in a
release—someone must manually determine what gets deployed.
5 Steps to Eliminating Database Change Risk
3
HAZARD: Database configuration “mismatches” across
environments
• Risk occurs when the state of the database is unknown—will the
changes that “worked” in your development environment also
work in QA, staging or production?
HAZARD: Too much information in a single database release
• Risk occurs when single changes contain too much
information—have you ever run across an SQL script containing
2,000 lines of code? If a problem occurs, someone will have
to manually inspect those lines to locate and troubleshoot
the issue.
Have you ever run
across a SQL script
containing 2,000 lines
of code? If a problem
occurs, someone will
have to manually
inspect those lines
to locate and troubleshoot the issue.
HAZARD: Troubleshooting the database manually
• Risk occurs when an issue is identified and you can’t easily
eliminate the database as a cause—productivity is decreased
and all too often the project is delayed.
HAZARD: Uncertainty about the impact of database changes
• Risk occurs when there is drift between environments—how do you know that what
you intended to occur in production will actually execute correctly?
Assess Hazards to Determine Risk
Hazards are assessed and risk is assigned in terms of probability and severity
of adverse impact of an event/occurrence.
So, there are two key inputs—the probability of a hazard occurring, and the hazard’s
impact on the task at hand. The intersection of these variables determines the level of risk.
The military employs a tool called a Risk Assessment Matrix for this purpose (see Figure 1).
Risk Assessment Matrix
PROBABILITY
SEVERITY
Frequent
Likely
Occasional
Seldom
Unlikely
Catastrophic
Extremely High
Extremely High
Critical
Extremely High
High
High
High
Moderate
High
Moderate
Low
Marginal
High
Negligible
Moderate
Moderate
Moderate
Low
Low
Low
Low
Low
Low
Figure 1
5 Steps to Eliminating Database Change Risk
4
Probability should be scientific, based on industry benchmarks or metrics from your
organization. Severity is largely subjective and should be defined according to the impact
to your organization. Now, let’s assign risk to each hazard according to the matrix (see
Figure 2).
TASK - Update the Database Schema
HAZARD
PROBABILITY
SEVERITY
RISK LEVEL
Frequent
Negligible
Moderate
Likely
Marginal
Moderate
Too much information
Occasional
Critical
High
Manual troubleshooting
Likely
Critical
High
Uncertainty about impact of changes
Likely
Critical
High
Misalignment of version numbers
DB configuration “mismatches”
Figure 2
When assigning risk to the task (i.e. updating the database schema), the rule of thumb is to
use the highest level identified from individual hazards—updating the schema is assigned a
risk level of “High.”
Develop Controls and Make Risk
Decisions
Controls are developed which will either reduce the risk of a hazardous event occurring, or
eliminate the risk entirely. To be effective, each control must meet the following criteria:
1. Suitability: The control must remove the hazard or mitigate residual risk to an
acceptable level.
2. Feasibility: The organization must have the capability to implement the control
3. Acceptability: The benefit gained must justify the cost in resources and time.act
5 Steps to Eliminating Database
Change Risk
Hazards are reassessed to determine the residual risk level (i.e. level of risk after the control
has been implemented). Risk decisions are based on residual risk only. If residual risk is
higher than acceptable by management, additional controls are developed and the hazards
reassessed.
5 Steps to Eliminating Database Change Risk
5
HAZARD: Misalignment of version numbers between application and DB code
CONTROL: Treat database changes like Tier 1 artifacts
• Check database changes into your source control repository
• Synchronize database changes with application code
HAZARD: Database configuration “mismatches” across environments
CONTROL: Maintain awareness about the state of the database
• Maintain a log that tracks the current state of the database
• Maintain a log that tracks what changes have been made to specific databases
HAZARD: Too much information in a single database release
CONTROL: Deploy database changes as atomic units—cohesive units with your
application code
• Break database changes down into the smallest units possible
HAZARD: Troubleshooting the database manually
CONTROL: Provide proof that the database is not at fault
• Require reporting within your organization to build an audit trail which can be quickly
referenced
HAZARD: Uncertainty about the impact of database changes on production
CONTROL: Maintain awareness concerning drift in your environments
• Employ automation that provides a log of changes made to specific environments
• Strive to ensure environment integrity and consistency
Now, we reassess each hazard and determine the residual risk (see Figure 3). Once
again applying our rule of thumb, updating the database now carries a residual risk of
“Moderate.” If management decides that this is acceptable, then we are complete with our
risk assessment.
TASK - Update the Database Schema
HAZARD
INITIAL RISK
LEVEL
CONTROLS
RESIDUAL
RISK LEVEL
Misalignment of version numbers
Moderate
• Check DB changes into source repository
• Synchronize DB changes with app code
Low
DB configuration “mismatches”
Moderate
• Log the current state of the DB
• Log what changes have been made
Low
Too much information
High
• Break DB changes down into atomic units
Low
Manual troubleshooting
High
• Require reporting to provide an audit trail
Moderate
Uncertainty about impact of changes
High
• Log environmental changes
• Ensure environment integrity/consistency
Moderate
Figure 3
5 Steps to Eliminating Database Change Risk
6
Implement Controls
[Datical DB] empowers
This is why your manager makes the big bucks—he’s the one
who has to enforce the controls. We’re betting that he tells you
that you’re now in charge of your organization’s risk management
process.
your developers to
Supervise and Evaluate
application updates,
Now, all you have to do is design and implement four daily logs and
approach so they don’t
accurately and efficiently
update the schema
simultaneously with their
using an object-oriented
a reporting system. Oh, and you’ll also have to go to everyone every
have to worry
day for at least three months to make sure that they’re using the
SQL syntax.
logs and reporting system correctly. After a period of time, you’ll
also have to check all of the logs against actual data to evaluate
whether the system is working. If it worked, you should immediately buy a Powerball ticket.
If it didn’t work quite the way you planned, it’s time to troubleshoot the process, identify
the discrepancies, and begin anew…
about
Repeating a key point:
“...controls are developed which will either reduce the risk of a hazardous event occurring,
or eliminate the risk entirely.”
Updating schemas is inherently risky. Developing a robust logging and reporting system is
time-consuming and frustrating in addition to being risky. Why not avoid the heartache and
eliminate the risks entirely? That’s where Datical comes in.
Datical DB
Datical DB is a database change management solution, based on Liquibase, specifically
developed to help you eliminate risk and uncertainty when updating your database as
part of the application release process. Unlike the current DBA tools available, Datical DB
employs a holistic approach while seamlessly integrating with your current process and
tools. It standardizes the process of updating your database schema so that you can keep
application and database code in synch throughout the application release process:
•
Maintains an automated running log of all executed database changes and compares
database configurations to provide instant visibility into the current state of your
database.
•
Organizes and tracks your SQL scripts while providing the entire team with visibility into
what’s happening in the database.
5 Steps to Eliminating Database Change Risk
7
•
Empowers your developers to accurately and efficiently update the schema
simultaneously with their application updates, using an object-oriented approach so
they don’t have to worry about SQL syntax.
•
Augments your DBA’s ability to facilitate the process, troubleshoot problems, and
recover from disaster.
•
Enables you to eliminate configuration drift between databases and perform fresh
installations in a matter of seconds.
•
Most importantly, it provides you with proactive environmental intelligence concerning
the impact of changes on the database, before you deploy.
Instead of providing yet another framework that requires you
to spend a lot of money, time and effort to change your existing
systems but still doesn’t provide change integration, Datical DB
simply works with the systems you have. Whether that’s Source
Control, Build Systems, Testing Systems and existing Automation
Frameworks—Datical DB makes it very easy to reduce errors
and application downtime, speed time to market and lower your
cost by:
Datical DB makes it
very easy to reduce
errors and application
downtime, speed
time to market and
lower your cost.
•
Eliminating database change risk and bottlenecks in your
develop-test-deploy process
•
Accelerating releases and automating deployment processes
•
Knowing how changes will impact production and any other environment
•
Improving compliance with auto-documenting change management processes
•
Increasing quality of service with a holistic, repeatable, database-agnostic approach
We invite you to learn more at www.datical.com.
5 Steps to Eliminating Database Change Risk
8
About Datical
Datical solves critical business pains by leveraging proven open source projects, such as
Liquibase. We enhance and extend open source projects while removing complexity. We
make them easy to use. We make them open. We make them flexible, secure and fully
supported.
But first, we complete extensive market validation research so we know we’re solving the
right problems and adding the right features, in an easy to use, non-disruptive way.
We’re not interested in checking off boxes on a feature comparison checklist. We’re focused
on solving our customer’s most complex database change problems, protecting critical data
and reducing risk.
That’s how we deliver serious value. And our customers see quick results.
We believe it’s because we’re fanatically market driven and have taken a different, modelbased approach in architecting our solutions. This makes it possible to fuel automation with
environmental intelligence, visibility and flexibility—extending the very best open source
projects for a future proof enterprise.
www.datical.com | [email protected] | Tel: 949.DATICAL
Copyright ©2017 Datical, Inc. Datical, all products prefaced by the word Datical and the Datical logos are either registered trademarks or trademarks of
Datical, Inc. in the United States and/or other countries. All other products mentioned are either registered trademarks or trademarks of their respective
corporations. 201701
5 Steps to Eliminating Database Change Risk
9