Business Continuity — Disaster Recovery
A business continuity plan helps companies keep critical
business functions running during an emergency. It includes
communicating with employees and recovering or relocating
facilities. Disaster recovery is the process of recovering or
rebuilding IT systems and data after an emergency.
10 • Business Continuity — Disaster Recovery
Structuring for
Sustainability
Last year, hurricanes Katrina
and Wilma slammed P&G’s
Folgers Coffee facilities and
Office Depot’s headquarters.
Here’s how solid planning
helped both firms bounce back.
Each year, Procter & Gamble updates its business continuity
(BC) and disaster recovery (DR) planning. It has redundant
telecommunications infrastructures. It regularly backs up critical
data and stores it in other regions. It has employee call trees and
an emergency 800-number.
Damon Frost
Coffee Category
Global Business Services Leader
Procter & Gamble
But all the planning in the world couldn’t have prepared
P&G’s leaders for the reality they faced when Hurricane
Katrina struck New Orleans in late August last year, where
four of its Folger’s Coffee facilities are located. Residents
poured out of the city, homes were submerged in water and
violent crime turned New Orleans into a virtual war zone.
“People weren’t allowed back into the neighborhoods, so we
had trouble getting in to assess the sites,” says P&G external
relations manager Lars Atorf. “We didn’t have power. We didn’t
have phones. We had to start like we were in the 18th century.”
Disasters, by their very nature, can leave even the most
prepared businesses in a state of disarray. But those that are
armed with a strong BC/DR plan fare far better than those that
start from scratch when disaster hits. In New Orleans, businesses
closed their doors for months on end, or permanently. P&G’s
facilities were back up and running in late September — the
first manufacturing facility in town to do so — and was fully
operational by November. About 90 percent of its employees
are back at work in New Orleans, far higher than the overall
rate of return among New Orleans’ residents.
“You come to a point where even a good business continuity
plan reaches its limits,” says Damon Frost, who started as
P&G’s coffee category global business services leader in
February. “We really had a very holistic analysis of what we
need to do to keep the business running.”
That holistic vision is the key to any BC/DR plan. After
Sept. 11, businesses around the country devised plans to
respond to a terrorist attack. Hurricane Katrina prompted
endless discussions about natural disaster planning. But the
key to surviving a disaster is to focus on the business and its
systems rather than the event itself. Instead of reacting to the
day’s headlines with disaster-specific plans, businesses need
proactive, holistic BC/DR plans that can cover them whether 4
Business Continuity — Disaster Recovery • 11
For business continuity starters,
focus on your key assets:
people and data. “Those are
the two major assets that you
cannot survive without.”
Tom Serio
Director of Global Business Continuity Management
Office Depot
they’re hit by a tornado, a virus attack, an Anthrax mailing or
the Avian flu.
“Disaster time is not the time to start planning,” says Tom
Serio, director of global business continuity management at
Office Depot. “It’s like the ‘Three Little Pigs’ children’s story.
When the sun’s shining, you should start planning.”
A good starting point is to consider the big buckets:
facilities, people, communications, data and systems. Within
each bucket, the plan should identify the bare minimum that’s
needed to keep the business running and determine ways to
ensure those basics are available. That can make or break a
business in a disaster.
“If you’ve never had a disaster, it’s hard to plan for one,”
P&G’s Frost says. “But it is important. It’s like insurance.”
Back to Basics
After Katrina, P&G executives flew in by helicopter to assess
the damage to the sites. They wanted to get back to business as
quickly as possible, but there was a problem: They didn’t have
workers to get the business back on its feet.
“We lost a significant portion of the population of that city,”
Frost says. “We learned the problem wasn’t so much about
systems as about people.”
P&G employees from other states volunteered to help out,
so the company helped them arrange for travel to and from
New Orleans. They created shifts where employees spent seven
days working and seven days off so they could return to their
families every other week.
To accommodate workers in New Orleans, P&G created its
own trailer complex, Gentilly Village, with everything workers
would need for daily life, such as health care, food and laundry
facilities. P&G flew in generators to provide its own electricity
and dug a well to provide water to wash coffee beans.
Gentilly Village also had more than 150 Federal Emergency
Management Agency trailers onsite. And P&G gave employees
interest-free loans to assist in the time of crisis.
“Things you take for granted, that you wouldn’t even think
about, proved to be the biggest immediate issues,” P&G’s
Atorf says.
12 • Business Continuity — Disaster Recovery
The other major issue in Katrina’s aftermath was
communications. Phones and e-mail went down, so P&G had
no way to communicate with customers, partners or emergency
officials. With the help of a supplier, it flew in a satellite phone
system. Getting that in place more rapidly is one of the big
adjustments in P&G’s post-Katrina disaster recovery plan,
Frost says.
“We have to rely on strong working relationships with our
external business partners,” Frost says. “Our ability to respond
to a disaster like Katrina is the result of a lot of hard-working
people. We had all hands on deck to recover, so we didn’t have
to do the heavy lifting alone.”
Despite the challenges, a lot of pieces fell smoothly into
place thanks to P&G’s BC/DR plans. The company, which
has dedicated BC/DR specialists, uses a matrix approach to
envision various scenarios. It has an IT-specific plan, sitespecific plans, function-based plans and so on.
Whenever P&G purchases new equipment, it considers a
backup plan and how to build maximum redundancy, Frost
says. During Katrina, for instance, the company was able to
reroute New Orleans calls to its headquarters in Cincinnati.
The company keeps backups of all its data and systems on
tapes that are shipped to other sites. The most critical systems are
shipped to a site in another city in case of a regional disaster.
The backup sites hold annual drills to rebuild their data
center from the ground up: booting up servers, restoring tape
and testing systems to ensure they’re working. The drills follow
a schedule dictating which systems need to be up by when,
Frost explains.
“It’s a bit of a learning process, but you have to step back
and think of all the scenarios that could occur,” Frost says.
“What happens if a machine goes down? What if my network
goes down?”
Think Small
For large corporations, BCP/DR plans can become extremely
complex, but medium-sized businesses can go a long way
toward preparing themselves for disaster with just a few
simple steps.
“It’s not as hard to be prepared as [some] companies think
it is,” says Office Depot’s Serio. “It’s never too late. You can’t
afford not to be prepared.”
For starters, focus on your key assets: people and data.
“Those are the two major assets that you cannot survive
without,” Serio says.
Backing up data offsite can be as simple as copying your
critical data onto CDs, external hard drives or flash drives. Just
remember to do regular backups and bring the storage device
offsite when you leave, Serio warns. He recommends keeping
disaster recovery plans on a flash drive so you always have it
right at your fingertips.
Another option for businesses that can’t afford offsite backup
facilities is to use electronic vaulting firms, which store data
on their servers at a cost, to back up data over the Internet.
Businesses with more than one location can even use their own
offices as backup sites, Serio advises.
At the systems level, sticking to off-the-shelf systems instead
of complex custom applications means that even if your
infrastructure is wiped out, you can easily replace the hardware
and software, Serio explains.
Before disaster is even on the horizon, it’s a good idea to
identify your most critical functions. Which employees, systems
and services are absolutely essential? When you’re in disaster
mode, you may not be able to run in full business mode.
Many businesses tell employees to work from home during a
disaster, but those workers might not know how to connect to
the company server remotely. Serio suggests including remote
worker training into your disaster recovery plans.
Planning is critical. “Without a plan,” Serio says, “you don’t
know what to recover and you don’t know who to call.”
Business Continuity/
Disaster Recovery Tips
Being ready for a business-halting
incident is essential for any size firm.
Incorporating these ideas into your
planning can help you recover
from disaster.
• Establish an emergency call-in number and call trees
to keep employees informed during a crisis.
• Get as much employee contact information as
possible: spouses’ cell phone numbers, home e-mail
addresses, phone numbers of close, out-of-state
relatives. Update it twice a year.
• Consolidate business-critical systems and keep them
in a location with a backup power generator.
• Mirror mission-critical systems and backup
data, then keep them (as well as a map of the
infrastructure) at an offsite location, preferably in
another region.
• Arrange for in-town and out-of-town areas to
set up as “command centers” in case the building
is inaccessible.
• Teach employees about disaster-recovery planning
not only for the business but for their families.
Ready and Waiting
When Hurricane Wilma swept through Florida last
October, knocking out power at Office Depot’s Delray Beach
headquarters, Serio, operating from a makeshift command
center at an Orlando hotel, already knew which of the
approximately 2200 corporate associates could work from
home, who needed to work from the company’s command
center and who could concentrate on caring for their families.
Office Depot’s Web site is backed up on a high-availability
offsite third-party system. Critical data is regularly backed up
to tapes and discs, then sent offsite and secured in a vault — so
it was available when disaster struck, he adds.
Such precautions are a given for businesses located in Florida
or Louisiana, where hurricanes are a regular occurrence, or
places like New York or Washington, D.C., which are on
constant alert for terrorism. Precautions should also be taken
by businesses without clear threats, Serio says. A tornado could
strike Boston, a blizzard could cripple Atlanta and a fire can
break out anywhere, at any time. “Never say never. Disaster
can happen to anybody, to any business.”
• Designate notebook computers to be used in case
you need to move operations.
• Develop a strong relationship with your business
partners. When disaster strikes, they can prove to
be invaluable allies in terms of replacing equipment,
offering advice and assistance and negotiating lastminute deals with manufacturers.
• Include in your disaster-recovery plan a list of
systems you would like to upgrade to at some point.
If those systems are destroyed and you’ve done the
research already, it may be more economical
to upgrade.
• If systems or Internet access is lost during an
emergency, update virus signatures as soon as
connectivity is restored.
• Update and test disaster-recovery plans annually.
CDW offers technology service support from top manufacturers
and service providers across all product categories.
Business Continuity — Disaster Recovery • 13