Learning from Disaster

ChrisWitt — Wed, 23 Mar 2011 14:40:18 +0000

What Can the IT Community Learn from the Disaster in Japan?

We all hope that we never have to experience the devastation that is being felt in Japan. They are trying to cope with numerous challenges relating to basic human needs like food, water, and shelter let alone basic infrastructure like power and communications. Would you be prepared if your data center was in the path of
such destruction?

As a data center manager, you are responsible to account for all scenarios that could impact data center availability. Let’s look at the challenges you would face if your data center was located in or near the areas of devastation. The areas of concern are power, people, structure, and communications.

Power
Odds are that you will not have power. This is a basic concern that every data center manager accounts for. The problem with this situation is the power will probably be out for an extended period of time. Normally this would not be too much of an issue because you have negotiated those diesel fuel delivery contracts. The only
problem is that there are no usable roads between you and the fuel source. Even if you are using natural gas, there is no guarantee the pipeline would survive the earthquake.

People
What is the status of your staff and can they get to the facility? Even if they can, would they? In a disaster, your staff is going to be more concerned with their own safety and the well being of their family. Your data center will be pretty far down the list.

Structure
An obvious question is: What is the status of the structure? The building may be damaged to the point of being unusable or dangerous.

Communications
If your data center has survived, you have personnel and power, you may still have issues if you don’t have your communication lines. Most organization will employ diverse, redundant connectivity. However, during a regional disaster, there is a very good probably that all of your connections will be impacted.

Key to Survival: Reundancy
So what does this mean? Because of the multiple catastrophic events, there is not a single data center that would be able to maintain operations. Your only solution is to fail over to a redundant data center. Many (but not all) organizations are using redundant data centers to account for catastrophic failures of their
primary facilities. However, many are keeping the data centers within close proximity to limit the amount of communication latency so they can maintain synchronous storage replication. This is around 30 miles or so. Again, using Japan as an example, the destruction radii would be large enough that there would be a good
chance of losing both data centers.

The bottom line is understand your risks and the probabilities. Make sure management is in agreement. Don’t get caught surprised; be prepared to handle the worse case scenarios. Remember, if you think “it can’t happen here”, so did every Japanese data center manager right before the earthquake hit.

WHAT TO PROTECT AGAINST?

ChrisWitt — Mon, 09 Aug 2010 16:56:19 +0000

Avoidance, Recovery, Continuity

Businesses can be interrupted by events as simple as a network outage, or as devastating as a tornado, fire, or even an act of terrorism. The most appropriate step you can take to reduce the impact of a disaster is to first realize that it could happen to your business.

Let’s make sure we’re all on the same page, definition-wise.

Disaster Recovery (DR) describes the strategy an organization employs to deal with potential technology disasters so that the effects will be minimized and the organization will be able to either maintain or quickly resume its mission-critical functions. It follows then that Disaster Recovery Planning (DRP) needs to focus on the data, hardware and software critical for a business to restart operations that have been shut down by a disaster.

Disaster Avoidance, as the name implies, is the process of preventing or significantly reducing the probability that a disaster caused by humans, machines, or forces of nature will occur; or if such an event does occur that the effects upon the organization’s technology systems are minimized to the greatest extent possible.

Business Continuity (BC) procedures kick in as soon as a disaster is triggered. These procedures are a progression of preordained tasks, manual or autonomic, aimed at enabling an organization to continue serving its customers during and after a disaster. It precedes, and ideally minimizes or precludes, the recovery process.

Business Continuity Planning (BCP), explained below, is an excellent starting point for a business to focus because it yields valuable input that can be used to develop cost-effective Disaster Avoidance policies. The BCP process requires the business to identify its mission-critical operations and indispensable processes and data that are essential to keeping the business functioning as a disaster is occurring. Only by knowing the value of the processes and data you are protecting can you have a basis for effectively allocating dollars towards protecting them with Disaster Avoidance methodologies.

After the BCP has prioritized the key business processes, the next step is to identify the specific and significant threats that could disrupt normal operations. And, finally, devise mitigation strategies to ensure effective and efficient organizational response to the challenges these specific threats create during and after a crisis. While there are certainly overlapping features and objectives of these three strategies, it’s important to understand and benefit from the distinctions, because they are certainly not mutually exclusive.

Disaster Avoidance policies and procedures will minimize your exposure to certain disasters. Business Continuity Planning will maximize your ability to keep mission-critical processes working as a disaster unfolds or to resume as soon as possible afterwards. And, the main objective of a Disaster Recovery Plan is to bring operations back as quickly and seamlessly as possible after they have been interrupted by an event.

Disaster Recovery – WAKE TSI

Learning from Disaster

What Can the IT Community Learn from the Disaster in Japan?

WHAT TO PROTECT AGAINST?

Avoidance, Recovery, Continuity