An audit is not a disaster: testing disaster recovery for regulation AND the real world
I have worked in Backup and Disaster Recovery (DR) for longer than I care to admit. Throughout my career I've recognised that it's easy to view DR providers as the insurance sales people of the IT ecosystem. IT systems are not built based on a need for DR. IT systems are built to run business applications while the protection of those systems plays a supporting role.
When it comes to choosing DR protection for an organisation we find that decisions are typically based on considerations such as cost, how critical a particular application or data set might be, and industry regulations.
Many well regulated industries have DR compliance requirements imposed on them which, if not met, can have serious implications on a business; many can face heavy fines or even being deregulated.
Finding a DR solution that enables your organisation to be compliant to your industry regulator is usually the highest priority, but the danger here is that plans are built to satisfy audits and not to meet the true demands of the business.
For many IT departments, the process of a DR audit on servers and applications is stressful and they are forced to focus on building test environments that are good enough to satisfy an auditor who generally is not an IT expert. Each year when the audit is complete, IT teams can breathe a huge sigh of relief and their executive management is happy in the belief that everything will be OK in the event of disaster.
If you look an IT manager in the eye and ask if they can recover everything in a time that the business needs or expects, they may not be so confident. Many IT managers have admitted to me that even though they pass the audit they tend to still have a "cross my fingers" approach to DR, in the knowledge that if a real disaster strikes, the recovery process may be more fraught than the business will accept.
Why is passing audit so different to a real world disaster?
Traditionally, recovery is complex and resource intensive. The process of setting up the logical and physical recovery environment just for testing can be time and resource intensive. This begs the question: if it takes all this work to prepare for a planned recovery, how do we cope for the unplanned disaster?
In addition, most disaster recovery test scenarios require that protection stops whilst the test is taking place, and the majority of backup and DR products don't have comprehensive test capability built in; often the only way to test is to do an actual full recovery. Overcoming this potential exposure to data loss is a genuine headache.
As a result of this risk and complexity, putting recovery to the test on a regular basis in real world scenarios rarely happens because it is just too disruptive to do so.
Testing for real world disaster and recovery
We all know that DR testing should not be about audit, it should be about actually ensuring that when disaster strikes the business is totally prepared. Testing should be an integral and regular part of the DR strategy to ensure that staff are confident and prepared when a crisis hits, to be sure that applications recover in a consistent state, and have a very clear idea of how much downtime the company will be exposed to.
Beyond the audit, an IT department is only ready when you can look them in the eye to ask that same question and they will answer you with complete confidence on their recovery capability, without blinking.
To do that, organisations need to work with a DR provider that offers non disruptive automated recovery testing which doesn't require additional infrastructure built into the product. Being able to test and prove recovery right up to running applications means that audit is something IT doesn't have to spend more than a few minutes to prepare for and testing becomes a standard part of regular operations.
Remember that passing an audit does not necessarily mean you are prepared for a real world disaster. Comprehensive, automated recovery testing doesn't have to be disruptive, and it will ensure the IT team can be confident and the executive management are not going to be let down when disaster hits.
Article by Andrew Martin, the Zerto director for Asia Pacific and Japan. Zerto providesdisaster recovery and business continuity software for virtualised data centers and cloud environments.
