High Availability and Disaster Recovery
High Availability and Disaster Recovery
Reliable network services do not depend on a single device, link, power source, or recovery plan. Learn how redundancy, fault tolerance, resilient facilities, recovery objectives, backups, and regular testing keep services available before, during, and after failure.
High Availability, Redundancy, and Fault Tolerance
High availability (HA) is a design objective that keeps network services accessible with minimal downtime. Hospitals, financial institutions, data centers, and other critical environments use HA because even a short outage may cause operational, financial, or safety consequences.
High availability
Uses resilient design, monitoring, and rapid failover to meet an uptime target. A brief interruption may occur while service moves to a healthy component.
Redundancy
Duplicates components, links, paths, or facilities so one failure does not remove the entire service. It eliminates single points of failure.
Fault tolerance
Allows a system to continue operating when a component fails, ideally with no visible interruption or data loss.
Active/active
Two or more nodes carry production traffic at the same time. This can improve capacity and availability, but requires load distribution and state coordination.
Active/passive
One node serves traffic while another waits in standby. Capacity is not aggregated, but failover protects availability when the active node fails.
Load Balancing, Multipathing, and NIC Teaming
| Technique | What it combines or distributes | Availability benefit |
|---|---|---|
| Load balancing | Requests or workloads across servers, links, or paths | Prevents overload and redirects work from failed nodes |
| Multipathing | Multiple physical paths between hosts and storage | Reroutes I/O after an adapter, switch, controller, or path failure |
| NIC teaming | Multiple network adapters as one logical interface | Provides link failover and, when supported, bandwidth aggregation |
A load balancer may use round-robin to rotate requests evenly or least connections to choose the server currently handling the fewest sessions. Health checks prevent new traffic from being sent to an unavailable node.
Switches, Routers, Firewalls, and Servers
Switch stacking
Multiple physical switches operate as one logical switching unit, commonly with unified management and fast internal failover.
Switch clustering
Separate switches are grouped and managed centrally without necessarily forming a physical stack or one switching plane.
Firewall and server clusters
Cluster members may share traffic in active/active mode or provide an active/passive standby. Application state and session synchronization determine how seamless failover is.
Diverse WAN paths
Multiple ISPs and physically diverse circuits reduce the risk that one carrier, cable route, or facility fault disconnects the organization.
First-Hop Redundancy Protocols
An FHRP lets several physical routers present a single virtual IP address and virtual MAC address as the hosts' default gateway. If the active router fails, a standby assumes the virtual identity.
| Protocol | Type | Core roles and behavior |
|---|---|---|
| HSRP | Cisco proprietary | Active and standby routers exchange hello messages; a hold timer helps identify failure |
| VRRP | Open standard | A master router owns the virtual gateway while backups can take over |
Power Redundancy
Network redundancy cannot preserve service if every device loses electrical power. Resilient facilities combine battery-backed transition power, controlled distribution, and a long-duration supply.
| Component | Role | Important distinction |
|---|---|---|
| Standby UPS | Switches to battery after utility power fails | Economical; a brief transfer time occurs |
| Line-interactive UPS | Corrects moderate voltage variation and switches to battery when needed | Better regulation than standby designs |
| Online UPS | Continuously powers the load through its inverter | Clean, consistent output and effectively no transfer delay; common for critical data-center loads |
| PDU | Distributes power to rack devices | Intelligent models meter load and permit remote control |
| Generator | Supplies long-duration backup power | Requires startup time, fuel, testing, and maintenance |
Environmental Controls and Fire Suppression
Temperature
Excessive heat shortens component life and can trigger shutdowns. HVAC design must move cool air through equipment and remove hot exhaust.
Humidity
High humidity promotes condensation and corrosion; very low humidity increases electrostatic-discharge risk.
Monitoring
Temperature, humidity, airflow, smoke, water, power, and door sensors should generate alerts before conditions become outages.
| Suppression system | How it behaves | Data-center consideration |
|---|---|---|
| Wet pipe | Pipes remain filled with water | Fast and simple, but accidental discharge can damage electronics |
| Dry pipe | Pipes contain pressurized air until activation | Reduces leakage risk; water still discharges after activation |
| Preaction | Detection event admits water before a sprinkler opens | Reduces accidental water discharge and is often used around valuable equipment |
| Deluge | All open nozzles discharge when activated | Large-volume water release is generally unsuitable for computing areas |
| Clean agent | Gas suppresses combustion without conductive residue | Preferred where electronics must be protected; requires safe design and enclosure integrity |
Disaster Recovery Sites
A recovery site provides an alternate environment when the primary facility cannot operate. Cost usually rises as recovery becomes faster and the alternate environment becomes more complete.
| Site | Readiness | Recovery speed | Relative cost |
|---|---|---|---|
| Cold site | Space, utilities, and basic connectivity; equipment and data must be supplied | Slowest | Lowest |
| Warm site | Some hardware, connectivity, and periodically restored data | Moderate | Moderate |
| Hot site | Fully equipped and continuously updated replica | Fastest physical-site recovery | Highest |
| Cloud site | Virtualized recovery environment provisioned or activated in a provider platform | Depends on replication and orchestration | Usage and service dependent |
RPO, RTO, MTTR, and MTBF
Recovery Point Objective (RPO)
The maximum acceptable data loss measured backward in time. An RPO of 15 minutes requires data protection frequent enough to recover to a point no more than 15 minutes before the incident.
Recovery Time Objective (RTO)
The maximum acceptable duration from disruption until a service must be restored. It drives the recovery site's readiness and restoration method.
Mean Time to Repair (MTTR)
The average time required to diagnose, repair, and return a failed component or service to operation. Lower is better.
Mean Time Between Failures (MTBF)
The average operational interval between one repairable failure and the next. Higher generally indicates greater reliability.
Backup and Restoration Strategies
Backups protect against hardware failure, corruption, human error, malicious deletion, ransomware, and site loss. Common destinations include disk-to-disk, disk-to-tape, and disk-to-cloud. Network-device configurations also require protected, versioned backups so routers, switches, firewalls, and controllers can be rebuilt rapidly.
Grandfather–Father–Son Rotation
Son
Frequent short-retention copies, commonly daily, provide recent restore points.
Father
Medium-retention copies, commonly weekly, preserve additional recovery history.
Grandfather
Long-retention copies, commonly monthly or yearly, support archival and compliance needs.
Rotation balances recent recovery options with long-term retention without keeping every backup forever. A sound design also separates copies from the production environment, encrypts sensitive backups, controls access, monitors jobs, and tests restoration.
Tabletop Exercises and Validation Testing
Tabletop exercise (TTX)
A discussion-based simulation in which participants work through a scenario. It validates decision paths, communications, roles, escalation, dependencies, and missing procedures without disrupting production.
Validation test
A practical check that recovery components and procedures work as intended. Results, failures, measured times, and participant feedback drive plan revisions.
Define: Choose objectives, scope, scenario, success criteria, observers, and safety limits.
Exercise: Follow the plan, record decisions and recovery times, and avoid hiding unexpected problems.
Review: Compare actual results with RPO, RTO, responsibilities, dependencies, and communication requirements.
Improve: Assign corrective actions, update documents and training, then retest.
Choose the Best Availability or Recovery Control
Select a scenario to reveal the most appropriate primary technique.
The recommended control or metric will appear here.