Close

ASA Failover and Redundancy

The following must be identical:

  1. Model (5505 or 5510 or 5520, etc)
  2. Amount of RAM
  3. Number of Interfaces (should be the same type as well)
  4. External Modules (CSC-SSM or IPS-SSM)
  5. Activation key with the same features
  • Failover mode
  • Encryption Level
  • Number of VPN peers

*** Same size Flash is not required ****

Definitions:

Active -> Responsibel for creating the state and translation tables, transferring the data packets and monitoring the other units.

Standby -> reponsible for monitoring the status of the active unit

The active and standby units are connected through a dedicated network link and send failover-related messages to each other. This connection, known as theFAILOVER CONTROL LINK is established over a dedicated failover LAN interface.

*** When failover occurs, the standby unit takes over the IP and MAC address that were used by the previous unit

FAILOVER CONTROL LINK communicates:

  • Unit state (active or standby)
  • Network link status
  • Hello or keepalive messages (which are sent on all interfaces. This protocol uses IP 105)
  • MAC address exchange
  • Configuration replication from the active to standby

Conditions that trigger failover:

  • – Administrator has manually switched over from active to standby

-> “no failover active” on the active unit

-> “failover active” on the standby unit

  • – Active unit has lost power or crashed due to a hardware or software defect
  • – Standby unit has stopped receiving hello (or keepalive) packets on the failover interface
  • If 3 consecutive hello packets are missed, additional testing packets are sent to the remaining data-passing interfaces. If it still does not receive a response from the active unit, it assumes that a failure has occured and takes over
  • – The failover control link is down
  • The ASA sends additional testing packets to the remaining interfaces to determine where the peer’s control interface is also down. If the peer’s control interface is also down, then failover does not occure and the failed interface is marked as “Failed”. However, if the peer’s control interface is not down, then failover occurs because the standby unit is deemed healthier than the current active.
  • – The link state of the data-passing interface is down
  • The ASA marks the interface as “Failed” and initiates the failover process. Additionally, if the standby unit does not receive the hello packets for two consecutive polling period on an interface, the appliance goes through a series of additional tests on the interface to determine the root cause of the problem

 

FAILOVER INTERFACE TESTS:

  1. Link up/down tests -> status of the NIC. For example hardware port failure, unplugged cable and a failure on the hub or switch to which the interfaces are connected
  2. Network Activity Test -> All packets received are counted for up to 5 seconds. If any packets are received, the interface is marked as operational
  3. ARP Test -> ARP table is read for the last 10 entries. It sends an ARP request to those machines one at a time and then counts packets for up to 5 seconds. If any packets are received, the interface is marked as operational
  4. Broadcast Ping Test -> Sends a broadcast ping request and then counts all received packets for up to 5 seconds. If any packets are received, the interface is operational.

*** If both active and standby interfaces fail all tests, then both interfaces go into the “unknown state”. The interface with the unknown state do not count toward the monitored inteface failover limit ***

CONNECTION TABLE: Connection entries include the source and destination IP address, protocol used, current state of the connection, the interface to which it is tied and the number of bytes transferred.

In stateful failover, the active unit sends updates to the standby unit whenever these is a change in the state table. In this mode, the active unit sends stateful updates over a dedicated link to the the standby unit. When the standby unit becomes active, it does not need to build any connection entries because all the entries already exist in its database.

*** You can use the same physical interface for both failover control and stateful link. However it is not recommended if your appliances generate a lot of state updates. Additionally, it is recommended that you use the fastest interface as the stateful link and that the latency for the link should be less than 10 ms to avoid performance degradation. ****

What is not replicated in stateful failover:

  1. Uauth cache
  2. URL Filtering cache
  3. TCP Intercept
  4. SNMP Firewall MIB
  5. Routing Table
  6. State Info for SSM
  7. Phone Proxy sessions
  8. IPv6 sessions

Types of failover -> Device-level failover and interface-level failover

Device-level failover -> Active/Standby and Active/Active

Active/Standby Failover:

  • Active unit passes traffic
  • Standby unit monitors active unit
  • Both units send hello messages to monitor each other’s state

 

ELECTION PROCESS

  1. When both are up and running, one unit assumes the role of active while the other unit assumes the standby role
  2. If both units boot up simultenously, the primary unit takes over the active role and the secondary goes into standby. The primary unit uses the active IP address and its MAC address as the Layer 3 and Layer 2 respectively. If failover occurs, the secondary keeps using the IP address and the primary’s MAC addrss as the active
  3. If one of the units boots up and detects an active failover unit, it goes into standby regardless of the primary or secondary designation
  4. If one of the units boots up and doesn’t detect an active failover unit, it goes into the active state regardless of its primary or secondary designation
  5. In case both appliances become active, the secondary changes its state to standby as soon as it discovers another active primary firewall while the primary remains active
  6. In case both units become standby, the primary changes its state to active, while the secondary remains standby after they detect each other’s state

ACTIVE/ACTIVE FAILOVER

  • Both units pass traffic actively
  • Only supported in multimode
  • Needs at least two user context to work properly
  • With stateful failover, both stand connection tables are replicated
  • Uses per-failover groups instead of per-context

**** It is recommended that you don’t oversubscribe the firewalls in active/active failover as during failover, one of the units has to pass all traffic

In active/active failover, it is possible that packets can leave one active unit and can return to the other active unit. The ASA implements a feature known as “asymmetric routing” to guide packets back to the context from which they originated.

ACTIVE/ACTIVE FAILOVER AND ASYMMETRIC ROUTING

  • Multiple ISPs or remote locations
  • Load-balancing or backup
  • ** only supported in multimode

If the asymmetric routing feature is enabled, the units restore asymmetric routed packets to the correct interface. The unit that sent the original SYN packet will replicate the connection table entry for the SYN packet to the standby unit over the stateful failover link. When the standby unit receives a packet but does not have an active connection, it checks other interfaces that are in the same asymmetric routing group for the corresponding connection. The Layer 2 information is written and forwarded to the original unit.

INTERFACE-LEVEL FAILOVER – REDUNDANT INTERFACES

During traditional device-level failover, traffic gets disrupted even when stateful failover is enabled.

Example

  • All incomplete TCP sessions
  • Routing tables (OSPF, RIP, EIGRP, etc)
  • Most Inspection Engines

Interface-level failover is used to avoid device-level switchover which is much more disruptive

Redundant interfaces are only supported in version 8.0 and higher.

GUIDELINES

  • All interfaces supported except Ma0/0 or Ma0/1
  • Up to 8 redundant interfaces
  • Network-related commands go on the redundant interface (nameif, security-level, IP address, etc)
  • Speed, duplex and shutdown go on the physical inteface
  • Link status for the physical is monitored by default
  • The standby interface drops all inbound packets and does not send any outbound packets
  • Interface stats on the redundant interface are a summation of the active and standby interfaces. As soon as a physical interface becomes a member of the redundant interface, its interface stats are cleared
  • Redundant interfaces use the MAC address of the first member physical interface unless you are manaully assigning a unique virtual MAC address on the redundant interface
  • *** The physical interfaces in the redundant interface must both be of the same physical type

*** The redundant interfaces can be configured with or without device-level failover

ACTIVE/STANDBY FAILOVER CONFIG

  1. Select the failover link
  2. Assign failover IP address
  3. Set the failover key (optional)
  4. Designate the primary unit
  5. Enable the stateful failover (optional)
  6. Enable failover globally
  7. Configure failover on the secondary unit

*** There can’t be a “nameif” command on an interface when configuring failover on it. Also “managment-only” needs to be removed.

You can use the “failover exec” command to send commands to the correct unit.

Failover link is used for failover control messages

*** If a failover key is not used, the active appliance sends all information in clear text, including the TCP/UDP states, the user credentials and VPN-related information.

You must designate the primary and secondary status through software configureation

If you want to use the failover control interface as the stateful failover link, use the “failover link” command without specifying the physical interface

BOOTSTRAP CONFIG ON SECONDARY

Enable the failover control interface

“no shutdown”

Failover designation as secondary

“failover lan unit secondary”

  • Failover link interace
  • “failover lan interface failover Gi0/1”
  • Same failover interface IP addresses
  • “failover interface ip failover 1.2.3.4 255.255.255.0 standby 1.2.3.5”
  • Same failover shared key
  • “failover key cisco 123”
  • Failover enable (enabled globally)
  • “failover”

 

ACTIVE/ACTIVE FAILOVER CONFIG

  • Select the failover link
  • Assign failover interface IP addresses
  • Set the failover key (optional)
  • Desginate the primary unit
  • Enable stateful failover (optional)
  • Set up failover group
  • Assign failover-group membership
  • “join-failover-group”
  • Assign interface IP addresses
  • Set up asymmetric routing (optional)
  • Enable failover globally
  • Configure failover on the secondary

**** Connection and translation tables are sent over the stateful failover link. So is the asymmetric routing updates

OPTIONS FOR FAILOVER GROUPS

  • Interface-policy
  • MAC address
  • polltime
  • Preempt
  • Primary
  • Replication
  • Secondary

**** You must enable HTTP inspection to allow the HTTP traffic to work in the Active/Active setup with asymmetric routing

*** The Admin context is always a part of failover group 1. If you don’t explicitly map a context to a failover group, then it is automatically assigned to group 1.

*** The “http replication” command in the failover group overrides the global “http replication” command

OPTIONAL FAILOVER COMMANDS

Specify failover MAC address – in Active/Active failover, during failover, the IP addresses ad MAC addresses are swapped between units. The new active unit sends out a gratuitous ARP on the network. Using gratuitious ARP, the Layer 2 devices including bridges and switches, also update the CAM tables with the MAC address and the updated switch port information. The use of virtual MAC addresses is useful to avoid disruptions. With the virtual MAC address, the appliances do not need to swap the MAC address.

ACTIVE/STANDBY MAC ADDRESS PRIORITY LIST

  • Use the virtual MAC cofigured on the interfaces (physical, logical or subinterface)
  • Use the “failover mac address ” configured globally
  • Use the burned-in MAC address

ACTIVE/ACTIVE MAC ADDRESS PRIORITY LIST

  • Use the virtual MAC at the interface level (physical, logical or subinterface)
  • Use the “mac address” in the failover group configuration mode
  • Use the “mac address auto” in the System Execution Space
  • Use the burned-in MAC on the physical interface

**** In Active/Active setup, the failover “mac address” command has no effect if issues in the global configuration mode

INTERFACE POLICY

  • Status of all interfaces monitored
  • If one of the interfaces fails to repsond, failover occurs

If you want to change this behavior, use the “criteria” tab in the ASDM

*** This setting is applied to the failover group in Active/Active

FAILOVER TIMERS

By default, ASA sends periodic keepalives (hello) packets to check the status of the peer failover unit. If the standby unit does not receive acknowledgement packets for the keepalives sent out, it initiates a failover only if it deems itself healthier than the current unit.

Two types of failover hello messages -> Unit (sent every second)

-> Interface (every 15 seconds)

In active/active failover, unit failover timers are changed in the System Execution Space. The interface failover timers are changed in the failover group policy.

*** When an ASA is configured for failover, whether active/standby or active/active, it monitors the status of all the main physical interfaces that have a “nameif” and an IP address configured.

*** Interface monitoring is disabled by default on all subinterfaces until you explicitly enable it.

*** Stateful failover is mandatory for a zero-downtime software upgrade.

Cisco ASA Firewall Active / Standby Failover

The Cisco ASA firewall is often an important device in the network. We use it for (remote access) VPNs, NAT/PAT, filtering and more. Since it’s such an important device it’s a good idea to have a second ASA in case the first one fails.

The ASA supports active/standby failover which means one ASA becomes the active device, it handles everything while the backup ASA is the standby device. It doesn’t do anything unless the active ASA fails.

The failover mechanism is stateful which means that the active ASA sends all stateful connection information state to the standby ASA. This includes TCP/UDP states, NAT translation tables, ARP table, VPN information and more.

When the active ASA fails, the standby ASA will take over and since it has all connection information, your users won’t notice anything…

There are a number of requirements if you want to use failover:

  • Platform has to be the same: for example 2x ASA 5510 or 2x ASA 5520.
  • Hardware must be the same: same number and type of interfaces. Flash memory and RAM has to be the same.
  • Same operating mode: routed or transparent mode and single or multiple context mode.
  • License has to be the same..number of VPN peers, encryption supported, etc.
  • Correct license. Some of the “lower” models require the Security Plus license for failover (the ASA 5510 is an example).

In this lesson we’ll take a look how to configure active/standby failover. Here’s the topology I will use:

ASA1 ASA2 Active Standby Failover

We have two ASA firewalls…ASA1 and ASA2. ASA1 will be the active firewall and ASA2 will be in standby mode. Their Ethernet 0/0 interfaces are connected to the “INSIDE” security zone while the Ethernet 0/1 interfaces are connected to the “OUTSIDE” security zone. The Ethernet 0/3 interface in the middle will be used to synchronize connection information for failover. R1 and R2 are only used so we can generate some traffic.

Configuration

We will start with the failover interface on ASA1. Make sure it’s not shut:

ASA1(config)# interface Ethernet 0/3
ASA1(config-if)# no shutdown

And then we configure this ASA to be the active (primary) device:

ASA1(config)# failover lan unit primary

Now we will configure Ethernet 0/3 to be the failover interface:

ASA1(config)# failover lan interface FAILOVER Ethernet 0/3 
INFO: Non-failover interface config is cleared on Ethernet0/3 and its sub-interfaces

And we’ll tell the ASA to use this interface for stateful failover:

ASA1(config)# failover link FAILOVER Ethernet 0/3

We can now configure the IP addresses on the failover interface. We need to use a dedicated subnet for this:

ASA1(config)# failover interface ip FAILOVER 192.168.12.1 255.255.255.0 standby 192.168.12.2

ASA1 (active) will use IP address 192.168.12.1 and ASA2 (standby) will use 192.168.12.2. Now we can enable failover:

ASA1(config)# failover

Failover is now configured on ASA1. Let’s configure some security zones and IP addresses on the “normal” Interfaces:

ASA1(config)# interface Ethernet 0/0
ASA1(config-if)# no shutdown
ASA1(config-if)# nameif INSIDE
ASA1(config-if)# ip address 192.168.1.254 255.255.255.0 standby 192.168.1.253
ASA1(config)# interface Ethernet 0/1
ASA1(config-if)# nameif OUTSIDE
ASA1(config-if)# ip address 192.168.2.254 255.255.255.0 standby 192.168.2.253

The ASA requires something that triggers the failover mechanism. An interface that fails is a good trigger. When the inside or outside interface fails, we should failover. By default all physical interfaces are monitored but let me show you the command anyway:

ASA1(config)# monitor-interface INSIDE
ASA1(config)# monitor-interface OUTSIDE

This is all we have to configure. We can now configure ASA2:

ASA2(config)# failover lan unit secondary
ASA2(config)# failover lan interface FAILOVER Ethernet 0/3
ASA2(config)# failover link FAILOVER Ethernet 0/3
ASA2(config)# failover interface ip FAILOVER 192.168.12.1 255.255.255.0 standby 192.168.12.2
ASA2(config)# failover

We configure ASA2 to be the standby device, its Ethernet 0/3 interface will be used for failover and we configure the active and standby IP addresses. Let’s enable this interface so that the ASA’s can talk with each other:

ASA2(config)# interface Ethernet 0/3
ASA2(config-if)# no shutdown

This is what you will see on ASA1 and ASA2:

ASA1#
Beginning configuration replication: Sending to mate.
End Configuration Replication to mate
ASA2#
Failover LAN became OK
Switchover enabled
Configuration has changed, replicate to mate.

State check detected an Active mate
Beginning configuration replication from mate.
End configuration replication from mate.

Switching to Standby

Failover is up and running, the configuration has been replicated from ASA1 to ASA2. Whenever you make changes to the configuration, you only have to save on the active ASA and it will be replicated to the standby ASA:

ASA1# write memory 
Building configuration...
Cryptochecksum: 690a4de8 e1179377 f8eabae6 8cf5242e 

3372 bytes copied in 3.240 secs (1124 bytes/sec)

After saving the configuration you will see this on the standby ASA:

ASA1# Cryptochecksum: 5739e8f7 32355bc0 a97e7dfa dd54ad71 

3373 bytes copied in 3.240 secs (1124 bytes/sec)

Let’s see if failover is really working shall we?

Verification

A simple method to verify if its working is to check the show failover command:

ASA1# show failover 
Failover On 
Failover unit Primary
Failover LAN Interface: FAILOVER Ethernet0/3 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 2 of 110 maximum
Version: Ours 9.1(5), Mate 9.1(5)
Last Failover at: 12:23:34 UTC Dec 19 2014
	This host: Primary - Active 
		Active time: 1664 (sec)
		slot 0: ASA5510 hw/sw rev (2.0/9.1(5)) status (Up Sys)
		  Interface INSIDE (192.168.1.254): Normal (Monitored)
		  Interface OUTSIDE (192.168.2.254): Normal (Monitored)
		slot 1: empty
	Other host: Secondary - Standby Ready 
		Active time: 31 (sec)
		slot 0: ASA5510 hw/sw rev (1.1/9.1(5)) status (Up Sys)
		  Interface INSIDE (192.168.1.253): Normal (Monitored)
		  Interface OUTSIDE (192.168.2.253): Normal (Monitored)
		slot 1: empty

Stateful Failover Logical Update Statistics
	Link : FAILOVER Ethernet0/3 (up)
	Stateful Obj 	xmit       xerr       rcv        rerr      
        General		90         0          89         0         
        sys cmd  	89         0          89         0         
        up time  	0          0          0          0         
        RPC services  	0          0          0          0         
        TCP conn 	0          0          0          0         
        UDP conn 	0          0          0          0         
        ARP tbl  	0          0          0          0         
        Xlate_Timeout  	0          0          0          0         
        IPv6 ND tbl  	0          0          0          0         
        VPN IKEv1 SA 	0          0          0          0         
        VPN IKEv1 P2 	0          0          0          0         
        VPN IKEv2 SA 	0          0          0          0         
        VPN IKEv2 P2 	0          0          0          0         
        VPN CTCP upd 	0          0          0          0         
        VPN SDI upd 	0          0          0          0         
        VPN DHCP upd 	0          0          0          0         
        SIP Session 	0          0          0          0         
        Route Session 	0          0          0          0         
        User-Identity 	1          0          0          0         
        CTS SGTNAME 	0          0          0          0         
        CTS PAC 	0          0          0          0         
        TrustSec-SXP 	0          0          0          0         
        IPv6 Route 	0          0          0          0         
              
        Logical Update Queue Information
                 	Cur 	Max 	Total
        Recv Q: 	0 	2 	91
        Xmit Q: 	0 	25 	482

This gives us a nice overview, you can see which device is active/standby but also what kind of stateful information is being exchanged. Let’s create some telnet traffic between R1 and R2 so that you can see that the firewalls are exchanging TCP connection information:

R1#telnet 192.168.2.2
Trying 192.168.2.2 ... Open

User Access Verification

Password: 

R2>enable
Password: 
R2#

When we check the show failover command again you will see this:

ASA1# show failover | include TCP
	TCP conn 	5          0          0          0

The connection information for my TCP session has been exchanged between the two ASAs. To really test our failover, we have to simulate a link failure. I’ll shut the interface on my switch that connects to the Ethernet 0/0 interface of ASA1:

SW1(config)#interface FastEthernet 0/14
SW1(config-if)#shutdown

Now you will see this on the active ASA:

Switching to Standby

And the standby ASA will tell us:

Switching to Active

Of course we can also check the show failover command again:

ASA1# show failover | include This host
	This host: Primary - Failed
ASA1# show failover | include This host
	This host: Secondary - Active

This proves that failover is working as it should, the standby ASA has become active after the link failure.

Active/standby failover does not use preemption. Once you enable the interface again, the currently active ASA will remain active.

That’s all there is for now! I hope you enjoyed this lesson, if you have any questions…feel free to leave a comment!

Back to top