Checkpoint: Troubleshooting VRRP on Nokia Firewalls

Troubleshooting VRRP on Nokia Checkpoint Firewalls

The purpose of this article is to help in troubleshooting VRRP related issues on NOkia Checkpoint Firewalls. One of the most common problems faced in Nokia VRRP Implementations is that interfaces on active and standby firewalls go into the master master state. THe main reason for this is because the individual vrids of the master and backup firewall are not able to see the vrrp multicast requests of each other.

 

The first step is to check the vrrp state of the interfaces. THis is how you can check that:

 

PrimaryFW-A[admin]# iclid
PrimaryFW-A> sh vrrp

 

VRRP State
Flags: On
6 interface enabled
6 virtual routers configured
0 in Init state
0 in Backup state
6 in Master state
PrimaryFW-A>
PrimaryFW-A> exit

 

Bye.
PrimaryFW-A[admin]#

 

SecondaryFW-B[admin]# iclid
SecondaryFW-B> sh vrrp

 

VRRP State
Flags: On
6 interface enabled
6 virtual routers configured
0 in Init state
4 in Backup state
2 in Master state
SecondaryFW-B>
SecondaryFW-B> exit

 

Bye.
SecondaryFW-B[admin]#

 

In the example shown you see that 2 interfaces each from both firewalls are in the Master state.

 

The next step should involve running tcpdumps to see if the vrrp multicasts are reaching the particular interface.

 

As the first troubleshooting measure, put a tcpdump on the problematic interface of the master and backup firewalls. If you want to know what the problematic interface is, “echo sh vrrp int | iclid” should give you the answer. It is that interface on the backup firewall which would be in a Master state.

 

PrimaryFW-A[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
00:46:11.379961 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:12.399982 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:13.479985 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:14.560007 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]

 

When you put a tcpdump on the Primary Firewall, you see that the vrrp multicast request is leaving the interface.

 

Next put the tcpdump on the secondary firewall.

 

SecondaryFW-B[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
00:19:38.507294 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:39.527316 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:40.607328 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:41.687351 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:42.707364 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]

 

Now you can see that the interface on both the primary and the secondary firewalls are broadcasting vrrp multicasts. This is because the vrrp multicasts are not reaching the firewalls interfaces. This means there is a communication breakdown which can be possibly caused by network issues.

 

Once the network issue is resolved, communication would be possible and the interface with the lower priority will go as the secondary or backup state.

 

Now let us discuss another scenario where there is a problem with the firewall interfaces in Master Master state.

 

Again put a tcpdump on both the interfaces in question:

 

PrimaryFW-A[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
00:46:11.206994 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]
00:46:11.379961 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:12.286990 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]
00:46:12.399982 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:13.307014 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]
00:46:13.479985 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:14.387098 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]
00:46:14.560007 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
00:46:15.467064 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]
00:46:15.580010 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]

 

SecondaryFW-B[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
00:19:38.507294 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:38.630075 I 10.10.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 100 [tos 0xc0]
00:19:39.527316 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:39.710131 I 10.10.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 100 [tos 0xc0]
00:19:40.607328 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:40.790142 I 10.10.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 100 [tos 0xc0]
00:19:41.687351 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]
00:19:41.810150 I 10.10.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 100 [tos 0xc0]

 

In the above example look at the vrid numbers of the incoming and outgoing packets. From the vrids you see that that the vrids donot match. This is an indication that the cabling is not correct. The cables going to vrid 102 and 103 are not connected correctly and they need to be swapped to fix this issue.

 

Swap the cables and the issue will be resolved. The firewall with the higher priority will go into the Master state.

 

A properly functioning firewall will be like this:

 

PrimaryFW-A[admin]# iclid
PrimaryFW-A> sh vrrp

 

VRRP State
Flags: On
6 interface enabled
6 virtual routers configured
0 in Init state
0 in Backup state
6 in Master state
PrimaryFW-A> exit

 

Bye.
PrimaryFW-A[admin]#

 

SecondaryFW-B[admin]# iclid
SecondaryFW-B> sh vrrp

 

VRRP State
Flags: On
6 interface enabled
6 virtual routers configured
0 in Init state
6 in Backup state
0 in Master state
SecondaryFW-B> exit

 

Bye.
SecondaryFW-B[admin]#

 

If you were to tcpdump the healthy interface, this is how it would look:

 

PrimaryFW-A[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
18:25:44.015711 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:25:45.095726 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:25:46.175751 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:25:47.195770 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:25:48.275819 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:25:49.355812 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
^C
97 packets received by filter
0 packets dropped by kernel
PrimaryFW-A[admin]#

 

SecondaryFW-B[admin]# tcpdump -i eth-s4p2c0 proto vrrp
tcpdump: listening on eth-s4p2c0
18:26:07.415446 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:08.495451 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:09.515480 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:10.595486 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:11.675485 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:12.695522 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
18:26:13.775590 I 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]
^C
14 packets received by filter
0 packets dropped by kernel
SecondaryFW-B[admin]#

Credit to Secmanager

 

Checkpoint: Configuring Simplified VRRP on Nokia IP Appliances

This is a popular one and one I keep coming back to so here it is ..

In this example, we will configure VRRP with Check Point VPN-1/FireWall-1 NGX using Simplified VRRP Configuration feature of Nokia IPSO. It is assumed that the physical interfaces on the Nokia appliances are already configured, that Check Point packages are already installed and that the Management server object is already configured. Before beginning, ensure that the time and date on the modules and Management Server are correct.

For this example IPSO 4.2 and NGX (R65) was used.

We will use the following IP addresses.

FirewallA (master) External Eth1c0 10.207.122.201/24
Sync Eth3c0 172.31.10.1/24
Internal Eth2c0 192.168.10.11/24
FirewallB (backup) External Eth1c0 10.207.122.202/24
Sync Eth3c0 172.31.10.2/24
Internal Eth2c0 192.168.10.12/24
VRRP Addresses External 10.207.122.205
Internal 192.168.10.205
Management Server 192.168.10.10

We will begin by configuring VRRP in Network Voyager.

  1. Log into Voyager on the master (Firewall A)
  2. Expand the Configuration section then expand HighAvailability and then finally click on VRRPThis page will allow you to create the Virtual Router. In Simplified VRRP, only one Virtual Router is created.
  3. In the box “Create a new Monitored-Circuit Virtual Router:” enter 10
  4. Click APPLY .As you can see, the new virtual router has a VRID of 10. Its default “Priority” is 100 and default “Hello Interval” is 1. As FireWall A will be our VRRP Master, leave the Priority set at 100.
  5. In the box “Priority Delta” enter 10
  6. In the box “Backup Address” enter our external VRRP IP address of 10.207.122.205
  7. Click APPLY .Upon refresh, we will be able to enter our second (internal) VRRP IP.
  8. In the new box labeled “Backup Address” enter our internal VRRP IP address of 192.168.10.205 . The “VMAC Mode” should be left to VRRP and the “Static VMAC” box left blankNote: – DO NOT enter the Sync interface here as we do not want the Sync network monitored.
  9. Make sure that the option at the top of the page “Accept Connections to VRRP IP’s” is set to Enabled
  10. Also, for now, we will set “Monitor Firewall State” to Disabled
  11. Don’t forget to click APPLY .
  12. Finally we can click the SAVE button.You will need to create hostsentries on each module. This is done in Voyager, under Configuration, System Configuration then Host Address.There should be 4 entries hereOne for localhost, one for the Management Server, one for external IP of the local host, and one for the external IP of the other member.

    So in our example here, we would have these entries:

    localhost 127.0.0.1

    Master 10.207.122.201

    Backup 10.207.122.202

    Management 192.168.10.10

    ** The hostnames are the same as the name of the objects that will be used to represent each of these in SmartDashboard

    Save your configuration

    Once these steps have been completed, log into the backup (Firewall B)

    Perform the same steps with only one exception

    The priority of the Virtual Router on the backup will be 95.

    All other settings are the same

    Now log into each Nokia Appliance through console and run cpconfig

    ** If this is the first time running cpconfig, you will have to go through the Check Point configuration prompts first.

    Select option #6, which should read “Enable cluster membership for this gateway”

    It will ask you if you are sure, select [y]

    ** If this option reads “”Disable cluster membership for this gateway” then it has already been enabled

    This completes the VRRP configuration on the Nokia Appliances.

Configuring the Smart Center Server (Management Server)

To avoid asymmetrical routing, we will need to add 2 static routes on the management server. This only applies if the management server is on an internal network behind the VRRP pair.

Static route 1

To reach the external interface of Firewall A, go to Firewall A’s internal interface

So using our addressing schemes here running a Microsoft Windows Management Server, the command would be:

route add 10.207.122.201 mask 255.255.255.255 192.168.10.11 -p

Static route 2

To reach the external interface of Firewall B, go to Firewall B’s internal interface

route add 10.207.122.202 mask 255.255.255.255 192.168.10.12 -p

This will prevent traffic from the Management Server destined for Firewall B to be routed through Firewall A

Log into SmartDashboard

Create a Check Point Gateway object for each firewall module and define the object with the member’s external IP address.

Example:

FirewallA

10.207.122.201

FirewallB

10.207.122.202

Creating the objects with the internal IP addresses will cause problems with VPNs

Establish SIC with each module.

** If you are having difficulty establishing SIC, make sure the module is reachable then run fw unloadlocal on the modules and try again.

In addition, under Checkpoint Products under the general properties, make sure that Firewall is checked.

In the “Topology” section of each object, click “Get”, then “Interfaces with topology”

This will fetch the interface configurations.

Be sure to click OK to save this information.

Now create a new Check Point Gateway Cluster

The cluster Object should be defined using the external VRRP address.

10.207.122.205

Additionally, ensure that “ClusterXL” is NOT checked under Checkpoint products.

In the Cluster Members section, click “Add” then select “Add Gateway to Cluster”

Select the object we created for Firewall A and click ok

A message will inform you that some of this object’s data will be lost. Select “Yes”

Now follow the same steps to add Firewall B to the cluster.

Click OK

In the Gateway Cluster Properties

Go into the “3rd Party Configuration” section

Make sure that “High Availability” is selected for the Cluster operation mode.

Change the 3rd Party Solution from OPSEC to Nokia VRRP

Make sure that the check box for “Use State Synchronization” is selected

Now back to the Topology section, click Edit Topology.  First try clicking “get interfaces with topology” to fetch the VRRP interface configuration.  It should then appear as shown in the table below.  If it does not, then you need to manually enter that information.  Based on the network scheme above in this example, this is what the table should appear once configured.

VRRP

network obj. FirewallA FirewallB Topology
Get Top. Get Top. Get Top.
Name Cluster Eth2c0 Eth2c0 Eth2c0
IP Addr 192.168.10.205 192.168.10.11 192.168.10.12 Internal
Net Mask 255.255.255.0 255.255.255.0 255.255.255.0
Name Cluster Eth1c0 Eth1c0 Eth1c0
IP Addr 10.207.122.205 10.207.122.201 10.207.122.202 External
Net Mask 255.255.255.0 255.255.255.0 255.255.255.0
Name 1st Sync Eth3c0 Eth3c0
IP Addr 172.31.10.1 172.31.10.2 Internal
Net Mask 255.255.255.0 255.255.255.0

**Important

DO NOT add the sync interface under the Cluster as it is not a monitored interface

Click OK. The cluster object configuration is now completed.

Now push policy to the cluster object. If it fails to push the policy, run fw unloadlocal on the modules again to ensure that there is not a policy installed that could be blocking communication.

If there are console error messages that show up when policy is pushed (for example, antispoofing is not correctly defined) then you will need to go back to the VRRP configuration page and enable “Monitor Firewall State”.  For more information about this option please refer to resolution kb1355466

Log back into Network Voyager, go back into the VRRP configuration page then click on VRRP Monitor.
On Firewall A, you should see 0 interfaces in Backup state and the number of monitored interfaces in Master state (In our example here, 2 interfaces would be in Master state)

On Firewall B, you should see 0 interfaces in Master state and the number of monitored interfaces in Backup state. (In our example here, 2 interfaces would be in Backup state)

By running the command cphaprob stat on each module, you should see the local sync interface and the member’s sync interface as “active”. This indicates that state sync is communicating.

This ends the VRRP configuration.