eVIP, Gateway Unavailable

Evolved Virtual IP

1	Introduction
1.1	Prerequisites
2	Alarm Description
2.1	Alarm Attributes
3	Procedure
3.1	L3 Connectivity
3.2	Configuration
3.3	Next Level Support

1 Introduction

This document is the Operating Instruction (OPI) for alarm eVIP, Gateway Unavailable.

1.1 Prerequisites

This section describes the possible documents, tools, and conditions needed before performing the steps described in Section 3 Procedure.

1.1.1 Documents

Before starting this procedure, ensure that the following document have been read:

eVIP Management Guide
eVIP Internetworking

2 Alarm Description

The alarm is issued when external gateway connectivity is lost or there is reoccurring instability (bouncing/flapping) with the connection to an external gateway.

The possible causes are as follows:

The external gateway is faulty.
The connection between the external gateway and the cluster is faulty.
The node in the cluster that is connected to the external gateway is faulty.
The external gateway is not used for packet forwarding, because there is a path through an other external gateway with lower cost.
The configuration is incorrect.
OSPF on the gateway router has chosen to withdraw the route.

The routing software supervises the connection between the external gateway and the connected node in the cluster. The method of supervising is dependent upon the configuration, for example, which routing protocol is used. In the case of OSPF there can be one supervised gateway or sometimes several, for static routes there can only be one supervised gateway router per FEE. In either case no alarm is raised until the FEE has no default gateways left to route packets to.

There are at least two external gateways configured for redundancy under each Abstract Load Balancer (ALB). If one of the external gateway routers becomes unavailable, redundancy is lost, however, traffic will use the remaining gateway router. If contact is lost to all external gateways under the same ALB then all traffic is lost.

The Supervised Remote Gateway feature is supported with the following eVIP routing types; ospfv2, ospfv3, bfd_ospfv2, bfd_ospfv3, bfd_static, and bfd_static6.

2.1 Alarm Attributes

This alarm is compliant with Ericsson SNMP Fault Management MIB, which conforms to X.733 alarm reporting function. However, the following X.733 parameters are not supported; Correlated Notifications, Additional Info, Monitored Attributes, Proposed Repair Action, Trend Indication, Threshold Information, Backed Up Object, and State Change Definition.

The most essential statical attributes of this alarm and their values are listed in Table 1:

Table 1 Alarm Attributes
Attribute Name	Attribute Value
majorType	193
minorType	2129526785 (0x7eee0001)
class	EvipSupervisedRemoteGateway
source	`evipSupervisedRemoteGatewayId=<ip_address>,evipFeeId=<name>,evipFeesId=1,evipAlbId=<name>,evipAlbsId=1,evipId=1`
specificProblem	eVIP, Gateway Unavailable
eventType	COMMUNICATION
activeSeverity	MAJOR

The Alarm Type of the alarm is identified by the two integers: majorType and minorType. The Alarm Type is unique within the system type and maps to the X.733 Managed Object Instance. The eventType, probableCause, and specificProblem are always the same for a given Alarm Type.

3 Procedure

To clear the alarm, the connection between the external gateway and the node in the cluster must be restored.

If the alarm is issued on installation or after a change in the configuration, it is likely that the problem is caused by a bad configuration.

The Managed Object pointed out by its Distinguished Name (DN) in the alarm, which is an object of the EvipSupervisedGateway class. This object has a description attribute, which is defined by the installer or by the operator. This attribute can contain site-specific information about the gateway, for example, the location and the person to contact.

3.1 L3 Connectivity

This section describes the procedure to check L3 connectivity.

3.1.1 Collect Information for L3 Connectivity

The supervised gateway address is present as part of the source attribute of the alarm, see Example 1. In cases where multiple Supervised Remote Gateway addresses have been configured under a single FEE, there may be several alarms for the same FEE. Be sure to check if there are other alarms from the same FEE and collect the info from each as shown in Example 1.

Example 1 Source Attribute, Gateway Address

>show ManagedElement=1,SystemFunctions=1,Fm=1,FmAlarm=6,source
   source="ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3,
EvipSupervisedRemoteGateway=192.168.77.99"

If an alarm has been raised for an FEE one should also check to see how many Supervised Remote Gateway Alarm objects are configured under the FEE in question. This can be done using the command shown in Example 2, the IP addresses from each of these objects should be noted and used to determine connectivity. Example 2 shows a configuration where there are multiple supervised gateways, typically there should be one.

Example 2 Source Attribute, Supervised Remote Gateway

>show ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3
EvipFee=fee_3
   externalInterface=eth3
   node=”1”
   state=”ACTIVE”
   EvipRoutingSetup=bfd_ospfv2
   EvipSupervisedRemoteGateway=192.168.77.98"
   EvipSupervisedRemoteGateway=192.168.77.99"

The DN of the eVIP Front-End Element (FEE) reporting the problem, and it is the part of attribute source up until EvipSupervisedRemoteGateway, see Example 3.

Example 3 Source Attribute, Distinguished Name of FEE

>show ManagedElement=1,SystemFunctions=1,Fm=1,FmAlarm=6,source
   source="ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3,
EvipSupervisedRemoteGateway=192.168.77.99"

The physical blade and external interface, that the eVIP FEE is using, can be obtained from the eVIP Managed Object Model, see Example 4. Please note that in virtual type deployments if the distribution attribute associated with the nodes has been configured as "floating", then the VM that the node is running on now may be different than the one that it was configured upon originally. This needs to be factored in if you are trying to locate the VM that the FEE in question is running on. Care must be taken to ensure that the necessary internal eVIP data is gathered when opening a trouble report on potential issues.

Example 4 Blade and External Interface of FEE

>show ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3,node
node="8"
>show ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3,
externalInterface
externalInterface="eth3"

The external IP address of the eVIP FEE can be obtained from the eVIP MOM, see Example 5.

Example 5 Local Address of FEE

>show all ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3,
EvipRoutingSetup=ospfv2,EvipParam=local_address
EvipParam=local_address
   value="192.168.14.10/30"

3.1.2 Check L3 Connectivity

Establish whether there is L3 connectivity between the eVIP FEE and the external gateway routers, or not, using the IP addresses gathered in Section 3.1.1 Collect Information for L3 Connectivity. In some cases, when OSPF is configured on the FEE, there could be more than one remote supervised gateway IP address configured under one or more FEEs to verify. With multiple supervised gateway IP addresses configured per FEE, the alarm will only be raised once all the supervised remote gateways under the FEE become unavailable. Since this scenario implies that all external connectivity is lost for the FEE, it is important to confirm how many supervised gateways are provisioned because it could mean that all connectivity to the FEE or ALB has been lost.

If there is only a single supervised remote gateway IP address configured per FEE, there may still be connectivity to one of the gateway routers, however, redundancy may have been lost because of a faulty router (for example). This situation needs to be investigated and corrected as the alarm is indicating that the preferred gateway for the FEE in question has gone away, even though traffic may still be flowing.

Checking connectivity to the gateways can be done by using ICMP echo-reply (ping) and should be done to all possible next hop addresses, as follows:

Try a continuous ping to the addresses of the external gateway routers from another device (if available) in the same LAN.
Try a continuous ping to the external address of the eVIP FEE from another device (if available) in the same LAN.
Try a continuous ping to the external address of the eVIP FEE from each of the external gateway routers (using the supervised gateway address as a source).

If all pings fail (for example, no L3 connectivity) or there are intermittent ping failures (route/link flapping) check the external gateway router and all intermediary network equipment (switches, cables, and so on) between the router and the eVIP FEE.

Also check the state of the physical interface which is used by the eVIP FEE (if logging on to the VM/blade is possible). Please note that it may be necessary to login to the network namespace of the FEE to check the interface state. See Example 6.

Example 6 State of Physical Interface

PL-2-8:~ # ifconfig eth3
eth3      Link encap:Ethernet  HWaddr 00:13:5E:E8:EB:A9  
          inet6 addr: fe80::213:5eff:fee8:eba9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16621 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14613 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1883100 (1.7 Mb)  TX bytes:1306050 (1.2 Mb)
          Memory:fdee0000-fdf00000

3.2 Configuration

If there is L3 connectivity but the alarm is still not cleared, check the routing setup in the eVIP MOM and also on the external router. Make sure that the IP addresses and the routing protocol settings are correct. For an example of full routing configuration on eVIP side, see Example 7.

Example 7 Full Routing Configuration on eVIP Side

>show all ManagedElement=1,Transport=1,Evip=1,EvipAlbs=1,EvipAlb=alb_1,EvipFees=1,EvipFee=fee_3
EvipFee=fee_3
   externalInterface="eth3"
   node="8"
   state="ACTIVE"
   EvipRoutingSetup=bfd_ospfv2
      EvipParam=bfd_interval
         value="300"
      EvipParam=echo
         value="no"
      EvipParam=minrx
         value="300"
      EvipParam=multiplier
         value="3"
   EvipRoutingSetup=ospfv2
      EvipParam=area
         value="10.4.35.1"
      EvipParam=area_type
         value="stub"
      EvipParam=dead_interval
         value="7"
      EvipParam=hello_interval
         value="1"
      EvipParam=local_address
         value="192.168.14.10/30"
      EvipParam=retransmit_interval
         value="5"
      EvipParam=router_id
         value="192.168.14.10"
      EvipParam=router_priority
         value="0"
      EvipParam=spf_delay
         value="500"
      EvipParam=spf_interval
         value="1000"
      EvipParam=transmit_delay
         value="1"
   EvipRoutingSetup=ospfv3
      EvipParam=area
         value="10.4.35.1"
      EvipParam=area_type
         value="stub"
      EvipParam=dead_interval
         value="40"
      EvipParam=hello_interval
         value="10"
      EvipParam=local_address
         value="2dec::101:6:2/112"
      EvipParam=retransmit_interval
         value="5"
      EvipParam=router_id
         value="192.168.14.10"
      EvipParam=router_priority
         value="0"
      EvipParam=spf_delay
         value="500"
      EvipParam=spf_interval
         value="1000"
      EvipParam=transmit_delay
         value="1"
   EvipSupervisedRemoteGateway=192.168.77.99

Determine if the eVIP or external router configuration has been altered compared to a previously known working configuration. Contact the local responsible if needed and troubleshoot of the gateway router.

For recommendations and limitations, refer to Section Interworking Rules, Recommendations, and Limitations in eVIP Internetworking.

3.3 Next Level Support

If the source of the problem is still unknown, contact the next level of support.