vDicos, Diameter Link Failure

1	Introduction
1.1	Alarm Description
1.2	Prerequisites
2	Procedure
2.1	Analyzing Alarm
2.2	Actions for Format Error of CER/CEA Messages
2.3	Actions for Configuration Fault
2.4	Actions for Link Inactivity, IP Network Failure, and Connection Failure
2.5	Actions for System Error

1 Introduction

This instruction concerns alarm handling.

1.1 Alarm Description

The alarm is issued when a Diameter connection has failed.

The possible alarm causes and fault locations are explained in Table 1.

Table 1 Alarm Causes
Alarm Cause	Description	Fault Reason	Fault Location	Impact
A Diameter connection has failed.	Format error of Capabilities Exchange Request (CER)/Capabilities Exchange Answer (CEA) messages	The received message has not been coded according to the diameter standard for coding parameters	Examine the log in `/opt/cdclsv/storage/log/lpmsv` to determine fault reason	Missing connection between one or more Diameter peer nodes, which can decrease the throughput of messages between Diameter applications
		One or more Attribute-Value Pairs (AVPs) are missing
		A vendor-defined mandatory AVP is received and that AVP is not defined in the receiving node
	Configuration fault	The security configuration is not the same on both sides	Software configuration
		An application is not installed on both sides
		The number of connections is not equal on both sides
	Link inactivity	A connection to a Diameter peer is broken because of link inactivity without response to a watchdog message	Network interface
	IP network failure	IP network failure	IP network
		Socket failure	Socket
		Malformed message	Network
	System error	Internal	Software
	Connection failure	Network configuration fault	Misconfigured IP addresses or port numbers
	Connection failure	Temporary fault	Any of the Diameter peer nodes

Note:

The alarm is cleared automatically in the following situations

Related peer node is disabled
When a peer node is disabled, all vDicos Diameter Link Failure alarms on connections related to this peer node are cleared and a new vDicos, Diameter Peer Node Disabled alarm is raised.
Own node is disabled
When the own node is disabled, alarms for connections (and peer nodes) related to the own node are cleared and a new vDicos, Diameter Own Node Disabled alarm is raised for the own node.
A Diameter link is disabled
When a Diameter link for this connection is disabled, the alarm is cleared and a new vDicos, Diameter Link Disabled alarm is raised for the connection.

The alarm attributes are listed and explained in Table 2.

Table 2 Alarm Attributes
Attribute Name	Attribute Value
Major Type	193
Minor Type	2250572777
Managed Object Class	DIA-CFG-Conn
Managed Object Instance	`ManagedElement=<node_name>,<ManagedFunction>=<FunctionId>,DIA-CFG-Application=DIA,DIA-CFG-StackContainer=<stackId>,DIA-CFG-PeerNodeContainer=<stackId>,DIA-CFG-NeighbourNode=<hostId>#<stackId>,DIA-CFG-Conn=<stackId>#<hostId>#<connId>`
Specific Problem	vDicos, Diameter Link Failure
Event Type	communicationsAlarm (2)
Probable Cause	gsm1211LinkFailure (517)
Additional Text	Detailed Information: Configuration fault, IRP Cause: 517 Detailed Information: Link inactivity, IRP Cause: 517 Detailed Information: IP network failure, IRP Cause: 517 Detailed Information: Format error of CER/CEA messages, IRP Cause: 517 Detailed Information: System error, IRP Cause: 517 Detailed Information: Connection failure, IRP Cause: 517
Perceived Severity	major (4)

1.2 Prerequisites

This section provides information on the documents, tools, and conditions that apply to the procedure.

1.2.1 Documents

This instruction references the following documents:

1.2.2 Tools

No tools are required.

1.2.3 Conditions

Before starting this procedure, ensure that the following conditions are met:

A vDicos, Diameter Link Failure alarm is raised.
An Ericsson Command-Line Interface (ECLI) session in Exec mode is in progress.

2 Procedure

This section describes the procedure to follow when this alarm is received.

2.1 Analyzing Alarm

Select action according to alarm attribute additionalText, as follows:

If Format error of CER/CEA messages, proceed with Section 2.2 Actions for Format Error of CER/CEA Messages.
If Configuration fault, proceed with Section 2.3 Actions for Configuration Fault.
If Link inactivity, IP network failure, or Connection failure, proceed with Section 2.4 Actions for Link Inactivity, IP Network Failure, and Connection Failure.
If System error, proceed with Section 2.5 Actions for System Error.

2.2 Actions for Format Error of CER/CEA Messages

Do the following:

Check the status of the links to the peer indicated by the alarm, for example:

>dn ManagedElement=NODE06ST,XyzFunction=xyz,DIA-CFG-Application=DIA,DIA-CFG-StackContainer=abc,DIA-CFG-PeerNodeContainer=abc,DIA-CFG-NeighbourNode=node12.ericsson.com\23abc

(DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>show-table -m DIA-CFG-Conn -p connId,linkStatus

The following is an example output:

============================================
| connId                      | linkStatus |
============================================
| abc\23node12....com\23conn1 | Down       |
| abc\23node12....com\23conn2 | Up         |
============================================

Is at least one of the connections established for the peer node?
Yes: Continue with the next step.

No: Proceed with Step 9.
Navigate to the DIA-CFG-Conn Managed Object (MO) with link status Down, for example:
>dn ManagedElement=NODE06ST,XyzFunction=xyz,DIA-CFG-Application=DIA,DIA-CFG-StackContainer=abc,DIA-CFG-PeerNodeContainer=abc,DIA-CFG-NeighbourNode=node12.ericsson.com\23abc,DIA-CFG-Conn=abc\23node12.ericsson.com\conn1
Disable and re-enable the connection:
(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1>configure

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>enabled=false

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>commit -s

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>enabled=true

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>commit
Verify the setting:
(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>show enabled
enabled=true
Navigate up one step to the peer node:
(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>up

Check the status of the links again, for example:

(DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>show-table -m DIA-CFG-Conn -p connId,linkStatus

The following is an example output:

============================================
| connId                      | linkStatus |
============================================
| abc\23node12....com\23conn1 | Up         |
| abc\23node12....com\23conn2 | Up         |
============================================

Are there additional connections with link status Down?
Yes: Proceed with Step 3.

No: Proceed with Step 11.
Disable and re-enable the peer node, for example:
(DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>configure

(config-DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>enabled=false

(config-DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>commit -s

(config-DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>enabled=true

(config-DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>commit
Verify the setting:
(DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>show enabled
enabled=true

Check the link status of the connections:

(DIA-CFG-NeighbourNode=node12.ericsson.com\23abc)>show-table -m DIA-CFG-Conn -p connId,linkStatus

The following is an example output:

============================================
| connId                      | linkStatus |
============================================
| abc\23node12....com\23conn1 | Up         |
| abc\23node12....com\23conn2 | Up         |
============================================

Is the connection established?
Yes: Continue with the next step.

No: Proceed with Step 14.
Is the alarm cleared?
Yes: Proceed with Step 16.

No: Continue with the next step.
Perform data collection, refer to Data Collection Guideline.
Consult the next level of maintenance support. Further actions are outside the scope of this instruction.
Job is completed.

2.3 Actions for Configuration Fault

Do the following:

Check the value of attribute blockReason of the DIA-CFG-Conn MO indicated by the alarm, for example:
>dn ManagedElement=NODE06ST,XyzFunction=xyz,DIA-CFG-Application=DIA,DIA-CFG-StackContainer=abc,DIA-CFG-PeerNodeContainer=abc,DIA-CFG-NeighbourNode=node12.ericsson.com\23abc,DIA-CFG-Conn=abc\23node12.ericsson.com\conn1

(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>show blockReason

The following is an example output:
blockReason=3007(application is unsupported)
Note:
Possible reasons are 3007(application is unsupported), 5010(there is no common application), 3010(the peer is unknown), or 5012(not enough resources defined).
Contact the relevant organization (deployment organization or network administrator) and provide the result code.
Job is completed.

2.4 Actions for Link Inactivity, IP Network Failure, and Connection Failure

Do the following:

Check the link status of the DIA-CFG-Conn MO indicated by the alarm, for example:
>dn ManagedElement=NODE06ST,XyzFunction=xyz,DIA-CFG-Application=DIA,DIA-CFG-StackContainer=abc,DIA-CFG-PeerNodeContainer=abc,DIA-CFG-NeighbourNode=node12.ericsson.com\23abc,DIA-CFG-Conn=abc\23node12.ericsson.com\conn1

(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>show linkStatus

The following is an example output:
linkStatus=Down
Is the connection established?
Yes: Proceed with Step 8.

No: Continue with the next step.
Disable and re-enable the connection:
(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>configure

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>enabled=false

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>commit -s

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>enabled=true

(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>commit
Verify the setting:
(config-DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>show enabled
enabled=true
Wait for automatic reconnection.
Check the link status:
(DIA-CFG-Conn=abc\23node12.ericsson.com\conn1)>show linkStatus

The following is an example output:
linkStatus=Up
Is the connection established?
Yes: Continue with the next step.

No: Proceed with Step 9.
Is the alarm cleared?
Yes: Proceed with Step 14.

No: Proceed with Step 12.
Use tools ping and traceroute to check the connection to the peer node.
Can the peer be reached within 10 seconds?

Yes: Proceed with Step 12.

No: Continue with the next step.
Contact the network administrator about a possible network fault.
Proceed with Step 14.
Perform data collection, refer to Data Collection Guideline.
Consult the next level of maintenance support. Further actions are outside the scope of this instruction.
Job is completed.

2.5 Actions for System Error

Do the following:

Perform data collection, refer to Data Collection Guideline.
Consult the next level of maintenance support. Further actions are outside the scope of this instruction.
Job is completed.