Contents

1   Introduction

This document describes the License Management, Key File Fault alarm and provides instructions for fault management.

1.1   Alarm Description

License Management, Key File Fault is raised when License Manager (LM) transitions to Locked mode. This is a critical situation that may prevent the Managed Element from using licensed features and functionality.

Locked mode is initiated at the end of the 24 hour Autonomous mode period in response to one of the following scenarios:

• The Sentinel RMS license server is unreachable.
• Any ELIM formatted license key file is missing or corrupted.

  Note:  

  In an ELIM deployment with multiple license key files, this alarm is raised in response to one or more missing or corrupted license key files.

  
  
• Arwa is unreachable.

This primary alarm is issued by the ManagedElement=1,SystemFunctions=1,Lm=1 Managed Object (MO).

Possible causes and fault locations are explained in Table 1.

Table 1    Alarm Causes

Alarm Cause	Description	Fault Reason	Fault Location	Impact
159	License Management, Key File Fault	One of the following scenarios remains in effect for more than 24 hours: The Sentinel RMS license server is unreachable. Any ELIM formatted license key file is missing or corrupted. Arwa is unreachable.	LM Server	No license handling.

The following consequences are expected if the alarm condition is not resolved:

• LM operates in Locked mode. During Locked mode, licensed features and capacities are blocked by LM until access to a Sentinel RMS license server, all ELIM formatted license key files, or Arwa is restored.

Note:  

The License Management, Key File Fault alarm can appear as a result of maintenance activity.

The alarm attributes are listed and explained in Table 2.

Table 2    Alarm Attributes

Attribute Name	Attribute Value
activeSeverity	CRITICAL
additionalInfo	N/A
additionalText	"Key file fault in Managed Element"
eventType	QUALITYOFSERVICEALARM
lastEventTime	A timestamp of the last alarm update, such as an alarm status change or severity change.
majorType	193
minorType	393221
originalAdditionalText	Content of the additionalText field when the alarm was raised.
originalEventTime	Timestamp when the alarm was raised.
originalSeverity	Value of the activeSeverity level when the alarm was raised.
probableCause	159 (Configuration or Customization Error)
sequenceNumber	The notification identity for this object instance. It is not the same as the fmAlarmId since multiple notifications may be sent for one alarm instance. This value changes for every notification (such as a severity change, alarm clear, and so on).
source	ManagedElement=1,SystemFunctions=1,Lm=1
specificProblem	License Management, Key File Fault

1.2   Prerequisites

This section lists the prerequisite documents, tools, and conditions for the alarm handling procedure.

1.2.1   Documents

Review the following documents before starting the procedure:

• Personal Health and Safety Information (Reference [1])
• System Safety Information (Reference [2])
• "Configuration Management Using CLI" in the COM Management Guide for your version of the product software.

1.2.2   Tools

Ensure that the following tools are available before starting the procedure:

• Element Management System (for example: OSS)
• COM CLI

For more information on these tools, refer to the documentation for your version of the product software.

1.2.3   Conditions

Ensure that the following conditions are met before starting the procedure:

• Verify that no ongoing maintenance activities are affecting the network or network elements.
• If applicable, know the IP address(es) and port number(s) of the Sentinel RMS license server.
• If applicable, know the Arwa license server address.
• Know the IP address of the blade where the active COM CLI is running.
• Have the proper authority to handle configuration management of the network elements.
• Be familiar with basic UNIX commands.

2   Procedure

This section describes the alarm handling procedure.

2.1   Analyzing the Alarm

License Management, Key File Fault is a persistent alarm that remains on the alarm list while LM is operating in Locked mode. The alarm will clear automatically when access to a Sentinel RMS license server, all ELIM formatted license key files, or Arwa is restored.

To better determine the fault location and cause:

1. Check if there are any other active network or network element related alarms.
2. For LM deployments using Sentinel RMS, check the IP address(es) and port number(s) of the Sentinel RMS license server in the Lm.referenceToLicenseServer attribute.
3. For LM deployments using ELIM or Arwa No Connection, check the KeyFileInformation.locatable attribute for each license key file.
4. For LM deployments using Arwa, check that the following Arwa connection requirements have been met:
   • A valid Arwa license server address is configured in the SentinelCloudRuntime.properties file.
   • A secure tunnel connection (such as Remote Support Gateway or Customer Access and Security) is established between the nodes running LM and the Arwa license server.
   • A valid Arwa Domain Name System (DNS) hostname entry is in the cluster configuration.
5. For LM deployments using an Arwa Intermittent Connection, check the licenseServerLastSuccessfulConnectTime attribute for the time of the last successful connection to Arwa.

2.2   Actions for Sentinel RMS Configuration Issues

IP addresses and port numbers of the Sentinel RMS license server are part of the LM configuration. A faulty configuration can lead to connectivity issues.

To correct issues with the Sentinel RMS license server configuration and clear the alarm:

1. From a terminal window, use ssh to connect to the COM CLI Management System server port (default 22) on the blade where the active COM CLI is running:

   ssh <username>@<blade_IP_address> -p 22 -t -s cli

   The COM CLI opens after supplying your password.

2. Verify that the Lm.referenceToLicenseServer parameter points to the correct license server host address(es) and port number(s):

   show verbose ManagedElement=1, SystemFunctions=1, Lm=1, referenceToLicenseServer

3. If required, update the IP address(es) and port number(s) of the Sentinel RMS license server.

   Execute the following commands at the COM CLI:

   configure

   ManagedElement=1, SystemFunctions=1, Lm=1

   referenceToLicenseServer=[<address_values>]

   Where <address_values> is a comma-separated list of "<FQDN_or_IP_Address>:<Port_Number>" pairs, one pair per server.

   For example:

   referenceToLicenseServer=["SC-1:5093","SC-2:5093"]

   After updating referenceToLicenseServer:

   commit

4. After verifying the configuration, check connectivity by triggering a refresh of the license inventory:

   ManagedElement=1, SystemFunctions=1, Lm=1, refreshLicenseInventory

   The system returns true if the action was executed successfully.

5. Verify that the license inventory has been synchronized with a Sentinel RMS license server by checking the lastLicenseInventoryRefresh timestamp:

   show ManagedElement=1, SystemFunctions=1, Lm=1, lastLicenseInventoryRefresh

   A recent timestamp indicates a successful update.

6. Check the alarm status.

   If the alarm is still active, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

   Note:  

   Emergency Unlock can be used to temporarily restore access to licensed functionality while the system is in locked mode. For more information on Emergency Unlock, refer to the LM User Guide for Sentinel RMS (Reference [3]).

   
   

2.3   Actions for License Key File Issues

Note:  

This procedure only applies to ELIM and Arwa No Connection deployments.

ELIM license key files are stored on the cluster's persistent storage path. Issues accessing persistent storage or a problem locating a license key file can force the License Manager into Locked mode.

To correct issues with ELIM license key files and clear the alarm:

1. From a terminal window, use ssh to connect to a System Controller (SC) blade as root.
2. At the command prompt, check connectivity with the persistent storage path:

   ls –l /storage/system/software/lm-apr9010503

   If persistent storage is inaccessible, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

3. Use ssh to connect to the COM CLI Management System server port (default 22) on the blade where the active COM CLI is running:

   ssh <username>@<blade_IP_address> -p 22 -t -s cli

   The COM CLI opens after supplying your password.

4. Verify that all ELIM license key files are locatable:

   show all ManagedElement=1, SystemFunctions=1, Lm=1, KeyFileManagement=1

   The contents of KeyFileManagement=1 are printed.

5. Verify the locatable attribute for each KeyFileInformation class.

   If one ore more license key files are missing or corrupted, LM will report locatable=false.

   For example:

   KeyFileManagement=1
      reportProgress
         actionId=0
         actionName="loadLicKeyFile"
         progressInfo=""
         progressPercentage=100
         result=SUCCESS
         resultInfo="Successfully loaded the new LKF"
         state=FINISHED
         timeActionCompleted="2014-05-13T14:12:34"
         timeActionStarted="2014-05-13T14:12:34"
         timeOfLastStatusUpdate="2014-05-13T14:12:34"
      KeyFileInformation=1
         installationTime="2014-05-13T14:12:34"
         locatable=true
         productType="SSR 8000"
      KeyFileInformation=2
         installationTime="2014-05-13T14:11:35"
         locatable=false
         productType="SASN"
      KeyFileInformation=3
         installationTime="2014-05-13T14:12:15"
         locatable=true
         productType="EDA 1500"
   

6. For each license key file with locatable=false, restore the files from a backup location to the correct storage path.

   Note:  

   Each license key file is stored in a hashed subdirectory. The path to the hashed directory contains the productType of the license key file.

   For example:

   /storage/system/software/lm-apr9010503/SSR\ 8000/8887563311a276a54cba15d6359a7f8c

   
   

   LM will detect the restored file within 1 minute.

7. If backup files are not available, order replacement license key files from the Ericsson software supply organization and have them installed.

   For more information on installing ELIM license key files, refer to "Installing License Key Files" in the LM User Guide for ELIM (Reference [4]).

   Note:  

   Emergency Unlock can be used to temporarily restore access to licensed functionality while the system is in locked mode. For more information on Emergency Unlock, refer to the LM User Guide for ELIM (Reference [4]).

   
   
8. Check the alarm status.

   If the alarm is still active, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

2.4   Actions for Arwa Connection Issues

Note:  

This procedure is not applicable to LM deployments using Arwa No Connection. When deployed in Arwa No Connection mode, LM uses ELIM formatted license key files to store license information locally. For more information on correcting license key file issues, refer to Section 2.3.

An Arwa connection requires the following:

• A valid Arwa license server address configured in the SentinelCloudRuntime.properties file.
• A secure tunnel connection (such as Remote Support Gateway or Customer Access and Security) between the nodes running LM and the Arwa license server.
• An Arwa Domain Name System (DNS) hostname entry in the cluster configuration.

Issues with any of these items can prevent the License Manager from connecting to Arwa and force it into Autonomous mode.

To correct issues with the Arwa connection and clear the alarm:

1. Ensure that a secure tunnel to the Arwa license server (such as Remote Support Gateway or Customer Access and Security) is established.

   If a secure tunnel is not available, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

2. From a terminal window, use ssh to connect to a SC blade as root.
3. Check that a valid Arwa DNS hostname entry is in the cluster configuration.

   If the Arwa DNS hostname is missing from the cluser configuration or invalid, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

4. At the command prompt, check connectivity with the SentinelCloudRuntime.properties file:

   ls -l /storage/system/software/lm-apr9010503/SentinelCloudRuntime.properties

   If SentinelCloudRuntime.properties is inaccessible, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

5. Verify that the SentinelCloudRuntime.properties file points to the correct Arwa license server address:

   grep YPSAddress /storage/system/software/lm-apr9010503/SentinelCloudRuntime.properties

6. If required, update the Arwa license server address:

   sed 's,https://<current_address>/YPServer,https://<new_address>/YPServer,g' -i /storage/system/software/lm-apr9010503/SentinelCloudRuntime.properties

7. Verify that the correct deployment type is set in the SentinelCloudRuntime.properties file:

   grep DeploymentType /storage/system/software/lm-apr9010503/SentinelCloudRuntime.properties

   The Arwa deployment type can have the following values:

   Cloud		Designates Arwa Permanent Connection mode.
   OnPremise		Designates Arwa Intermittent Connection mode.

8. If required, update the Arwa deployment type:

   sed 's,"DeploymentType" value="<current_value>","DeploymentType" value="<new_value>",g' -i /storage/system/software/lm-apr9010503/SentinelCloudRuntime.properties

9. After verifying the configuration, use ssh to connect to the COM CLI Management System server port (default 22) on the blade where the active COM CLI is running:

   ssh <username>@<blade_IP_address> -p 22 -t -s cli

   The COM CLI opens after supplying your password.

10. Check connectivity with Arwa.

    For Arwa Permanent Connection:

    1. Trigger a refresh of the license inventory:

       ManagedElement=1, SystemFunctions=1, Lm=1, refreshLicenseInventory

       The system returns true if the action was executed successfully.

    2. Verify that the license inventory has been synchronized with Arwa by checking the lastLicenseInventoryRefresh timestamp:

       show ManagedElement=1, SystemFunctions=1, Lm=1, lastLicenseInventoryRefresh

       A recent timestamp indicates a successful update.

    For Arwa Intermittent Connection:

    1. Trigger a manual connection to Arwa:

       ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, connectToArwa

       The system returns true if the action was executed successfully.

    2. Check the result of the connectToArwa operation:

       show ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, reportProgress, result

       result=SUCCESS indicates that the connection to Arwa was successful.

    3. If result=FAILURE, check the connection status to determine the cause:

       show ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, reportProgress

11. Check the alarm status.

    If the alarm is still active, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

    Note:  

    Emergency Unlock can be used to temporarily restore access to licensed functionality while the system is in locked mode. For more information on Emergency Unlock, refer to the LM User Guide for Arwa (Reference [5]).

    
    

2.5   Actions for Intermittent Connection Issues

When operating in Intermittent Connection mode, LM only connects to Arwa manually or at regular intervals of 80 days. If the last connection attempt failed, LM transitions to Autonomous mode.

To manually reconnect to Arwa and clear the alarm:

1. Ensure that a secure tunnel to the Arwa license server (such as Remote Support Gateway or Customer Access and Security) is established.

   If a secure tunnel is not available, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

2. Use ssh to connect to the COM CLI Management System server port (default 22) on the blade where the active COM CLI is running:

   ssh <username>@<blade_IP_address> -p 22 -t -s cli

   The COM CLI opens after supplying your password.

3. Trigger a manual connection to Arwa:

   ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, connectToArwa

   The system returns true if the action was executed successfully.

4. Check the result of the connectToArwa operation:

   show ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, reportProgress, result

   result=SUCCESS indicates that the connection to Arwa was successful.

5. If result=FAILURE, check the connection status to determine the cause:

   show ManagedElement=1, SystemFunctions=1, Lm=1, ArwaConfiguration=1, reportProgress

6. Check the alarm status.

   If the alarm is still active, consult the next level of maintenance support. Further actions are outside the scope of this instruction.

   Note:  

   Emergency Unlock can be used to temporarily restore access to licensed functionality while the system is in locked mode. For more information on Emergency Unlock, refer to the LM User Guide for Arwa (Reference [5]).

   
   

Reference List

[1] Personal Health and Safety Information, 12446-2885 Uen
[2] System Safety Information, 12446-2886 Uen
[3] LM User Guide for Sentinel RMS, 1/1553-APR 901 0503/5 Uen
[4] LM User Guide for ELIM, 2/1553-APR 901 0503/5 Uen
[5] LM User Guide for Arwa, 3/1553-APR 901 0503/5 Uen