Contents

1   Description

This instruction describes how to heal a Virtualized Network Function (VNF) in the VNF Lifecycle Management (VNF-LCM). The healing can be manual or automated triggered by alarms.

The VNF-LCM procedures use workflow instances. The following figure shows an example of a workflow instance, where workflow progress can be tracked in the Workflow Diagram view. The boxes in the Workflow Diagram only represent the stages of the various procedures; operations are performed in the Task view.

For more information about the VNF-LCM, see CSCF VNF Lifecycle Management.

2   Procedure

2.1   Heal a VNF

Prerequisites

• No tools are required.
• The following conditions must apply:
  • The VNF is instantiated and scaled out.
  • The VNF-LCM is available using either the Operations Support System for Radio and Core (OSS-RC) or the Ericsson Network Manager (ENM).
  • An Ericsson Command-Line Interface (ECLI) session in Exec mode is in progress (if a VNF is to be healed manually).

Steps

1. Select the appropriate action:
   • Heal the VNF automatically, triggered on the reception of the CLM Cluster Node Unavailable alarm from the VNF instance, proceed with Section 2.2 Heal a VNF Automatically.
   • Heal the VNF manually, proceed with Section 2.3 Heal a VNF Manually.

2.2   Heal a VNF Automatically

Prerequisites

• The VNF is onboarded using the VNF-LCM. During onboarding, an autostart-rule is specified in Section Prepare Workflow-Based VNF Operations in Onboard OpenStack Virtual Deployment Package on VNF-LCM, 68/1543-AVA 901 30/2 Uen.

Steps

1. Track the progress of the auto-heal VNF workflow in the Instance Activity view.

   The Heal VNF workflow consists of a forceful scale-in and a scale-out operation. The following three workflow instances are shown for the auto-heal VNF workflow in the Instance Activity view:

   • Heal VNF
   • Scale-in (Heal VNF)
   • Scale-out (Heal VNF)

   Note:  

   It is recommended to lower the node (VM) alarm time-out on the VNF instance from 15 minutes to 5 minutes to trigger the CLM Cluster Node Unavailable alarm, if a VM has lost contact with the remaining cluster members for more than 5 minutes. To lower the value of the node (VM) alarm time-out, use command cmw-node-alarm-timeout 300.

   The Heal VNF workflow can only heal the VNF if sufficient compute resource is available for OpenStack's Nova scheduler.

   Healing of non-scalable VMs (SC-1, SC-2, PL-3, PL-4) is not possible.

   
   
2. If the VNF to be healed was instantiated using the VNF-LCM and the Heal VNF workflow is to be started manually, then proceed with Section 2.3 Heal a VNF Manually.

2.3   Heal a VNF Manually

Steps

1. In the VNF-LCM Workflows view, select Heal VNF and then click Start a New Instance.

   

2. In the Start a Workflow view, fill out the Instance Name field and then click Submit.
3. Select the newly created workflow from the Instance Activity panel.
4. In the Workflow Instance view, select the VNF instance to be healed, and then click Submit.

   

5. In the Input additional parameters for workflow view, specify the Universally Unique Identifier (UUID) of the VM to be removed from the cluster, and then click Submit.

   

6. Get the UUID of unavailable or failed PL VM from the Ericsson Command-Line Interface (ECLI):

   >show -r ManagedElement=1,Equipment=1

The VNF instance is scaled-in and the specified VM is forcefully removed from the cluster. After this, the VNF instance is scaled-out, and a PL is added to the cluster.

2.4   Troubleshooting

If the workflow execution fails, inspect the relevant logs to identify the cause of the failure.

Steps

1. Increase the log level from INFO to DEBUG. For information on how to change log level, see VNF Lifecycle Manager System Administrator Guide, 1543-APR 901 0578.
2. Inspect the following logs to identify the cause of the failure:
   • Jboss Server log: /ericsson/3pp/jboss/standalone/log/server.log
   • System log: /var/log/messages
   • Workflow log: the Workflow Log view in the VNF-LCM
3. If the Workflow Log view reports Authentication failed, repair the Secure Shell (SSH) key between the VNF-LCM and the CSCF. See Section Check SSH Key for Authentication in CSCF Troubleshooting Guideline.
4. If a problem cannot be solved, consult the next level of maintenance support and provide the logs. Further actions are outside the scope of this instruction.