MTAS VNF Life Cycle Management Guide
MTAS

Contents

1Introduction

2
Prerequisites
2.1Hardware and Software

3
Onboarding
3.1Onboard VNF Package on VNF-LCM
3.2Onboard VNF Package on EO

4
Procedures
4.1Instantiate a VNF Using VNF-LCM
4.2Instantiate a VNF Using EO
4.3Scale-out a VNF
4.4Scale-In a VNF
4.5Heal a VNF
4.6Terminate a VNF Using VNF-LCM
4.7Terminate a VNF Using EO
4.8Upgrade a VNF

5
Logging

1   Introduction

This document describes system administration tasks performed in the VNF Life Cycle Manager (VNF-LCM).

The VNF-LCM provides a workflow execution environment and a web-based application for managing VNF life cycle procedures. The workflows can be executed through the Or-Vnfm interface, which is a REST API provided by VNF-LCM. The procedures are realized by executing ordered sequences of steps, called workflows, installed in the VNF-LCM. Each workflow must be provided with VNF-specific input parameters during execution.

The workflow manages virtual resources for the VNF by communicating with the Virtual Infrastructure Manager (VIM) and with the NVFO (Network Functions Virtualization Orcherstrator) in case of full stack.. For example, if Ericsson CEE is used, the HOT API is used at the Vi-Vnfm reference point.

The workflow calls LCM scripts to interact with the VNF for VNF-specific operations, such as configuration importing. The LCM scripts access the VNF using the SSH or the NETCONF Northbound Interface (NBI).

Figure 1 shows an overview of a full Ericsson Stack and also the difference to a small stack.

The workflows can be executed through the following:

Figure 1   VNF-LCM Overview

The following workflow-based life cycle management procedures can be performed:

The following limitations apply for workflows execution through the Or-Vnfm interface:

2   Prerequisites

This section describes the prerequisites which must be fulfilled before the MTAS can be installed.

2.1   Hardware and Software

The following hardware (virtual and physical), and software is required:

3   Onboarding

3.1   Onboard VNF Package on VNF-LCM

This section describes how to prepare for workflow-based VNF operations using VNF-LCM. Performing this procedure is a prerequisite for life cycle operations.

Note:  
For Openstack the onboarding process has changed to VNF package upload and unpack. For VMware it's still the old legacy way!

  1. Decompress the vMTAS workflow pack CXP9034815_1-<R-state>.tar.gz where <R-state> defines the revision state of the package.

    cd /home/cloud-user

    tar -zxvf CXP9034815_1-<R-state>.tar.gz

  2. Install the vIMS workflow bundle:
    1. Log on to vnflaf-services VM as cloud-user and switch to root user.
    2. Verify that the bundle is not already installed by running the List command.

      wfmgr bundle list

      If the version of vIMSWorkflows is older than the version included in the vMTAS Workflow Pack CXP9034815_1-<R-state>.tar.gz, install the newer version.

    3. Install the workflow bundle package by running the install command.

      wfmgr bundle install --package=\
      <workflow_bundle_rpm_file _path>

    For more information on workflow bundle installation in VNF-LCM, refer to ENM/OSS-RC documentation.

  3. OpenStack only onboarding steps:

    1. Decompress vMTAS VNF package <VNF package>.zip (which was created in MTAS SW Installation and uploaded to VFN-LCM):

      unzip /home/cloud-user/<VNF package>.zip -d /vnflcm-ext/backups/workflows/vnfd/

    2. Copy the VNF configuration files which are previously created. See (link to MTAS Installation):

      cd /vnflcm-ext/backups/workflows/vnfd/\
      <VNF package>/configurations/<VNF_1 configuration>

      cp -r /home/cloud-user/<VNF_1 configuration> .

      Note:  
      Each directory in configurations contains a VNF instance-specific environment file (env.yaml ), configuration files evip_cli.txt for eVIP, ss7.conf for SS7, and pdb_bundle.zip for application configuration. Each directory also contains id_rsa.pub public key of the VNF-LCM's jboss_user. The files evip_cli.txt, ss7.conf, and pdb_bundle.zip are optional.

    To use the same VNF package for instantiate another VNF (VNF2) the following steps are needed:

    • Create the VNF2 specific configuration files as It is described in MTAS Installation (section 3.3.1.6).
    • Create the configuration directory for VNF2:

      mkdir /vnflcm-ext/backups/workflows/vnfd/\
      <VNF package>/configurations/<VNF_2 configuration>

    • Copy the VNF2 specific configuration files to the configuration directory:

      cd /vnflcm-ext/backups/workflows/vnfd/\
      <VNF package>/configurations/<VNF_2 configuration>

      cp -r /home/cloud-user/<VNF_2 configuration> .

  4. VMware only onboarding steps:
    1. Decompress vMTAS LCM scripts CXP9034788_1-<R-state>.tar.gz where <R-state> defines the revision state of the package:

      mkdir /vnflcm-ext/backups/workflows/vnfd/\
      <VNFType__VNFVersion>

      cd /vnflcm-ext/backups/workflows/vnfd/\
      <VNFType__VNFVersion>

      tar -zxvf /home/cloud-user/CXP9034788_1-<R-state>.tar.gz

      Note:  
      Follow the naming convention: VNF type and VNF version separated by __. (Where "__" is a double underscore.)

    2. In /vnflcm-ext/backups/workflows/vnfd/<VNFType__VNFVersion>, create a configurations subdirectory with read permission for the jboss_user and a child directory for each VNF configuration (that is, VNF instance), and copy the VNF configuration to the child directory:

      mkdir /vnflcm-ext/backups/workflows/vnfd/\
      <VNFType__VNFVersion>/configurations

      mkdir /vnflcm-ext/backups/workflows/vnfd/\
      <VNFType__VNFVersion>/configurations/<VNF_1 configuration>

      cd /vnflcm-ext/backups/workflows/vnfd/\
      <VNFType__VNFVersion>/configurations/<VNF_1 configuration>

      cp -r /home/cloud-user/<VNF_1 configuration> .

      Note:  
      Each directory in configurations contains a VNF instance-specific environment file (env-vcd.yaml ), configuration files evip_cli.txt for eVIP, ss7.conf for SS7, and pdb_bundle.zip for application configuration. Each directory also contains id_rsa.pub public key of the VNF-LCM's jboss_user. The files evip_cli.txt, ss7.conf, and pdb_bundle.zip are optional.

  5. If the SSH key is not available yet for jboss, create it using the following commands:

    su jboss_user

    ssh-keygen -t rsa

    exit

    Note:  
    The use of encrypted private keys is not supported in the current release (that is, do not use passphrase).

  6. Copy the public SSH key into the configuration directory of VNF instance:

    cp /home/jboss_user/.ssh/id_rsa.pub /vnflcm-ext/backups/workflows/vnfd/<VNF package>\
    /configurations/<VNF_1 configuration>/

    Note:  
    The public key must be added in the configuration directory for each instance.

  7. Verify the structure of the /vnflcm-ext/backups/workflows/vnfd/<VNF package> directory:

    In OpenStack based cloud environment:

    `-- vMTAS-<version>-falvorId
     |-- configurations
     |        |-- <VNF_1 configuration>
     |        |       |-- env.yaml
     |        |       |-- evip_cli.txt*
     |        |       |-- ss7.conf*
     |        |       |-- instance1_pdb_bundle.zip*
     |        |       `-- id_rsa.pub
     |        `-- <VNF_2 configuration>
     |                |-- env.yaml
     |                |-- evip_cli.txt*
     |                |-- ss7.conf*
     |                |-- instance1_pdb_bundle.zip*
     |                `-- id_rsa.pub
     |`-- mtas_hot.yaml
     |-- Resources
     |        |-- HotFiles
     |        |       `-- mtas_hot_scaling.yaml
     |        |-- LcmScripts
     |        |       |-- common.py
     |        |       |-- parsers.py
     |        |       |-- post_instantiation.py
     |        |       |-- post_scale_in.py
     |        |       |-- post_scale_out.py
     |        |       |-- pre_heal.py
     |        |       |-- pre_instantiation.py
     |        |       |-- pre_scale_in.py
     |        |       |-- pre_scale_out.py
     |        |       |-- pre_termination.py
     |        |       |-- pre_upgrade.py
     |        |       `-- VERSION
     `-- VnfdWrapperFiles
             |-- VNFD_Wrapper.json
             `-- vnfLcmOperationsConfiguration.json

    In VMware based cloud environment: 

    `-- vMTAS__<x.y> 
     |-- vnfLcmOperationsConfiguration.json 
     |-- configurations 
     |   |-- <VNF_1 configuration> 
     |   |   |-- env-vcd.yaml 
     |   |   |-- id_rsa.pub 
     |   |   |-- evip_cli.txt* 
     |   |   |-- ss7.conf* 
     |   |   `-- instance1_pdb_bundle.zip* 
     |   `-- <VNF_2 configuration> 
     |       |-- env-vcd.yaml 
     |       |-- id_rsa.pub 
     |       |-- evip_cli.txt* 
     |       |-- ss7.conf* 
     |       `-- instance1_pdb_bundle.zip* 
     `-- lcmScripts

    The files marked with * are optional.

  8. Only for Openstack based cloud environment: If automatic invocation of Heal VNF WF (so called auto-heal WF) is to be enabled:
    Note:  
    For CEE deployments, this step is not necessarily needed since CEE's CM-HA feature can be primary used to Heal a VNF.

    1. Verify the presence of file vIMS-autostart-rules.xml in services VM of VNF-LCM:

      ll /vnflcm-ext/current/workflows/auto-start-rules/vIMS-autostart-rules.xml

    2. If the file is not present, copy the template autostart-rule file to the above mentioned directory and remove all existing template workflowTriggerRule:

      sed '24,45d' /opt/ericsson/ERICvIMSWorkflowsworkflows/autostart-rules/vIMS-autostart-rules.xml \

      > /vnflcm-ext/current/workflows/auto-start-rules/vIMS-autostart-rules.xml

      Note:  
      It can be needed to verify that the first template workflowTriggerRule starts in line 24 and the last workflowTriggerRule ends in line 45.

    3. The file /vnflcm-ext/current/workflows/auto-start-rules/vIMS-autostart-rules.xml must contain the following workflowTriggerRule: 
      <wft:workflowTriggerRule name="vMTAS-AutoHealing" id="vMTAS-AutoHealing" type="FM_ALARM">
             <wft:triggerCondition>
                     <wft:attributeName>specificProblem</wft:attributeName>
                     <wft:triggerConditionOperator>EQUALS</wft:triggerConditionOperator>
                     <wft:attributeValue>COM SA, CLM Cluster Node Unavailable</wft:attributeValue>
             </wft:triggerCondition>
             <wft:triggerAction WorkflowDefinitionName="Heal VNF"
                     WorkflowBundleName="vIMSWorkflows">
                     <wft:param />
             </wft:triggerAction>
     </wft:workflowTriggerRule>
    4. Verify that the content of /vnflcm-ext/current/workflows/auto-start-rules/vIMS-autostart-rules.xml is similar to the following example: (it can contain additional workflowTriggerRule):

    Note:  
    If it is needed to enable auto-healing only for a particular VNF instance, the workflowTriggerRule can be refined by adding additional triggerCondition(s) to it.

3.2   Onboard VNF Package on EO

Note:  
This section is only applicable for full Ericsson stack deployments

This section describes how to prepare for workflow-based VNF operations using EO.

Performing this procedure is a prerequisite for life cycle operations.

Prerequisites

Steps

  1. In EO, click Resources > Packages.
  2. On the opening Upload VNF Package view, in New Package File, add the <VNF Package>.zip file which is created in MTAS Installation (link!).
  3. In the New Configuration File new, upload the Resources/VnfdWrapperFiles/VNFD_Wrapper.json file.

    The content of the VNFD_Wrapper.json file is now displayed in the New Configuration File Content field.

  4. Copy the value of parameter vnfdId from the configuration file to the New Package Name field.
  5. Fill in the following mandatory fields:
    • Package Version
    • Package Vendor
    • Software Version
    • Package Category
    • Description
  6. In the Managers field, select the used VNF-LCM and click Start.
    Result:
    The onboarded package is now visible in the EO under Resources > Packages.

4   Procedures

The following sections describe how to perform LCM operations.

VNF-LCM procedures use workflow instances. Figure 2 shows the example of a workflow instance, where workflow progress can be tracked in the Workflow Diagram view. The Workflow Diagram only represents stages of the various procedures, operations are performed in the Task view.

Figure 2   Workflow Instance Overview

4.1   Instantiate a VNF Using VNF-LCM

This section describes how to instantiate a VNF using VNF-LCM.

To instantiate a VNF:

  1. Navigate to the VNF-LCM Workflows > Instantiate VNF > Start a New Instance. See Figure 3.

Figure 3   Workflows

  1. In Start a Workflow, fill out the Instance Name, and click Submit.
  2. Select the newly created workflow from the Instance Activity panel.
  3. On the Workflow Instance screen, add VNF Name ,VNF Instance Description, and Select VNF descriptor id, and click Submit. See example in Figure 4.

    Select the Add Network Element in ENM/OSS-RC check box to add the new VNF in the network management application.

    Note:  
    The Select VNF descriptor id* displays VNF releases available for instantiation in the /vnflcm-ext/backups/workflows/vnfd/ directory.

Figure 4   Instantiate VNF

  1. In Select VIM, choose Select VIM to be used, and click Submit. See example in Figure 5.

Figure 5   Select VIM

  1. In Select Tenant, choose Select Tenant to be used, and click Submit. See example in Figure 6.

Figure 6   Select Tenant

  1. On the Select VDC screen, select the virtual Data Center (vDC) in which the new VNF is instantiated, and clickSubmit. See example in Figure 7.

Figure 7  

  1. On the Get Instance Configuration screen, choose Select Configuration for the VNF instance*, and click Submit. See example in Figure 8.
    Note:  
    The Select Configuration for the VNF instance* displays VNF configurations available for instantiation in the /vnflcmext/backups/workflows/vnfd/<VNF package>/configurations directory.

Figure 8   Get Instance Configuration

  1. If the ENM/OSS-RC network management application is used, provide VNF instance-specific parameters for ENM/OSS-RC topology update, and click Submit. See example in Figure 9.
    Note:  
    To fill out the Network element version supported by OSS-RC/ENM *, check the supported VNF version on OSS-RC/ENM. For example: on ENM, the supported VNF version can be checked with the following command: cmedit describe --netype <VNF_type>

Figure 9   Enter the Parameters Required by OSS/ENM

Note:  
If the installation fails, remove the VNF instance using the Terminate VNF WF (FORCEFUL), see Section 4.6 Terminate a VNF Using VNF-LCM. If the user cannot select the VNF instance in Step 4 in Section 4.6 of the Terminate VNF workflow, then the VNF instance must be deleted manually from the cloud environment.

4.2   Instantiate a VNF Using EO

Note:  
It is not recommended to perform any life cycle operation using VNF-LCM on a VNF that was instantiated using EO.

Steps

  1. In EO, start a new VAPP, click Resources > Packages. Select the package to deploy, and click Deploy.
  2. On the Search for Offers view, select values for the mandatory fields, and click Next.
  3. On the Choose Offer view, select the VNF package to be used for instantiation, and click Next.
  4. On the Set VIM Criteria view, click Next.
  5. On the Choose VIM Zone view, select the desired VIM zone, and click Next.
  6. On the Enter Attributes view, provide values for the fields of the VAPP Attributes tab:
    • Virtual Application Name, which is also the name of the Heat stack
    • VNF Manager - The VNF Manager chosen during onboarding is pre-selected.
    • Flavor - The virtual hardware configuration of the VNF, for example: number of vCPU and amount of memory.
      Note:  
      The value provided for Flavor is not used at this point, but as it is a mandatory field, a dummy value must be provided.

  7. On the Enter Attributes view, provide the VNFD_Wrapper.json file on the Configuration tab. It is possible to overwrite the VNFD_Wrapper.json file provided during onboarding either by uploading an updated file or by modifying the attribute values on the Configuration tab.
  8. Click Submit Order to finalize instantiation.
    Result:
    On the opening window, click View Asset List to check the status of the instantiation order. EO triggers an instantiation workflow in VNF-LCM, and the VNF becomes visible in the used VNFM.

4.3   Scale-out a VNF

This section describes how to scale-out a VNF using VNF-LCM.

Continue with this procedure only if the VNF to be scaled-out is instantiated using the VNF-LCM.

To scale-out a VNF:

  1. In the VNF-LCM, click Start a Workflow > Scale VNF > Start a New Instance. See Figure 3.
  1. In Start a Workflow, fill out Instance Name, and click Submit.
  2. Select the newly created workflow from the Instance Activity.
  3. In Workflow Instance, select the VNF instance to be scaled out, select scaling type (Scale Out), specify the number of VMs to be added to the VNF, and click Submit. See example in Figure 10.

Figure 10   Collect User Data for Scale-Out

The VNF instance is scaled-out, new VMs are added to the cluster.

4.4   Scale-In a VNF

This section describes how to scale-in a VNF using VNF-LCM.

Continue with this procedure only if the VNF to be scaled-in was instantiated using the VNF-LCM.

Prerequisites

To gracefully scale-in a node, the system status must meet the following conditions:

ManagedElement=1,MtasFunction=MtasFunction,MtasSupportFunctions=0,
CarSelApplication=CarrierSelect,CarSelDialedStringAnalysisTable=0,
carSelDialedStringAnalysisTableSynchronization 
ManagedElement=1,MtasFunction=MtasFunction,MtasSupportFunctions=0,
CarSelApplication=CarrierSelect,CarSelCarrierTable=0,
carSelCarrierTableSynchronization 
ManagedElement=1,MtasFunction=MtasFunction,MtasSupportFunctions=0,
NumAnaApplication=NumberAnalysis,NumAnaLocalCallTable=0,
numAnaLocalCallTableSynchronization 
ManagedElement=1,MtasFunction=MtasFunction,MtasSupportFunctions=0,
NumberNormalisation=NumberNormalisation,
numberNormalisationTableSync

For more information, refer to MTAS Troubleshooting Guideline.

To scale-in VNF:

  1. In the VNF-LCM, click Start a Workflow > Scale VNF >Start a New Instance. See Figure 3.
  1. In Start a Workflow, fill out Instance Name, and click Submit.
  2. Select the newly created workflow from the Instance Activity panel.
  3. In Workflow Instance, select the VNF instance to be scaled in, select scaling type (Scale In), specify Number of VMs to Scale-In*, and click Submit. See example in Figure 11.

Figure 11   Collect User Data for Scale-In

The VNF instance is scaled-in, the specified number of VMs is removed from the cluster.

  1. On the Collect extra parameters screen, select Scale-in type and optionally specify the list of VM UUIDs to be Scale-in with, and click Submit. See example in Figure 12.
    Note:  
    FORCEFUL Scale-in is a disruptive action therefore it is only to be used for Scale-in unavailable/failed PL(s). (The virtual resources reserved for the PL(s) are freed up first and then the MTAS cluster is "cleaned".)

Figure 12   Collect Extra Parameters

The VNF instance is scaled-in, the specified number of PLs is removed from the cluster.

Note:  
If any UUID was specified in Step 5, the VMs with the specified UUIDs are removed, otherwise VMs with the highest index in their name in OpenStack are removed.

4.5   Heal a VNF

This section describes how to heal a VNF from compute resource or network resource (for instance Neutron port) failure using VNF-LCM.

Heal VNF WF can be started two ways:

To enable the use of auto-heal VNF WF, set up an autostart-rule as it is specified in Step 9 of Onboarding.

The progress of auto-heal VNF WF can be tracked on the Instance Activity panel of VNF-LCM. The Heal VNF WF consists a FORCEFUL Scale-in and a Scale-out operation so that three WF instances are visible for the auto-heal VNF WF (Heal VNF, Scale-in (Heal VNF) and Scale-out (Heal VNF)) on the Instance Activity panel.

Note:  
  • It is recommended to lower the node (VM) alarm timeout on the VNF instance from 15 minutes to 5 minutes in order to trigger the CLM Cluster Node Unavailable alarm, if a VM has lost contact with the remaining cluster members for more than 5 minutes. To lower the value of the node (VM) alarm timeout, use command: cmw-node-alarm-timeout 300
  • The Heal VNF WF can only heal the VNF if sufficient compute resource is available for OpenStack's Nova scheduler.
  • Healing of non-scalable VMs (SC-1, SC-2, PL-3, PL-4) is not possible.

Continue with this procedure only if the VNF to be healed was instantiated using the VNF-LCM and the Heal VNF WF shall be started manually.

To heal the VNF manually:

  1. In the VNF-LCM Workflows, select Heal VNF, and click Start a New Instance. See Figure 3.
  1. In Start a Workflow, fill out Instance Name, and click Submit.
  2. Select the newly created workflow from the Instance Activity panel.
  3. In Workflow Instance , choose Select VNF instance*, and click Submit. See example in Figure 13.

Figure 13   Collect User Data for Healing

  1. In Input additional parameters for workflow, specify UUID of VM to be removed from the cluster, and click Submit. See example in Figure 14.

Figure 14   Collect Extra Parameters

  1. Get UUID of unavailable/failed PL(VM) from ECLI using command:
    show -r ManagedElement=1,Equipment=1

The VNF instance is scaled-in, the specified VM is forcefully removed from the cluster. After this, the VNF instance is scaled-out, and a PL is added to the cluster.

4.6   Terminate a VNF Using VNF-LCM

This section describes how to terminate a VNF using VNF-LCM.

Continue with this procedure only if the VNF to be terminated is instantiated using the VNF-LCM.

Steps

  1. In the VNF-LCM Workflows, select Terminate VNF, and click Start a New Instance. See Figure 3.
  1. In Start a Workflow, fill out the Instance Name field, and click Submit.
  2. Select the newly created workflow from the Instance Activity panel.
  3. On the Workflow Instance screen, choose Select VNF instance and click Submit. See example in Figure 15.

Figure 15   Collect User Data for Terminate

Graceful The VNF instance is gracefully locked (by setting mtasFunctionAdministrativeState to SHUTTING DOWN(2)), it gradually stops processing traffic. The VNF is terminated after the expiration of the graceful termination period or even earlier when all ongoing sessions have stopped on the node.
Forceful The VNF is terminated immediately, all ongoing traffic is lost. This option must be confirmed on the next screen, as it stops all traffic.
Graceful termination timeout (sec) The graceful termination timeout value defines after how many seconds the VNF is terminated when graceful termination has been applied but there is still ongoing traffic. Default value: -1, meaning that there is no graceful termination period, that is, the VNF is terminated only after the VNF stopped processing traffic.

The VNF instance is terminated. In case of Graceful termination, the VNF instance stops processing traffic gracefully and then is terminated.

4.7   Terminate a VNF Using EO

Steps

  1. In EO, select the VAPP to terminate and click Delete.
  2. On the Terminate VAPP form, provide termination options, and click Terminate.

    The following termination options are available:

    • Graceful

      The VMs in the cluster are gracefully locked, the VNF instance gradually stops processing traffic. The VNF is terminated after the expiration of the graceful termination period.

    • Forceful

      The VNF is terminated immediately, all ongoing traffic is lost. This option must be confirmed on the next view, as it stops all traffic.

    • Graceful termination timeout (sec)

      The graceful termination time-out value defines after how many seconds the VNF is terminated when graceful termination has been applied but there is still ongoing traffic. Default value: -1, meaning that there is no graceful termination period, that is, the VNF is terminated only after all VMs stopped processing traffic.

    Note:  
    Do not modify configuration data in the Configuration Parameters field.

  3. Click Terminate.
    Result:
    The VMs in the cluster are terminated with the method selected in Step 2, the VNF instance stops processing traffic, and is terminated.

4.8   Upgrade a VNF

The Upgrade VNF WF is not supported by the current release.

5   Logging

In case of an unsuccessful workflow execution, find more information on the cause of the failure:

Legal | © Ericsson AB 2017, 2018