Configure Scale-Out

Ericsson Service-Aware Policy Controller

1	Configure Scale-Out Introduction
1.1	Configure Scale-Out Prerequisites
2	Configure Scale-Out Procedure
3	VM Creation using ECM
4	VM Creation using Atlas/Director
5	VM Creation using VMware vSphere Web Client
6	VM Creation in using VMware vCloud Director

1 Configure Scale-Out Introduction

This document describes how to perform scale-out operation, which adds one Virtual Machine (VM), acting as PayLoad (PL), to the SAPC vApp.

1.1 Configure Scale-Out Prerequisites

This section describes the prerequisites, which must be fulfilled before using the procedure.

1.1.1 Conditions

The following conditions must apply:

Other upgrade operations are not ongoing, for example, software upgrade.
An Ericsson Command-Line Interface (ECLI) session in Exec mode is in progress.
For the specific case of deployments over OpenStack based NFV Infrastructure (NFVI) using ECM (Ericsson Cloud Manager), Atlas or Director, the collocation of the new scaled-out PL with the SAPC VMs already deployed, depends on the deployment type defined in the SAPC.cfg configuration file:
- compact: the scaled PLs belong to the same anti-affinity group that the PLs of the initial deployment. Therefore, they cannot be deployed together in the same physical host.
- single_box: the scaled PLs belong to the same affinity group that the remaining VMs of the SAPC. Therefore, all VMs are deployed together in the same physical host.
- multiple_host: the scaled PLs belong to the same anti-affinity group that the remaining VMs of the SAPC. Therefore, every VM is deployed in a different physical host.
- custom: user-defined affinity rules, There are some limitations, depending on the cloud environment chosen for deployment.
  Note:
  Additional information about the SAPC.cfg configuration file can be found on the SAPC VNF Descriptor Generator Tool.
For the specific case of deployments using ECM, to run the scale-out script for adding VMs, it is required to have a host acting as "preparation server". This preparation server must have connectivity towards the ECM server.
For the specific case of deployments using VMware vCloud Director, during the SAPC vApp deployment, the OVF package to be used as template to scale-out PL VMs was uploaded to the catalog. The OVF package for scale-out already uploaded to the catalog is required during this procedure.

2 Configure Scale-Out Procedure

To scale-out a VM:

Use the specific Cloud Manager to create one VM that act as PL.
Caution!
The VM must have the same number of Virtual CPUs (vCPUs), the same amount of RAM, and the same number of ports as the other PLs in the vApp.

The SAPC offers procedures for four different Cloud Managers:
1. For ECM deployments over CEE environments, follow the procedure described in Section 3.
2. For Atlas/Director deployments over OpenStack environments, follow the procedure described in Section 4.
3. For VMware vSphere Client deployments over VMware environments, follow the procedure described in Section 5.
4. For VMware vCloud Director deployments over VMware environments, follow the procedure described in Section 6.
Wait until the new VM automatically PXE boot from the System Controller (SC) node VMs.
Navigate to the CrM managed object, for example:
> dn ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1

Verify that the scale-out process has started:

(CrM=1)> show -r

The following is an example output:

Note:: PL-3 is the first PL of the SAPC vApp. As the two first PLs (PL-3 & PL-4) belong to the initial deployment, the number of the scaled-out PLs is PL-5 onwards.

CrM=1
   autoRoleAssignment=ENABLED

   ComputeResourceRole=PL-3
      adminState=UNLOCKED
      instantiationState=INSTANTIATED
      operationalState=ENABLED
      provides="ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1,Role=Default-Role"
      uses="ManagedElement=1,Equipment=1,ComputeResource=PL-3"
   ComputeResourceRole=PL-4
      adminState=UNLOCKED
      instantiationState=INSTANTIATED
      operationalState=ENABLED
      provides="ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1,Role=Default-Role"
      uses="ManagedElement=1,Equipment=1,ComputeResource=PL-4"
   ComputeResourceRole=PL-5
      adminState=UNLOCKED
      instantiationState=INSTANTIATING
      operationalState=DISABLED
      provides="ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1,Role=Default-Role"
      uses="ManagedElement=1,Equipment=1,ComputeResource=PL-5"
   Role=SYSTEM
      isProvidedBy
      scalability=NON_SCALABLE
   Role=Default-Role
      isProvidedBy
         "ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1,ComputeResourceRole=PL-3"
         "ManagedElement=1,SystemFunctions=1,SysM=1,CrM=1,ComputeResourceRole=PL-4"
      scalability=SCALABLE

This example shows that instantiationState has changed to INSTANTIATING for node PL-5. It means that the Scale out has started for node PL-5.

Continue to check the progress until the scale-out process has ended and the added node has joined the cluster:

(CrM=1)>show -m ComputeResourceRole -p instantiationState,operationalState

The following example output shows the final result:

ComputeResourceRole=PL-3
   instantiationState=INSTANTIATED
   operationalState=ENABLED
ComputeResourceRole=PL-4
   instantiationState=INSTANTIATED
   operationalState=ENABLED
ComputeResourceRole=PL-5
   instantiationState=INSTANTIATED
   operationalState=ENABLED

This example shows that instantiationState has changed to INSTANTIATED for node PL-5. It means that PL-5 is added to the cluster.

The example also shows that operationalState has changed to ENABLED for node PL-5. It means that node PL-5 has joined the cluster.

Perform a health check specific for the scaling procedure of the PL. The use is shown:
sapcadmin@SC-X:~>sapcScaleOutHealthCheck <PL-X>

Usage sapcHealthCheck PL-Num PL-Num: mandatory argument. It indicates the new PL machine scaled out in the cluster during the whole procedure. Example: PL-5.

The next example shows the script output for a succeed state:

sapcadmin@SC-X:~>sapcScaleOutHealthCheck PL-5
```
Checking PL-5 node becomes up at TIPC level...
The new neighbor 5 is up at TIPC level.

DBS is Started in PL-5. DBS level...

Scale-Out for PL-5 has been successfully performed.

It is recommended to perform a whole SAPC Health Check:
sapcHealthCheck
```
Perform a health check, refer to Health Check chapters available in the Deployment Instruction documents: SAPC VNF Deployment Instruction for OpenStack and SAPC VNF Deployment Instruction for VMware.
The new PL starts processing traffic automatically.
Repeat the procedure as many times as new PLs are needed.
Caution!
It is not supported to start a new PL scale-out procedure while there is another PL scaling ongoing.
Perform a system data backup following the produce described in System Backup and Restore.

3 VM Creation using ECM

The steps to be executed are:

From the preparation server, a tool to create automatically VMs in ECM, delivered as part of the Virtual Delivery Package (VDP) for OpenStack (vdp_sapc_qcow2_cxp9032849_<revision>.tar.gz) must be executed. Before the execution, the following information must be provided in the "ecm" and "scaling" sections of the configuration file SAPC.cfg:
Note:
Detailed description of the parameters contained on the scaling section of SAPC.cfg can be found on the SAPC VNF Descriptor Generator Tool.
- The name of the SAPC Virtual Application (vApp).
  Note:
  In case, more than one SAPC vApps are deployed on the same Virtual Data Center (VDC), a prefix is added to the name of the additional instances. This prefix must be also provided as part of the name of the vApp to scale-out.
- VDC where the vApp is deployed.
- Tenant of the VDC where the vApp is deployed.
- ECM server hostname or IP Address.
- User to connect with the ECM server. Password is optional, in case it is not provided in the configuration file, the user is prompted.
- Number of PLs to scale-out: 1.
The rest of the parameters of the configuration file SAPC.cfg can be ignored.
The SAPC_orchestrator, located in the bin/ directory of the VDP, is the tool which creates VMs in ECM. To execute it, the following arguments are available:
Note:
For the execution of the script from the bin directory of the VDP, only the presence of the configuration file SAPC.cfg and the --option argument with the operation to perform is needed.
- --help or -h: if this argument is present, an explanation for every argument is shown and the execution finishes.
- --version: if this argument is present, the version is shown and the execution finishes.
- --option: mandatory argument. It indicates the operation to perform. Possible values are: onboarding, instantiation, scaleout, scalein, termination.
- --vdp: mandatory argument for onboarding option. It indicates the path to the VDP.
- --cfg <CFG> or -c <CFG>: mandatory argument. It indicates where the SAPC.cfg configuration file is. The default SAPC.cfg configuration file is placed in the root directory of the Virtual Delivery Package explained in SAPC VNF Descriptor Generator Tool.
- --debug: optional argument for debugging and troubleshooting purposes.
- --log <LOG>: optional argument to log the messages to a file instead of showing them into the standard output.
Then, an example of the SAPC_orchestrator execution is:

<PreparationServer># SAPC_orchestrator --option scaleout --cfg SAPC.cfg
Once the ECM has received the order from the previous script, a new VM is displayed on ECM belonging to the SAPC vApp.
After some minutes, the new VM will be operational and integrated as a new PL in the SAPC vApp.

4 VM Creation using Atlas/Director

The steps to be executed are:

Connect by SSH to the machine running Atlas/Director using the provided credentials.
Execute the following command to obtain the current value of the parameter pls.
heat stack-show <stack_name> | grep -i '"pls": "'

Note:
For all the commands to be executed, it is needed to set up the following environment variables:
- OS_AUTH_URL points to Keystone service.
- OS_CACERT for the verification of the server certificate.
- OS_TENANT_NAME contains the name of the tenant where the stack was created.
- OS_USERNAME: user with delete rights.
- OS_PASSWORD: password for OS_USERNAME.
Execute the following command to add the latest <PL-N> to the desired stack. The value of N is the number of pls obtained in previous command:
heat stack-update -f <stack_yaml_file> -e <stack_params_file> -r <stack_name> -P 'pls=N+1'

Note:
See SAPC VNF Deployment Instruction for OpenStack for description of other parameters.
Run the command "<atlasvm># heat stack-list"; the status for the stack changes toUPDATE_IN_PROGRESS, until operation is finished and the status changes to UPDATE_COMPLETED.

5 VM Creation using VMware vSphere Web Client

The steps to be executed are:

Clone one of the PLs that is running in the SAPC vApp, selecting the VM from the Host and Clusters view, right mouse click it and select Clone... and Clone to Virtual Machine....
Rename the VM to PL-X.
Include the new VM in the SAPC vApp.
Select a destination storage for the VM files.
Select the customization option for the guest operating system.
Ready to Complete. Click Finish to start a task that creates the new VM.
Once the VM in cloned, edit its settings to ensure that the MAC assignation of every vNIC of the new VM is automatic. Select PL-X VM from the Host and Clusters view, right mouse click it and select Edit Settings.

Once the VM Properties window is shown, position on each of the Network Adapters and verify that Automatic option is selected in MAC Address field. Once all Network Adapters for PL-X VM have been verified, click OK to confirm the changes.
Power on the new VM, selecting PL-X VM from the SAPC vApp, right mouse click it and select Power > Power On .
After some minutes, the new VM is operational and integrated as a new PL in the SAPC vApp.

6 VM Creation in using VMware vCloud Director

During the SAPC vApp deployment, the OVF package to be used as template to scale-out PL VMs was uploaded to the VMware vCloud Director Catalog. During this procedure it is used.

The steps to be executed are:

Go to the VMware vCloud Director Catalog where the OVF package to be used as template for the scale-out of PL VMs was uploaded. Check that the ScaleOut of PL is available in the catalog (the default name is SAPC_PL_ScaleOut_Template).
Go to the SAPC vApp View, and click on the VMs tab. Then, click on the + icon.
In the window pop-up, look for the template to be used for the PL scale-out. It contains just one VM whose name is PL-X. Click on the PL-X VM.
Click on the Add button below, and then, click Next.
Edit properly the name of the new PL VM to be added to the SAPC vApp. In this example, considering that the initial deployment has just been done, and PL-3 and PL-4 are the only PL VMs that exist, the next PL to be added is PL-5. Select also the desired storage for the VM, and click on Next.
The virtual networks for the PL are now shown.

Configure the virtual networks in the proper order, according to SAPC VNF Network Configuration Guide (and also according to the order presented in the OVF descriptor content). Then, click Next.
Click Next.
Review the details shown in the summary, and click Finish.
The newly created PL VM appears as part of the SAPC vApp. Initially, the SAPC vApp appears as Updating....and the new PL VM with status Busy.

After some seconds, depending on the hardware used in the VMware environment, the SAPC vApp appears as Partially Running, given that the new PL VM is on Powered Off status.
Right mouse click over the new PL VM, and then click on Power On.
Finally, the new PL VM status becomes Powered On, and the SAPC vApp appears as Running.
After some minutes, the new VM is operational and integrated as a new PL in the SAPC vApp.