Contents

1   Introduction

This document describes the data migration from HP 15B FD1 CP3 to IPWorks 2. It mainly focuses on giving a brief overview of the data migration scripts and the rule files for DNS, ENUM, AAA Diameter, AAA Radius, and DHCP configurations, and database migration.

2   Overview

This section provides a brief description of the backup and migration from HP to IPWorks 2.

• Backup

  The backup includes backup scripts and a backup rule file from HP.

  It backs up the configurations files and database data for DNS, ENUM, AAA Diameter, AAA Radius, and DHCP services.

• Migration

  The migration includes the DNS, ENUM, AAA Diameter, AAA Radius, DHCP service configuration, and database migration.

3   Architecture for the Backup and Migration

Data migration contains backup from HP and migration from HP to IPWorks 2.

The scripts and correspond rule files are used for backup and migration.

The migration uses the backup packages.

Figure 1 describes the architecture for the backup and migration.

• Backup: stores the backup scripts.
• dest: stores the generated files after running the backup and migrate scripts.
• migration: stores the migration scripts.
• ruleconf: stores all rule files.
• util: Util scripts are called by the backup and migration scripts.

4   Backup

This section provides the description of the following aspects:

• The backup scripts and the rule file for configurations and databases backup.
• The backup scripts for the network backup.

4.1   Backup scripts and rule file for configurations and databases backup

This backup contains common configuration backup and service backup based on the services, such as DNS, ENUM, AAA, and DHCP.

1. Backup script: ipwbackup_for_datamigration.sh

   Script usage:

   ./ipwbackup_for_datamigration.sh <DNS|ENUM|AAA-Dia|AAA-Rad|DHCP|COMMON|DB>

   Description:

   DNS: backup DNS services

   ENUM: backup ENUM services

   AAA-Dia: backup AAA Diameter services

   AAA-Rad: backup AAA Radius services

   DHCP: backup DHCP services

   COMMON: backup COMMON configuration

   DB: backup DB dump

   • For common configuration backup

     For example:

     # ./ipwbackup_for_datamigration.sh COMMON

     --------------------------------------

     IPWORKS Backup/Migration Utility

     --------------------------------------

     Checking ipworks.conf file....OK

     Backing up COMMON configuration files....Successfully.

     Creating tar archive...Successfully.

     Creating compressed gzip archive file...Successfully.

     BACKUP OPERATION COMPLETED SUCCESSFULLY

     The file /tmp/ipwbackup_074445_06212017_tesla-ipworks02_for_COMMON.tar.gz has been created .

     --------

     WARNING

     --------

     File Created :

     /tmp/ipwbackup_074445_06212017_tesla-ipworks02_for_COMMON.tar.gz

     THE FILE WILL BE ERASED PERMANENTLY ON REBOOT.

     PLEASE BACKUP THE FILE TO A REMOTE DIRECTORY / ANY STORAGE MEDUIM (eg. CD,TAPE DRIVE).

   • For service configuration backup

     Back up all the service configurations with service name.

     For example:

     # ./ipwbackup_for_datamigration.sh DNS

     --------------------------------------

     IPWORKS Backup/Migration Utility

     --------------------------------------

     Backing up DNS services configuration files...Successfully.

     Creating tar archive...Successfully.

     Creating compressed gzip archive file...Successfully.

     BACKUP OPERATION COMPLETED SUCCESSFULLY.

     The file ../dest/ipwbackup_090423_06132017_ci-wsqr2fwytfsyipworks28_ for_DNS.tar.gz has been created.

     File Created:

     ../dest/ipwbackup_090423_06132017_ci-wsqr2fwytfsyipworks28_ for_DNS.tar.gz

     THE FILE WILL BE ERASED PERMANENTLY ON REBOOT.

     PLEASE BACKUP THE FILE TO A REMOTE DIRECTORY / ANY STORAGE MEDUIM (eg. CD,TAPE DRIVE).

   • For database backup

     Backup the databases dumped from database IPWorks and Mysql.

     For example:

     # ./ipwbackup_for_datamigration.sh DB

     --------------------------------------

     IPWORKS Backup/Migration Utility

     --------------------------------------

     Taking ipworks Database backup...successfully.

     Creating tar archive...Successfully.

     Creating compressed gzip archive file...Successfully.

     BACKUP OPERATION COMPLETED SUCCESSFULLY.

     The file ../dest/ipwbackup_090822_06132017_ci-wsqr2fwytfsyipworks28_ for_DB.tar.gz has been created .

     File Created :

     ../dest/ipwbackup_090822_06132017_ci-wsqr2fwytfsyipworks28_ for_DB.tar.gz

     THE FILE WILL BE ERASED PERMANENTLY ON REBOOT.

     PLEASE BACKUP THE FILE TO A REMOTE DIRECTORY / ANY STORAGE MEDUIM (eg. CD,TAPE DRIVE).

2. The rule file ipw_service_backup_rule.csv for configurations and databases backup

   This rule file ipw_service_backup_rule.csv defines the files and folders which should be backed up.

   There are following attributes in the ipw_service_backup_rule.csv:

   • The first column: the service name, including DNS, ENUM, AAA-Dia, AAA-Rad, DHCP, and COMMON.

     DNS: the DNS service

     ENUM: the ENUM service

     AAA-Dia: the AAA diameter service

     AAA-Rad: the AAA Radius service

     DHCP: the DHCP service

     COMMON: the common part configuration, such as SS configuration, licence configuration, and backup and restore configuration.

   • The second column: type for backup, including FILE and FOLDER.

     FILE: back up a file specified in the third column.

     FOLDER: back up all the files in the folder specified in the third column.

   • The third column: the backup content, including the file name and path, or the folder name and path.

     For example: ipw_service_backup_rule.csv

     

4.2   Backup script for network backup

The script ipwbackup_serverinfo.sh gets the server name and address from the following tables in DB dump file and the host name of IP address from /etc/hosts.

Table name:

• ENUMSERVER
• dnsserver
• dnsserver_address
• aaaserver
• aaaserver_address
• dhcpv4server
• dhcpv4server_address

For example:

# ./ipwbackup_serverinfo.sh

File ../dest/server_address.csv is generated.

Successfully.

5   Migration

This section provides:

• Description of the migration scripts for configurations and databases.
• Description of the rule files for configurations and databases migration.

5.1   Migration Scripts for Configurations and Databases

Configuration migration includes DNS, ENUM, AAA Diameter, AAA Radius, DHCP, and Common part.

The databases migration includes DNS, ENUM, AAA, DHCP, COMMON part, which use the mapping rules in rule files to do the migration.

The parameters in configuration files in HP have mainly the following scenarios to be set into IPWorks 2:

• The parameters in HP configuration file need migrate to the IPWorks 2 configuration file.
• The parameters in HP configuration file need migrate to the ECIM of IPWorks 2.
• The parameters in HP configuration file are removed or not used for IPWorks 2, discard them.

For more information, refer to the section 5.2.

The database in HP has the following scenarios to migrate to IPWorks 2:

• Back up the database and migrate to IPWorks 2 database directly.
• Some database tables in HP need to migrate to modeling of IPWorks 2.

For more information, refer to the section 5.2.

5.1.1   Service Configuration Migration

# ./ipw_migrate_service.py -h

usage: ipw_migrate_service.py [-h] {RADIUS,DIAMETER,COMMON,DNS,ENUM,DHCP} host file

Migrate IPworks by services

positional arguments:

{RADIUS,DIAMETER,COMMON,DNS,ENUM,DHCP}

service name

host host name

file a tar.gz backup file

optional arguments:

-h, --help show this help message and exit

A NETCONF xml file including the parameters will be generated which needs to imported into ECIM using netconf command manually.

5.1.2   Database Migration

Database migration includes DNS, ENUM, AAA, DHCP, and COMMON part.

Use the mapping rules in rule files to do the migration.

• The script ipw_migrate_db.py

  # ./ipw_migrate_db.py -h

  usage: ipw_migrate_db.py [-h] {AAA,COMMON,DNS,ENUM,DHCP} file

  Migrate IPworks Database

  positional arguments:

  {AAA,COMMON,DNS,ENUM,DHCP}

  service name

  file tar.gz backup file containing DB dump files

  optional arguments:

  -h, --help show this help message and exit

  In case of AAA, a NETCONF xml file including the parameters will be generated which needs to imported into ECIM using netconf command manually.

• The script ipw_migrate_conf_db.py

  There are some IPWorks features combined with some fields in IPWorks database on HP platform to define the parameters, which will be imported into ECIM.

  The configuration for the features in IPWorks 2 will be configured only in ECIM.

  In this case, the related parameters in conf files and database tables in HP need to migrate to corresponding parameters in ECIM of IPWorks 2.

  This script will handle this case.

  # ./ipw_migrate_conf_db.py -h

  usage: ipw_migrate_conf_db.py [-h] dbfile conffile

  Migrate IPworks db and config file to modeling

  positional arguments:

  dbfile tar.gz backup file containing DB dump files

  conffile a tar.gz backup file for service

  optional arguments:

  -h, --help show this help message and exit

  A NETCONF xml file including the parameters will be generated which needs to imported into ECIM using netconf command manually.

5.2   Rule Files for Configurations and Databases Migration

The data migration from HP to IPWorks 2 follows the following csv rule files:

• common_mapping_rule.csv

  The mapping rule file, "common_mapping_rule.csv", is used for the configuration migration from 15B FD1 CP3 to IPWorks 2.

  There are following attributes in the common_mapping_rule.csv.

  • src: the configuration file in HP.
  • section: section in HP configuration file, if not contains as blank.
  • key: the parameter key in the configuration file.
  • rule: The rule supports followings:
    • netconf mapping: The parameter in configuration file is set to the modeling. It depends on the "cliss dn" and "cliss attribute".
    • row mapping: The attributes of destination file, destination section, and destination key are required during the row mapping of the configuration parameter from HP to IPWorks 2.
    • discard: The configuration parameter is discarded in IPWorks 2
    • file mapping: The attribute of destination file is required during the file mapping of the configuration file from HP to IPWorks 2.
  • file/cliss dn: destination file and destination cliss dn.

    The file is as a destination file existed in IPWorks 2, and some or all parameters are consistent with them in 15B FD1 CP3.

    The cliss dn is the netconf path, to which will be set the 15B parameter key.

  • section: the section in the destination configuration file, if not contains as blank.
  • key/cliss attribute: the parameter name in netconf.

    Note:  

    <hostname> in the rule file is replaced by <SC Node hostname>.

    
    
• db_mapping_rule.csv

  The mapping rule file, "db_mapping_rule.csv", is used for the database migration from HP to IPWorks 2.

  There are following attributes in the db_mapping_rule.csv.

  • server: includes DNS, ENUM, AAA, DHCP and common
  • table: the table belongs to src_database in HP database.
  • src_database: the databases named in HP
  • dest_database: the databases named in IPWorks 2
  • rule: The rule supports followings:
    • discard: The table does not exist or need to migrate to IPWorks 2.
    • copy mapping: The table is not changed and has no records in IPWork 1.
    • truncatePreCopy: The table is not changed, but there is one or more records in IPWorks 2. Truncate the table before overwriting it.
    • netconf mapping: This rule will migrate the fields in the tables to modeling in IPWorks 2. And it will relate to the table_mapping_netconf_rule.csv.
    • row redundancy: The table is changed, one or more fields are discarded in IPWorks 2. The fields can be specified with the "changed fields" rule.
  • changed fields: the table fields are changed in IPWorks 2. If more than one field is changed, use "|" to specify the fields.
• dns_mapping_rule.csv

  The mapping rule file, "dns_mapping_rule.csv", is used for the configuration migration from 15B FD1 CP3 to IPWorks 2.

  There are following attributes in the dns_mapping_rule.csv.

  • src: the configuration file in 15B FD1 CP3.
  • section: section in 15B configuration file, if not contains as blank.
  • key: the parameter key belongs to section in the configuration file.
  • rule: The rule supports followings:
    • netconf mapping: The attributes of cliss dn, and cliss are required during the netconf mapping of the configuration file from HP to IPWorks 2.
    • row mapping: The attributes of destination file, destination section, and destination key are required during the row mapping of the configuration parameter from HP to IPWorks 2.
    • discard: The configuration parameter is discarded in IPWorks 2.
    • file mapping: The attribute of destination file is required during the file mapping of the configuration file from HP to IPWorks 2.
  • file/cliss dn: destination file and destination cliss dn.

    The file is as a destination file existed in IPWorks 2, and some or all parameters are consistent with them in HP.

    The cliss dn is the netconf path, to which will be set the HP parameter key.

  • section: the section in the destination configuration file, if not contains as blank.
  • key/cliss attribute: the parameter name in netconf.

  Note:  

  <hostname> in rule file is replaced by <PL Node>.

  
  
• enum_mapping_rule.csv

  The mapping rule file, "enum_mapping_rule.csv", is used for the configuration migration from 15B FD1 CP3 to IPWorks 2.

  There are following attributes in the enum_mapping_rule.csv.

  • src: the configuration file in HP.
  • section: section in HP configuration file, if not contains as blank.
  • key: the parameter key belongs to section in the configuration file.
  • rule: The rule supports followings:
    • netconf mapping: The attributes of cliss dn, and cliss are required during the netconf mapping of the configuration file from HP to IPWorks 2.
    • row mapping: The attributes of destination file, destination section, and destination key are required during the row mapping of the configuration parameter from HP to IPWorks 2.
    • discard: The configuration parameter is discarded in IPWorks 2
    • file mapping: The attribute of destination file is required during the file mapping of the configuration file from HP to IPWorks 2.
  • dest/cliss dn: destination file and destination cliss dn.

    The dest is as a destination file existed in IPWorks 2, and some or all parameters are consistent with them in HP.

    The cliss dn is the netconf path, to which will be set the HP parameter key.

  • section: the section in the destination configuration file, if not contains as blank.
  • key/cliss attribute: the parameter name in netconf.

  Note:  

  <hostname> in rule file is replaced by <PL Node>.

  
  
• diameter_mapping_rule.csv

  The mapping rule file, "diameter_mapping_rule.csv", is used for the configuration migration from HP to IPWorks 2.

  There are following attributes in the diameter_mapping_rule.csv.

  • src: the configuration file in HP.
  • section: section in HP configuration file, if not contains as blank.
  • key: the parameter key belongs to section in the configuration file.
  • rule: The rule supports followings:
    • netconf mapping: The attributes of cliss dn, and cliss are required during the netconf mapping of the configuration file from HP to IPWorks 2.
    • row mapping: The attributes of destination file, destination section, and destination key are required during the row mapping of the configuration parameter from HP to IPWorks 2.
    • discard: The configuration parameter is discarded in IPWorks 2.
    • file mapping: The attribute of destination file is required during the file mapping of the configuration file from HP to IPWorks 2.
  • file/cliss dn: destination file and destination cliss dn.

    The file is as a destination file existed in IPWorks 2, and some or all parameters are consistent with them in HP.

    The cliss dn is the netconf path, to which will be set the HP parameter key.

  • section: the section in the destination configuration file, if not contains as blank.
  • key/cliss attribute: the parameter name in netconf.

  Note:  

  <hostname> in rule file is replaced by <PL Node>.

  
  
• radius_mapping_rule.csv

  The mapping rule file, "radius_mapping_rule.csv", is used for the configuration migration from HP to IPWorks 2.

  There are following attributes in the radius_mapping_rule.csv.

  • src: the configuration file in HP.
  • section: section in HP configuration file, if not contains as blank.

    If there’s any parameter with the same name in the same configuration file, and belong to different sections, the parameters need to be specified the section.

  • key: the parameter key belongs to section in the configuration file.
  • rule: The rule supports followings:
    • netconf mapping: The attributes of cliss dn, and cliss are required during the netconf mapping of the configuration file from HP to IPWorks 2.
    • row mapping: The attributes of destination file, destination section, and destination key are required during the row mapping of the configuration parameter from HP to IPWorks 2.
    • discard: The configuration parameter is discarded in IPWorks 2
    • file mapping: The attribute of destination file is required during the file mapping of the configuration file from HP to IPWorks 2.
  
  
  • file/cliss dn: destination file and destination cliss dn.

    The file is as a destination file existed in IPWorks 2, and some or all parameters are consistent with them in 15B FD1 CP3.

    The cliss dn is the netconf path, to which will be set the 15B parameter key.

  • section: the section in the destination configuration file, if not contains as blank.
  • key/cliss attribute: the parameter name in netconf.

  Note:  

  <hostname> in rule file is replaced by <PL Node>.

  
  
• table_mapping_netconf_rule.csv

  The mapping rule file, "table_mapping_netconf_rule.csv", is used for the configuration migration from 15B FD1 CP3 to IPWorks 2.

  There are following attributes in the table_mapping_netconf_rule.csv.

  • table: the table in HP database.
  • field: belongs to the table.
  • rule: the rule supports followings:
    • directly mapping: the field in the table is consistent with the netconf parameter directly.
    • pending mapping: the field mapping depends on the pending field in the table.
    • pending join mapping: the field mapping depends on other table’s field, need to join the depending table.
  • pending field: other field depends on it to set the cliss dn.
  • filed value: the value of the pending field.
  • join table: the table in which the field mapping depends on the "pending field" and "join field".
  • join field: the value of "join field" is equal the value of "pending field".
  • select left field: the table fields are changed in IPWorks 2. More than one field is changed, use "|" to specify the fields.
  • select right field: the filed in "join table", and will be set to netconf.
  • cliss dn: the netconf path, to which will be set the HP parameter key.
  • cliss attribute: the parameter name in netconf.

  Note:  

  If the "pending field" is not blank and not sure its value, can be specified with <NOT NULL> value in "field value".

  
  
• conf_db_mapping_netconf_rule.csv

  The mapping rule file, "conf_db_mapping_netconf_rule.csv", is used for the configuration migration from HP to IPWorks 2.

  There are following attributes in the conf_db_mapping_netconf_rule.csv.

  • src: the configuration file in HP.
  • section: section in HP configuration file, if not contains as blank.
  • key: the parameter key belongs to section in the configuration file.
  • confvalue_enum: all the possible values of the parameter. use "|" to specify the values. For example: true|false.

    It combines with "fieldvalue_enum" to determine the value of the "cliss attribute".

  • table: the table in HP database.
  • field: belongs to the table.
  • fieldvalue_enum: all the possible values of the field. Use "|" to specify the values. For example: true|false.
  • policy: all the combination value with "confvalue_enum" and "fieldvalue_enum". Use "|" to specify the values.

    The value is specified by “|”, the first is defined as “0”, the second is “1”, and so on. For example:

    confvalue_enum	fieldvalue_enum
    true|false	true|false

    =>

    confvalue_enum	fieldvalue_enum
    0|1	0|1

    The combination value with "confvalue_enum" and "fieldvalue_enum", determine the value of the "cliss attribute".

    For example:

    confvalue_enum	fieldvalue_enum	csvMode
    0	0	SESSION_BASED_CSV
    0	1	DISABLE_CSV
    1	0	MESSAGE_BASED_CSV
    1	1	DISABLE_CSV

    The "policy" is like the following:

    policy

    00:SESSION_BASED_CSV|01:DISABLE_CSV|10:MESSAGE_BASED_CSV|11:DISABLE_CSV

  • cliss dn: the netconf path, to which will be set the HP parameter key.
  • cliss attribute: the parameter name in netconf.