HiCHperfpkg
Author: Maik Ernst (maik.ernst@hitachivantara.com)

Date: 2018-04-20
Version: 8.0.4

Consists of:
bin/ppsetup.pl                  	v1.3.2
bin/ppcollect.pl                	v5.0.4
bin/pptransfer.pl               	v2.3.1
bin/ppmonitor.pl                	v1.6.0
lib/modules/HiCHpp_RAIDCOM.pm   	v2.3.0
lib/modules/HiCH_LOG.pm         	v1.0.0
lib/modules/HiCH_RESTSVP.pm     	v1.1.0
lib/modules/c.pl                	v0.9.1
bin/curl.exe				v7.58.0 
(x86_64-w64-mingw32) libcurl/7.58.0 OpenSSL/1.0.2h (WinSSL) zlib/1.2.8 WinIDN libssh2/1.7.0_DEV
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTPS-proxy MultiSSL
bin/7-ZipPortable/7-Zip64/7z.exe	v18.01
bin/7-ZipPortable/7-Zip/7z.exe		v18.01
bin/putty/*				v0.70
doc/HiCHperfpackage_Install_v8.0.4.pdf	v8.0.4

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Previous versions:

Date: 2018-02-14
Version: 8.0.3
Consists of:
bin/ppsetup.pl                  	v1.3.2
bin/ppcollect.pl                	v5.0.3
bin/pptransfer.pl               	v2.3.0
bin/ppmonitor.pl                	v1.6.0
lib/modules/HiCHpp_RAIDCOM.pm   	v2.2.1
lib/modules/HiCH_LOG.pm         	v1.0.0
lib/modules/HiCH_RESTSVP.pm     	v1.1.0
lib/modules/c.pl                	v0.9.1
bin/curl.exe				v7.58.0 
(x86_64-w64-mingw32) libcurl/7.58.0 OpenSSL/1.0.2h (WinSSL) zlib/1.2.8 WinIDN libssh2/1.7.0_DEV
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTPS-proxy MultiSSL
bin/7-ZipPortable/7-Zip64/7z.exe	v18.01
bin/7-ZipPortable/7-Zip/7z.exe		v18.01
bin/putty/*				v0.70

Date: 2016-11-26
Version: 8.0.2
Consists of:
bin/ppsetup.pl                  v1.3.2
bin/ppcollect.pl                v5.0.2
bin/pptransfer.pl               v2.3.0
bin/ppmonitor.pl                v1.6.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.2.1
lib/modules/HiCH_LOG.pm         v1.0.0
lib/modules/HiCH_RESTSVP.pm     v1.0.0
lib/modules/c.pl                v0.9.1

Date: 2016-04-02
Version: 7.3.1
Consists of:
bin/ppsetup.pl                  v1.2.1
bin/ppcollect.pl                v4.3.1
bin/pptransfer.pl               v2.2.1
bin/ppmonitor.pl                v1.5.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.1.0

Date: 2016-03-27
Version: 7.3.0
Consists of:
bin/ppsetup.pl                  v1.2.0
bin/ppcollect.pl                v4.3.0
bin/pptransfer.pl               v2.2.0
bin/ppmonitor.pl                v1.5.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.1.0

Date: 2015-08-30
Version: 7.2.0
bin/ppsetup.pl                  v1.1.2
bin/ppcollect.pl                v4.2.0
bin/pptransfer.pl               v2.1.0
bin/ppmonitor.pl                v1.4.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.0.0

Date: 2015-07-24
Version: 7.1.3
bin/ppsetup.pl                  v1.1.2
bin/ppcollect.pl                v4.1.2
bin/pptransfer.pl               v2.1.0
bin/ppmonitor.pl                v1.4.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.0.0

############################################################################
Modification history

bin/ppsetup.pl
# MODIFICATIONS
#   2015-06-17 - v1.0.0  - initial script
#   2015-07-02 - v1.0.1  - Fixing Gx00 bug (ip instead of svpip in cmd.txt and other order cmd.txt)
#   2015-07-02 - v1.0.1    Don't allow "." and " " in script path. Disable raidcom for Gx00
#   2015-07-03 - v1.1.0  - Check for controller IPs on Gx00 (GUM) and allow raidcom.
#   2015-07-04 - v1.1.1  - Add " to command.txt outpath
#   2015-07-24 - v1.1.2  - Changing default from "ppmonitor_switch = on" to "ppmonitor_switch = off"
#   2016-03-25 - v1.2.0  - LOG->autoflush(1), format change, adding CFGFILE pptransfer_proxyoptions
#                        - transfer_https_pxy_user and transfer_https_pxy_pwd check (@ and & not allowed)
#                        - adding ppcollect_keepdays, pptransfer_keepdays, ppmonitor_keepdays
#                        - adding DO_UPGRADE (change config files)
#   2016-04-02 - v1.2.1  - Change screenoutput 7.3.0 to 7.3.x
#   2016-11-19 - v1.3.0  - adding logging module HiCH_LOG, change logging call to have more parameters (e.g. verbose)
#                        - adding ppcollect_memoryxmx to setup inital and upgrade
#                        - fixing bug in upgrade procedure (range instead of shorange)
#                        - change shortrange default from -1200 to -1230:
#                        - adding G1500 support 
#                        - adding conf file updates 
#                        - adding cmdtxt updates 
#                        - Change screenoutput from 7.3.x to 8.0.x
#   2016-11-24 - v1.3.1  - adding restapi user and rest api password to initial and upgrade procedure
#   2016-11-26 - v1.3.2  - adding restapi password encryption


bin/ppcollect.pl
#    MODIFICATIONS
#    2013-07-09 - v1.0   - initial script
#    2013-07-09 - v1.1   - changed from only tar output file to tar/gzip
#    2013-07-10 - v1.2   - Adding VSP/USPV/HUSVM switch for correct java export tool command
#                          Deleting existing outpath before export tool start to avoid issue if the script got aborted (e.g. strg-c)
#                          Adding script version information
#                          Set -w automatically when osname is *Win*
#                          Set script start time to variable SHORTRANGE if getting Shortrange was not possible from logfile
#                          Bugfix sort on modified time of export tool log file
#    2013-07-23 - v1.3   - Changing default in command txt files to shortrange of 12hours (shortrange  -1200:).
#                          The reason for this is that you get partially invalid values (-4) in LU_dat as the collection period 
#                          does not match the avaiable data (max 24 hours) when the collection takes too long.
#    2013-09-20 - v1.4   - Adding longrange -600000 to all command.txt
#    2013-09-25 - v1.5   - Removing longrange -600000 from all command.txt
#    2013-10-03 - v1.6   - Adding error output of EXPORTTOOL_RUN
#    2013-12-05 - v1.7   - Change command.txt settings from "PhyPG" to "PhyPG Short" and "PhyLDEV" to "PhyLDEV Short"
#                          The reason for this is to shorten the runtime of the export tool
#    2014-01-06 - v1.8   - Changing die-function to be logged in file
#    2014-02-07 - v2.0   - raidcom collection added => use HDSPERF_RAIDCOM.pm module
#    2014-02-07 - v2.1   - adding FindBin, adding HORCM_DIR absolute path due to issue with starting the instance
#    2014-02-17 - v2.2   - adding parsable file (semicolon delimiter) from raidcom output
#    2014-02-21 - v2.3   - add raidcom collection of parity groups
#    2014-04-07 - v2.4   - add raidcom collection of pool and dp_pool
#                          Adding "short" to all long range groups in all command.txt that only short range gets collected. 
#                          The reason for this is to shorten the runtime.
#    2014-04-07 - v3.0   - Adding G1000 support
#                          Changing absolute path handling (=> abs_path) + slashes to backslash as path
#                          Adding -m at raidcom collection to get all MUs for raidscan
#                          But only for TC and not for SI as the collection would take too long.
#    2014-06-13 - v3.1   - Adding 7zip support for archiving with -z. This is to avoid high memory usage if the server has not enough memory to archive.
#                          (When you get error message: "out of memory")
#    2014-12-26 - v3.2   - Adding log file output
#    2015-04-02 - v3.3   - Adding -f option to use FC device instead of IP command device
#                          Adding -o option to collect only raidcom data and skip export tool data collection
#                          Adding Sys::Hostname to get hostname of script
#    2015-04-30 - v3.4   - Using export tool JSanRmiApiSx instead of JSanRmiApiEx for G1000
#    2015-05-02 - v4.0   - Change from parameter to config file => pm file config::tiny needed
#                          Changing script name from HDSperfcollect.pl to HCHppcollect.pl to meet swiss script standard
#    2015-05-05          - Bugfix (missing 3 in in CDM file for G1000 when using FC command device)
#    2015-05-10 - v4.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group 
#                          (tgz only works but only with absolute pahts => do not use it anymore)
#                          Log format changed (LOG_MSG)
#                          Adding new parameter like ppcollect_ccicollectionlevel
#    2015-07-02 - v4.1.1 - Allow G200, G400, G600 and G800 as parameter
#                          Adding ip-regexp from cmd.txt for G800/G600/G400/G200 (ip instead of svpip)
#                          Adding GUM ip regexp from cmd.txt for G800/G600/G400/G200 
#    2015-07-05 - v4.1.2 - Bugfix regexp to get outpath correctly from cmd.txt
#                          Bugfix unquote outpath
#    2015-08-29 - v4.2.0 - Implement G1000 MC v3 export toool changed
#                          (also for G800/G600/G400/G200 as assumption that it will be the same soon)
#                          Change variable to hash for export tool
#    2016-03-25 - v4.3.0 - LOG->autoflush(1), fix DELETE_OLD_SAVES DELOD log message
#                        - adding DELETE_OLD_LOGS / log file rotation ppcollect_logkeepdays
#    2016-04-02 - v4.3.1 - Change log output PWD = ***
#    2016-11-19 - v5.0.0 - adding logging module HiCH_LOG, change logging call to have more parameters (e.g. verbose)
#                        - adding REST API collection and method of collection levels (1-7)
#                        - adding ppcollect_memoryxmx=variable
#                        - adding G1500 support
#                        - changing DO_ARCHIV
#    2016-11-24 - v5.0.1 - adding restapi user and rest api password due to raidcom/REST api issue with same user on SVP
#    2016-11-26 - v5.0.2 - adding restapi password decryption

bin/pptransfer.pl
#    MODIFICATIONS
#    2013-09-26 - v1.0   - initial script
#    2013-10-02 - v1.1   - Adding spool directory equal to transfer directory
#                          Adding -e entry
#    2014-01-06 - v1.2   - Changing die-function to be logged in file
#                          Changing ADD_SPOOLLIST_NEW function.
#                          NEW: Adding file to spool list txt file AFTER sucessful scp
#    2014-01-06 - v1.3   - Adding -r entry (rotate option)
#                          With this entry the data in directory defined with -d will be deleted after specified days.
#    2014-02-20 - v1.4   - Change default ioportal target to ftp.hdsioportal.com
#                          Adding plink.exe command at the beginning of every transfter to make sure not to have keyprint issue
#                          This could be the case if different users are starting the transfer
#                          Fixing file to spool list will only be added if exit code of scp is 0 which means sucessful
#    2014-06-17 - v1.5   - Adding multiple suffix support (=> 7zip support)
#    2014-12-26 - v1.6   - Adding log file output
#    2015-04-05 - v1.7   - Adding "log" to suffix list that "scriptlog" dir can be transfered
#                          Adding Sys::Hostname to get hostname of script
#    2015-04-20 - v1.8   - Changing default  $REMOTE_HOST to "ftp.hdsioportal.com"
#    2015-05-01 - v2.0   - Change from parameter to config file => pm file config::tiny needed
#                          Adding https via curl as transfer method
#                          Changing script name from HDSperftransfer.pl to HCHpptransfer.pl to meet swiss script standard
#    2015-05-01 - v2.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group
#                          Log format changed (LOG_MSG)
#                          Debuglevel, CURL parameter change 
#    2016-03-25 - v2.2.0 - adding LOG->autoflush(1)
#                        - adding pptransfer_proxyoptions
#                        - changing CURL syntax to pptransfer_proxyoptions
#                        - addicng check HTTPS_PROXYUSER / HTTPS_PROXYPWD (@ and & not allowed)
#                        - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays
#    2016-04-02 - v2.2.1 - Change log output CURL RUN USER PWD  = ***
#    2016-11-19 - v2.3.0 - adding logging module HiCH_LOG, change logging call to have more parameters (e.g. verbose)
#                        - update of curl version 7.42 to 7.51
#                          old:
#                          curl 7.42.0 (i386-pc-win32) libcurl/7.42.0 OpenSSL/1.0.2a zlib/1.2.8
#                          Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
#                          Features: AsynchDNS IPv6 Largefile NTLM SSL libz
#                          new:
#                          curl 7.51.0 (i386-pc-win32) libcurl/7.51.0 OpenSSL/1.1.0c zlib/1.2.8 WinIDN libssh2/1.8.0 nghttp2/1.16.1
#                          Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
#                          Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTP2

bin/ppmonitor.pl
#    MODIFICATIONS
#    2014-01-03 - v1.0   - initial script
#    2014-06-16 - v1.1   - Moving up to folder ppmonitor. 
#                          Adding .zip as suffix to check
#                          Adding pptransfer checks (-i). 
#    2014-04-07 - v1.2   - Adding Sys::Hostname to get hostname of script
#    2015-05-02 - v1.3   - Changing script name from ppmonitor.pl to ppmonitor.pl to meet swiss script standard
#                          Adding CURL transfer check
#    2015-05-10 - v1.4.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group
#    2016-03-25 - v1.5.0 - LOG->autoflush(1), fix logfile checks
#                        - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays
#    2016-11-19 - v1.6.0 - adding logging module HiCH_LOG, change logging call to have more parameters (e.g. verbose)

lib/modules/HiCHpp_RAIDCOM.pm
#    MODIFICATIONS
#    2014-02-07 - v1.0   - initial module
#    2014-02-11 - v1.1   - adding -CLI to raidscan
#    2014-02-14 - v1.2   - adding CHECK_OLD_INSTANCE
#    2014-02-16 - v1.3   - add an output file of raidcom which is parsable with semicolon as delimiter
#                          This is quite difficult as the current delimiter is a space and the values can have spaces also.
#                          Not every output of a values is known therefore there are a lot of guesses if strings,digits,etc. 
#                          The following values has been testes and they can have one or more spaces in their output:
#                          meta_resource, pool name, host group name, HMO
#                          For now it looks it is working but troubleshooting within the regex might be needed. 
#                          If you experience issue please let me know.
#    2014-02-21 - v1.4   - add raidcom collection of parity groups
#    2014-04-07 - v1.5   - adding -fx to every collection
#                        - add raidcom get pool, raidcom get dp_pool -key opt
#    2014-05-22 - v1.6   - Adding -m at raidcom collection to get all MUs for raidscan.
#                          But only for TC and not for SI as the collection would take too long.
#    2015-04-02 - v1.7   - Adding -f at HDSperfcollect to use FC device instead of IP command device
#    2015-05-02 - v1.8   - Changing module name from HDSPERF_RAIDCOM.pm to HCHpp_RAIDCOM.pm to meet swiss script standard
#    2015-05-05            Bugfix (missing 3 in in CDM file for G1000 when using FC command device)
#    2015-05-05 - v1.9   - add raidcom level to disable some collections.
#                          0 = collect all, 1 = disable raidscan, 2 = disable raidscan and host_grp
#                          change log function
#    2015-07-03 - v2.0.0 - adding raidcom support for G800,G600,G400,G200$ (GUM IPs instead of SVP IP for these systems)
#    2016-03-25 - v2.1.0 - adding raidcom get dp_pool, fixing CMD_RAIDCOM_GET_POOL_OPT, changing randcom from 200 to 500 for horcm instance
#                        - delete old horcm log dir of horcm instance
#                        - adding 2 / 3 options for CMD_RAIDCOM_GET_POOL_OPT and CMD_RAIDCOM_GET_DPPOOL
#    2016-11-19 - v2.2.0 - adding logging module HiCH_LOG, change logging call
#                        - adding G1500 support
#                        - adding logging module HiCH_LOG, change logging call
#                        - adding raidcom -h (raw only)
#                        - adding raidcom get pool -key saving (raw only)
#                        - adding raidcom get pool -key fmc (raw only)
#                        - adding raidcom get resource -key opt  (raw only)
#                        - making get ports unique
#                        - fix parser initilaze value for get partiy grp, get port on Gx00, get dppool, get hostgroup on Gx00
#    2016-11-24 - v2.2.1 - adding fucntion CHECK_HORCM_LOGIN to check if still logged in. But not in use yet.

lib/modules/HiCH_LOG.pm
# MODIFICATIONS
#    2016-11-19 - v1.0.0 - initial module
#                          Functions: LOG_MSG_OPEN, LOG_MSG_CLOSE,LOG_MSG

lib/modules/HiCH_RESTSVP.pm
#    2016-11-19 - v1.0.0 - initial module
#                          Functions: REST_GET_LDEVS, REST_GET_LUPATH, REST_GET_HOSTWWNS, REST_GET_HOSTGROUPS, REST_GET_PARITYGROUPS, 
#                          REST_GET_POOLS, REST_GET_PORTS, REST_GET_POOLS, REST_GET_PORTS, REST_GET_STORAGE, REST_GET_VERSION,
#                          REST_DISCARD_SESSION, REST_CREATE_SESSION, REST_REQUEST, REST_REQUEST_SESSION

lib/modules/c.pl
# MODIFICATIONS
#    2016-11-26 - v0.9.1 - initial script