HiCHperfpkg
Author: Maik Ernst (maik.ernst@hds.com)

Date: 2016-04-02
Version: 7.3.1

Consists of:
bin/ppsetup.pl                  v1.2.1
bin/ppcollect.pl                v4.3.1
bin/pptransfer.pl               v2.2.1
bin/ppmonitor.pl                v1.5.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.1.0

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Previous versions:

Date: 2016-03-27
Version: 7.3.0
Consists of:
bin/ppsetup.pl                  v1.2.0
bin/ppcollect.pl                v4.3.0
bin/pptransfer.pl               v2.2.0
bin/ppmonitor.pl                v1.5.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.1.0

Date: 2015-08-30
Version: 7.2.0
bin/ppsetup.pl                  v1.1.2
bin/ppcollect.pl                v4.2.0
bin/pptransfer.pl               v2.1.0
bin/ppmonitor.pl                v1.4.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.0.0

Date: 2015-07-24
Version: 7.1.3
bin/ppsetup.pl                  v1.1.2
bin/ppcollect.pl                v4.1.2
bin/pptransfer.pl               v2.1.0
bin/ppmonitor.pl                v1.4.0
lib/modules/HiCHpp_RAIDCOM.pm   v2.0.0

############################################################################
Modification history

bin/ppsetup.pl
# MODIFICATIONS
# 2015-06-17 - v1.0.0    - initial script
# 2015-07-02 - v1.0.1    - Fixing Gx00 bug (ip instead of svpip in cmd.txt and other order cmd.txt)
# 2015-07-02 - v1.0.1      Don't allow "." and " " in script path. Disable raidcom for Gx00
# 2015-07-03 - v1.1.0    - Check for controller IPs on Gx00 (GUM) and allow raidcom.
# 2015-07-04 - v1.1.1    - Add " to command.txt outpath
# 2015-07-24 - v1.1.2    - Changing default from "ppmonitor_switch = on" to "ppmonitor_switch = off"
# 2016-03-25 - v1.2.0    - LOG->autoflush(1), format change, adding CFGFILE pptransfer_proxyoptions
#                        - transfer_https_pxy_user and transfer_https_pxy_pwd check (@ and & not allowed)
#                        - adding ppcollect_keepdays, pptransfer_keepdays, ppmonitor_keepdays
#                        - adding DO_UPGRADE (change config files)
# 2016-04-02 - v1.2.1    - Change screenoutput 7.3.0 to 7.3.x

bin/ppcollect.pl
#    MODIFICATIONS
#    2013-07-09 - v1.0   - initial script
#    2013-07-09 - v1.1   - changed from only tar output file to tar/gzip
#    2013-07-10 - v1.2   - Adding VSP/USPV/HUSVM switch for correct java export tool command
#                          Deleting existing outpath before export tool start to avoid issue if the script got aborted (e.g. strg-c)
#                          Adding script version information
#                          Set -w automatically when osname is *Win*
#                          Set script start time to variable SHORTRANGE if getting Shortrange was not possible from logfile
#                          Bugfix sort on modified time of export tool log file
#    2013-07-23 - v1.3   - Changing default in command txt files to shortrange of 12hours (shortrange  -1200:).
#                          The reason for this is that you get partially invalid values (-4) in LU_dat as the collection period 
#                          does not match the avaiable data (max 24 hours) when the collection takes too long.
#    2013-09-20 - v1.4   - Adding longrange -600000 to all command.txt
#    2013-09-25 - v1.5   - Removing longrange -600000 from all command.txt
#    2013-10-03 - v1.6   - Adding error output of EXPORTTOOL_RUN
#    2013-12-05 - v1.7   - Change command.txt settings from "PhyPG" to "PhyPG Short" and "PhyLDEV" to "PhyLDEV Short"
#                          The reason for this is to shorten the runtime of the export tool
#    2014-01-06 - v1.8   - Changing die-function to be logged in file
#    2014-02-07 - v2.0   - raidcom collection added => use HDSPERF_RAIDCOM.pm module
#    2014-02-07 - v2.1   - adding FindBin, adding HORCM_DIR absolute path due to issue with starting the instance
#    2014-02-17 - v2.2   - adding parsable file (semicolon delimiter) from raidcom output
#    2014-02-21 - v2.3   - add raidcom collection of parity groups
#    2014-04-07 - v2.4   - add raidcom collection of pool and dp_pool
#                          Adding "short" to all long range groups in all command.txt that only short range gets collected. 
#                          The reason for this is to shorten the runtime.
#    2014-04-07 - v3.0   - Adding G1000 support
#                          Changing absolute path handling (=> abs_path) + slashes to backslash as path
#                          Adding -m at raidcom collection to get all MUs for raidscan
#                          But only for TC and not for SI as the collection would take too long.
#    2014-06-13 - v3.1   - Adding 7zip support for archiving with -z. This is to avoid high memory usage if the server has not enough memory to archive.
#                          (When you get error message: "out of memory")
#    2014-12-26 - v3.2   - Adding log file output
#    2015-04-02 - v3.3   - Adding -f option to use FC device instead of IP command device
#                          Adding -o option to collect only raidcom data and skip export tool data collection
#                          Adding Sys::Hostname to get hostname of script
#    2015-04-30 - v3.4   - Using export tool JSanRmiApiSx instead of JSanRmiApiEx for G1000
#    2015-05-02 - v4.0   - Change from parameter to config file => pm file config::tiny needed
#                          Changing script name from HDSperfcollect.pl to HCHppcollect.pl to meet swiss script standard
#    2015-05-05          - Bugfix (missing 3 in in CDM file for G1000 when using FC command device)
#    2015-05-10 - v4.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group 
#                          (tgz only works but only with absolute pahts => do not use it anymore)
#                          Log format changed (LOG_MSG)
#                          Adding new parameter like ppcollect_ccicollectionlevel
#    2015-07-02 - v4.1.1 - Allow G200, G400, G600 and G800 as parameter
#                          Adding ip-regexp from cmd.txt for G800/G600/G400/G200 (ip instead of svpip)
#                          Adding GUM ip regexp from cmd.txt for G800/G600/G400/G200 
#    2015-07-05 - v4.1.2 - Bugfix regexp to get outpath correctly from cmd.txt
#                          Bugfix unquote outpath
#    2015-08-29 - v4.2.0 - Implement G1000 MC v3 export toool changed
#                          (also for G800/G600/G400/G200 as assumption that it will be the same soon)	
#                          Change variable to hash for export tool
#    2016-03-25 - v4.3.0 - LOG->autoflush(1), fix DELETE_OLD_SAVES DELOD log message
#                        - adding DELETE_OLD_LOGS / log file rotation ppcollect_logkeepdays
#    2016-04-02 - v4.3.1 - Change log output PWD = ***

bin/pptransfer.pl
#    MODIFICATIONS
#    2013-09-26 - v1.0   - initial script
#    2013-10-02 - v1.1   - Adding spool directory equal to transfer directory
#                          Adding -e entry
#    2014-01-06 - v1.2   - Changing die-function to be logged in file
#                          Changing ADD_SPOOLLIST_NEW function.
#                          NEW: Adding file to spool list txt file AFTER sucessful scp
#    2014-01-06 - v1.3   - Adding -r entry (rotate option)
#                          With this entry the data in directory defined with -d will be deleted after specified days.
#    2014-02-20 - v1.4   - Change default ioportal target to ftp.hdsioportal.com
#                          Adding plink.exe command at the beginning of every transfter to make sure not to have keyprint issue
#                          This could be the case if different users are starting the transfer
#                          Fixing file to spool list will only be added if exit code of scp is 0 which means sucessful
#    2014-06-17 - v1.5   - Adding multiple suffix support (=> 7zip support)
#    2014-12-26 - v1.6   - Adding log file output
#    2015-04-05 - v1.7   - Adding "log" to suffix list that "scriptlog" dir can be transfered
#                          Adding Sys::Hostname to get hostname of script
#    2015-04-20 - v1.8   - Changing default  $REMOTE_HOST to "ftp.hdsioportal.com"
#    2015-05-01 - v2.0   - Change from parameter to config file => pm file config::tiny needed
#                          Adding https via curl as transfer method
#                          Changing script name from HDSperftransfer.pl to HCHpptransfer.pl to meet swiss script standard
#    2015-05-01 - v2.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group
#                          Log format changed (LOG_MSG)
#                          Debuglevel, CURL parameter change 
#    2016-03-25 - v2.2.0 - adding LOG->autoflush(1)
#                        - adding pptransfer_proxyoptions
#                        - changing CURL syntax to pptransfer_proxyoptions
#                        - addicng check HTTPS_PROXYUSER / HTTPS_PROXYPWD (@ and & not allowed)
#                        - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays
#    2016-04-02 - v2.2.1 - Change log output CURL RUN USER PWD  = ***

bin/ppmonitor.pl
#    MODIFICATIONS
#    2014-01-03 - v1.0   - initial script
#    2014-06-16 - v1.1   - Moving up to folder ppmonitor. 
#                          Adding .zip as suffix to check
#                          Adding pptransfer checks (-i). 
#    2014-04-07 - v1.2   - Adding Sys::Hostname to get hostname of script
#    2015-05-02 - v1.3   - Changing script name from ppmonitor.pl to ppmonitor.pl to meet swiss script standard
#                          Adding CURL transfer check
#    2015-05-10 - v1.4.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group
#    2016-03-25 - v1.5.0 - LOG->autoflush(1), fix logfile checks
#                        - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays

lib/modules/HiCHpp_RAIDCOM.pm
#    MODIFICATIONS
#    2014-02-07 - v1.0   - initial module
#    2014-02-11 - v1.1   - adding -CLI to raidscan
#    2014-02-14 - v1.2   - adding CHECK_OLD_INSTANCE
#    2014-02-16 - v1.3   - add an output file of raidcom which is parsable with semicolon as delimiter
#                          This is quite difficult as the current delimiter is a space and the values can have spaces also.
#                          Not every output of a values is known therefore there are a lot of guesses if strings,digits,etc. 
#                          The following values has been testes and they can have one or more spaces in their output:
#                          meta_resource, pool name, host group name, HMO
#                          For now it looks it is working but troubleshooting within the regex might be needed. 
#                          If you experience issue please let me know.
#    2014-02-21 - v1.4   - add raidcom collection of parity groups
#    2014-04-07 - v1.5   - adding -fx to every collection
#                        - add raidcom get pool, raidcom get dp_pool -key opt
#    2014-05-22 - v1.6   - Adding -m at raidcom collection to get all MUs for raidscan.
#                          But only for TC and not for SI as the collection would take too long.
#    2015-04-02 - v1.7   - Adding -f at HDSperfcollect to use FC device instead of IP command device
#    2015-05-02 - v1.8   - Changing module name from HDSPERF_RAIDCOM.pm to HCHpp_RAIDCOM.pm to meet swiss script standard
#    2015-05-05            Bugfix (missing 3 in in CDM file for G1000 when using FC command device)
#    2015-05-05 - v1.9   - add raidcom level to disable some collections.
#                          0 = collect all, 1 = disable raidscan, 2 = disable raidscan and host_grp
#                          change log function
#    2015-07-03 - v2.0.0 - adding raidcom support for G800,G600,G400,G200$ (GUM IPs instead of SVP IP for these systems)
#    2016-03-25 - v2.1.0 - adding raidcom get dp_pool, fixing CMD_RAIDCOM_GET_POOL_OPT, changing randcom from 200 to 500 for horcm instance
#                        - delete old horcm log dir of horcm instance
#                        - adding 2 / 3 options for CMD_RAIDCOM_GET_POOL_OPT and CMD_RAIDCOM_GET_DPPOOL