Contents
1 Introduction
This document describes how to configure HP BladeSystem c7000 to be used in the Cloud Execution Environment (CEE).
This document is to be used as part of one of the following procedures:
- An overall installation flow, refer to the guide CEE Installation.
- A region expansion flow, refer to HP c7000 Server HW Expansion.
This document supports the reference CEE configuration as described in document BOM for Certified HW Configurations, Reference [1].
This document has steps that must be repeated per enclosure (up to three enclosures), and per blade (up to 16 blades per enclosure). The following figure depicts the overall HP BladeSystem c7000 configuration flow.
|
Section |
Action |
Repeat |
|---|---|---|
|
Configure HP Enclosure |
||
|
Set Enclosure IP Addresses |
| |
|
Configure OA |
| |
|
| ||
|
Configure Virtual Connect |
| |
|
Post-Installation Activities |
Allow several hours to configure a 48-blade system (over 1800 steps), if all prerequisites are in place.
1.1 Terminology
The terms used in this document to describe resources are listed in the following table, with corresponding terms that occur in the system dialog.
|
Term |
Corresponding Term in System Dialog |
|---|---|
|
Enclosure |
Shelf |
|
Blade |
Server |
|
Server number |
Bay number |
CIDR (Classless Inter-Domain Routing) is a term used when calculating IP address and netmask values.
1.2 Prerequisites
This section describes the prerequisites that must be fulfilled before configuring the HP BladeSystem c7000.
1.2.1 Documents
Before starting this procedure, ensure that the following documents have been read and understood:
The following document is used in the procedure:
1.2.2 Tools
The following tools are required:
1.2.3 Installation Data
A customer-specific IP and VLAN plan, based on Reference [2], is required.
The address variables used in the document IP and VLAN plan, Reference [2], are used throughout this document, and are summarized in Table 3.
|
VLAN |
Variable Name |
Number of IP Addresses Used |
|---|---|---|
|
<Onboard Adm. (static) IP1> |
3 | |
|
<Onboard Adm. (static) IP2> |
3 | |
|
subrack_ctrl_sp |
<iLO (static)> |
48(1) |
|
subrack_ctrl_sp |
<VC Flex (static)> |
6 |
|
subrack_ctrl_sp |
<Traffic_switch VRRP (static)> |
1 |
|
subrack_ctrl_sp |
<Netmask subrack_ctrl_sp> |
1(2) |
(1) Static IP addresses for HP visual management interface
(2) The CIDR must be used to calculate the IP address
and netmask values to use
Other site-specific data is listed in Table 4.
|
Resource |
Variable Name |
Number of Variables Used |
|---|---|---|
|
Primary OA IP address |
<primary_oa_ip> |
3 (one per enclosure) |
|
CEE Region |
<cee_region_name> |
1 |
|
Enclosure |
<enclosure_id> |
3 |
|
The factory default username for system |
Administrator(1) |
|
|
The factory default password for the primary OA |
<password_primary_oa> |
|
|
The factory default password for the secondary OA |
<password_secondary_oa> |
|
|
The factory default password for Virtual Connect 1 |
<password_vc_1> |
|
|
The factory default password for Virtual Connect 2 |
<password_vc_2> |
|
|
<username_oa> |
3 (one per enclosure) | |
|
Password for the new OA user |
<password_for_username_oa> |
3 (one per enclosure) |
|
New Virtual Connect username(6) |
<username_vc> |
3 (one per enclosure) |
|
Password for the new Virtual Connect user |
<password_for_username_vc> |
3 (one per enclosure) |
(1) Factory default username
for the primary OA, secondary OA, virtual connect 1, and virtual connect
2
(2) The same username
is used with different passwords at the different hardware units.
(3) This password can be found on the pull-out stickers attached to the
OA.
(4) This password can be found on the Virtual Connect label.
(5) The same user is used on both primary
and secondary OA.
(6) The same user is used
on both Virtual Connect 1 and 2.
1.2.4 Conditions
This document assumes that the HP BladeSystem c7000 enclosure has not been configured before, or has been restored to factory default. Use the command SET FACTORY to reset to factory default. Refer to HP BladeSystem Onboard Administrator Command-Line Interface User Guide, Reference [3].
2 Configuration of HP BladeSystem c7000 Enclosure
This section describes how to configure the HP BladeSystem c7000 enclosure
2.1 Configure HP Enclosure
The HP enclosure configuration covered in this section is a manual procedure.
This section must be done for all enclosures.
- Note:
- The guide is for a fully equipped three enclosure system.
2.1.1 Set Enclosure IP Addresses
For all enclosures, use the front panel display, which is called Insight Display in HP documentation.
Set Enclosure OA1 IP Address:
- Use the arrow keys and move to Enclosure Settings, and press OK.
- Move to OA1 IPv6 and press OK.
- Move to OA1 IPv4 and press OK.
- Select Static Network Mode and press Accept.
- Set the network values for the enclosure using the arrow
keys:
IP Address: <Onboard Adm. (static) IP1>
netmask: <Netmask subrack_ctrl_sp>
gateway: <Traffic_switch VRRP (static)>
- Move to Accept and press OK.
Set Enclosure OA2 IP Address:
- Move to OA1 IPv6 and press OK.
- Move to OA2 IPv4 and press OK.
- Select Static Network Mode and press Accept.
- Set the network values for the enclosure with help of
the arrow keys:
IP Address: <Onboard Adm. (static) IP2>
netmask: <Netmask subrack_ctrl_sp>
gateway: <Traffic_switch VRRP (static)>
- Move to Accept and press OK.
- Move to Accept All and press OK.
- When all the enclosure IP addresses are set, go to Section 2.1.2.
2.1.2 Configure OA
For all enclosures, do the following:
- Identify the active OA. The active OA is indicated by the green Active indicator, see Figure 1.
- Disconnect the RJ45 cable from the active OA in the enclosure.
- Connect the kickstart server RJ45 cable to the active OA in the enclosure.
- Log on to the active OA from the kickstart server as user
Administrator with factory password for the active OA of the enclosure.
ssh Administrator@<Onboard Adm. (static) IP1>|⇒ <Onboard Adm. (static) IP2>
A hash symbol is displayed: #
- Verify that you have reached active OA by checking that the printout, when logging on, includes the following text: OA role active.
- Create a new OA user
with new password by using the following commands:
- add user <username_oa>
New Password:
Enter <password_for_username_oa>
Confirm
Enter <password_for_username_oa> again.
- Note:
- This new user can be used at both the primary and the secondary OA.
- Provide administrator rights to the new user:
set user access <username_oa> Administrator - assign server 1,2,3,4,5,6,7,8,9,10,11,12,13,14⇒
,15,16 <username_oa> - assign interconnect 1,2,7,8 <username_oa>
- assign oa <username_oa>
- add user <username_oa>
- Disable automatic boot for new hardware:
set server powerdelay all nopoweron
- Set the system name for the enclosure:
set enclosure name <cee_region_name>_enc<enclosure_id>
- Configure ILO:
set ebipa server <iLO (static) for enclosure> <Netmask subrack_ctrl_sp> 1-16
- Note:
- For the first enclosure, take the first IP address in range/subnet.
For the second enclosure, take the 17th IP address in the range.
For the third enclosure, take the 33rd IP address in the range.
Entering anything other than 'YES' will result in the command not executing.
Changing the IP address for device (iLO) bays that are enabled
causes the iLOs in those bays to be reset.
Are you sure you want to change the IP address for the specified
device (iLO) bays?Enter YES
enable ebipa server 1-16
Entering anything other than 'YES' will result in the command not executing.
Changing the IP address for device (iLO) bays that are enabled
causes the iLOs in those bays to be reset.
Are you sure you want to change the IP address for the specified
device (iLO) bays?Enter YES
set ebipa interconnect <VC Flex (static) per enclosure> <Netmask subrack_ctrl_sp> 1-2
- Note:
- Index VC Flex per enclosure. For a system with three enclosures, use the first, the third, and the fifth IP addresses.
Entering anything other than 'YES' will result in the command not executing.
It may take each interconnect several minutes to acquire the new settings.
Are you sure you want to change the IP address for the specified
interconnect bays?Enter YES
enable ebipa interconnect 1-2
Entering anything other than 'YES' will result in the command not executing.
If the interconnects received IP addresses from an external DHCP server,
the EBIPA settings will not take effect until the external DHCP server lease expires.
Are you sure you want to enable EBIPA for the interconnect bays?Enter YES
- Power-off all blades
poweroff server all force
- Continue with Section 2.2.
2.2 Configure BIOS
To configure the blade BIOS, do the following:
- Use HP c7000 Server BIOS Configuration and perform all the configuration steps described in that instruction. It includes procedures for configuring the following:
- When performed the configuration steps described in HP c7000 Server BIOS Configuration, continue with Section 2.3.
2.3 Configure Virtual Connect
The HP Virtual Connect configuration covered in this section is a manual procedure.
This section must be done for all enclosures.
Do the following:
- Disconnect the RJ45 cable in active OA in enclosure.
- Connect kickstart server RJ45 cable to active OA in enclosure.
- Log on to the active OA from the kickstart server as user <username_oa> (created in Step 6 in Section 2.1.2) with the password for the user.
The active OA is indicated by the green Active indicator.
ssh <username_oa>@<primary_oa_ip in first enclosure>
password
<password_for_username_oa>
- Connect to Virtual Connect:
connect interconnect <bay_number>
- Note:
- The bay number can be identified from the output of the show interconnect list command.
The output can be:
ERROR: Connection lost : The command session may have expired
Ignore the warning, and continue at the prompt:
<site specific unit number> login:
Enter Administrator
Password:
<password_vc_1>
- Reset to factory default:
delete domain
If factory default is set, the output is:
ERROR: Enclosure not imported, No enclosures currently exist in the domain. One
or more enclosures must be imported into the domain before additional
management operations are allowed. Enclosures can be imported using the
'import enclosure' command.If that output is seen, go directly to Step 6.
If Virtual Connect has been configured before, the following scenario happens:
WARNING: Deleting the domain will remove the entire virtual connect domain
configuration and reset it back to the original defaults. After the domain has
been deleted, you will also be logged out and the virtual connect manager will
reset.
Are you sure you want to continue? (yes/no) :Enter yes
SUCCESS: Domain deleted : You will now be logged out and the virtual connect
manager will reset. It may take a few minutes for the virtual connect manager
to become available again. - Reconnect to Virtual
Connect:
- Note:
- As stated in previous printout, It may take a few minutes for the virtual connect manager to become available again.
connect interconnect <bay_number>
The output can be:
ERROR: Connection lost : The command session may have expired
Ignore the warning, and continue at the prompt:
<site specific unit number> login:
Enter Administrator
Password:
<password_vc_1>
- Import the enclosure:
import enclosure username=<username_oa> password=<password_for_username_oa>
Importing enclosure, please wait
SUCCESS: Enclosure imported- Note:
- This takes about five minutes.
- Create new user <username_vc>:
add user <username_vc> password=<password_for_username_vc> privileges=domain,network,server
SUCCESS: User added: <username_vc>
set user <username_vc> fullname="Administrator" enabled=true
SUCCESS: User modified: <username_vc>
- Note:
- This new user can be used at both Virtual Connect 1 and 2.
- Set a name on the Domain:
set domain name=<cee_region_name>
SUCCESS: Domain settings modified
- Create control network and add two uplink ports to it:
set domain MacType=VC-Defined MacPool=<enclosure_id + 1>
SUCCESS: Domain settings modified
set domain WwnType=Factory-Default
SUCCESS: Domain settings modified
set mac-cache Enabled=true Refresh=1
SUCCESS: MAC cache failover settings modified
set igmp Enabled=false
SUCCESS: IGMP settings modified
set enet-vlan -quiet VlanCapacity=Expanded
SUCCESS: Ethernet settings modified
set statistics-throughput -quiet Enabled=true SampleRate=5m
SUCCESS: Throughput statistics settings have been modified.
set port-protect networkLoop=Enabled
SUCCESS: Ethernet networkLoop or pauseFlood Protection settings modified
set port-protect pauseFlood=Enabled
SUCCESS: Ethernet networkLoop or pauseFlood Protection settings modified
set lacp-timer Default=Short
SUCCESS: LACP Timer domain default modified
set advanced-networking -quiet PacketBufferOverallocationRatio=1 FlowControl=auto
SUCCESS: Advanced Networking parameters modified
set link-dist-interval Interval=30
SUCCESS: FC Link Re-Distribution Interval modified
- Create uplink set:
add uplinkset X460_SUS ConnectionMode=Auto LacpTimer=Domain-Default
SUCCESS: Shared uplink port set added: X460_SUS
add uplinkport enc0:1:X1 UplinkSet=X460_SUS Speed=Auto
SUCCESS: Port added: enc0:1:X1
add uplinkport enc0:2:X1 UplinkSet=X460_SUS Speed=Auto
SUCCESS: Port added: enc0:2:X1
- Note:
- One can add more ports (X3 to X8) if they are connected.
- Create VLANs:
add network fuel_ctrl_sp -quiet UplinkSet=X460_SUS VLanID=28 NAGs=Default NativeVLAN=Enabled Color=green
SUCCESS: Network added: fuel_ctrl_sp
set network fuel_ctrl_sp SmartLink=Disabled
SUCCESS: Network modified: fuel_ctrl_sp
add network cee_ctrl_sp -quiet UplinkSet=X460_SUS VLanID=2 NAGs=Default
SUCCESS: Network added: cee_ctrl_sp
set network cee_ctrl_sp SmartLink=Disabled
SUCCESS: Network modified: cee_ctrl_sp
add network subrack_ctrl_sp -quiet UplinkSet=X460_SUS VLanID=3 NAGs=Default
SUCCESS: Network added: subrack_ctrl_sp
set network subrack_ctrl_sp SmartLink=Disabled
SUCCESS: Network modified: subrack_ctrl_sp
- Limit the link speeds to 1 Gb:
set enet-vlan PrefSpeedType=Custom PrefSpeed=1000⇒ MaxSpeedType=Custom MaxSpeed=1000 SUCCESS: Ethernet settings modified
- For each blade in every enclosure:
- Create blade profile:
add profile Profile_enc<enclosure_id>_Bay-<Server_Number> -NoDefaultEnetConn -NoDefaultFcConn -NoDefaultFcoeConn NAG=Default
SUCCESS: Profile added: Profile_enc<enclosure_id>_Bay-<Server_Number>
add enet-connection Profile_enc<enclosure_id>_Bay-<Server_Number> Network=Unassigned PXE=Enabled
SUCCESS: Connection added to server profile: Profile_enc<enclosure_id>_Bay-<Server_Number>
add enet-connection Profile_enc<enclosure_id>_Bay-<Server_Number> Network=Unassigned PXE=Disabled
SUCCESS: Connection added to server profile: Profile_enc<enclosure_id>_Bay-<Server_Number>
- Connect server profile to fuel network:
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:1 fuel_ctrl_sp Untagged=true
SUCCESS: Server port mapping added
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:2 fuel_ctrl_sp Untagged=true
SUCCESS: Server port mapping added
- Connect server profile to control network:
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:1 cee_ctrl_sp VLanID=<cee_ctrl_sp>
SUCCESS: Server port mapping added
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:2 cee_ctrl_sp VLanID=<cee_ctrl_sp>
SUCCESS: Server port mapping added
- Connect server profile to subrack network:
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:1 subrack_ctrl_sp VLanID=subrack_ctrl_sp
SUCCESS: Server port mapping added
add server-port-map Profile_enc<enclosure_id>_Bay-<Server_Number>:2 subrack_ctrl_sp VLanID=subrack_ctrl_sp
SUCCESS: Server port mapping added
- Create blade profile:
- For each blade 1–16 in every enclosure 1–3:
- Assign server profile to blade:
assign profile Profile_enc<enclosure_id>_Bay-<Server_Number> enc<enclosure_id>:<Server_Number>
SUCCESS: Profile 'Profile_enc<enclosure_id>_Bay-<Server_Number>' assigned to device bay enc<enclosure_id>:<Server_Number>
- Assign server profile to blade:
- Disconnect switch connection with Ctrl+Underscore:
Ctrl+Shift+Minus
Command: D)isconnect, C)hange settings, send B)reak, E)xit command mode X)modem send >
Enter D
- Disconnect from primary OA:
exit
- Disconnect the RJ45 cable in active OA in first enclosure.
- Connect kickstart server RJ45 cable to active OA in first enclosure.
- Continue with Section 3
3 Post-installation Activities
3.1 Post-Installation Test
Verify HP Enclosure Configuration
Do for all enclosures:
- Log on to active OA from kickstart server as <username_oa> with password for the user.
Use the primary OA IP for the first enclosure.
ssh <username_oa>@<primary_oa_ip>
<username_oa>@<primary_oa_ip>'s password:
Enter <password_for_username_oa>
A hash symbol is displayed: #
- Note:
- If not the active OA is reached, the output contains the
following text:
OA Role: Standby
Locate the active OA indicated by the green Active indicator, as shown in Figure 1.
- Print configuration:
show config
- Note:
- A long printout is shown, store it for future reference.
- Exit from OA:
exit
Verify Virtual Connect Configuration
- Log on to active OA from kickstart server as <username_oa> with password for the user.
Use the primary OA IP for the first enclosure.
ssh <username_oa>@<primary_oa_ip>
<username_oa>@<primary_oa_ip>'s password:
Enter password_for_username_oa
A hash symbol is displayed: #
- Note:
- If not the active OA is reached, the output contains the
following text:
OA Role: Standby
Locate the active OA indicated by the green Active indicator, as shown in Figure 1.
- Connect to Virtual Connect:
connect interconnect 1
Press [Enter] to display the switch console:
Press Enter twice
- Print configuration:
show config
- Note:
- A printout is shown, verify that your changes in enclosure settings are shown.
- Exit switch with Ctrl+Underscore:
Ctrl+Shift+Minus
Command: D)isconnect, C)hange settings, send B)reak, E)xit command mode X)modem send >
Enter D
- Exit from OA:
exit
- Disconnect the RJ45 cable that was connected from kickstart server in Step 2 in Section 2.3.
- Connect the RJ45 cable that was disconnected in Step 1 in Section 2.3.
3.2 Enable NTP in HP BladeSystem c7000 Onboard Administrator
- Connect RJ45 cable to the control switch if not already connected.
- Log on to the active OA from the kickstart server as <username_oa> with password for the user.
The active OA is indicated by the green Active indicator.
ssh Administrator@<primary_oa_ip for first enclosure>A hash symbol is displayed: #
- Verify that you have reached active, printout when logging on includes: OA role active.
- Issue:
enable ntp - Disconnect from primary OA:
exit - Remove kickstart RJ45 cable from the control switch that was inserted in Step 1.
Reference List
| [1] BOM for Certified HW Configurations, 1/006 51-CSA 113 125/5 Uen |
| [2] IP and VLAN plan, 2/102 62-CRA 119 1862/5 Uen |
| [3] HP BladeSystem Onboard Administrator Command Line Interface User Guide, http://www.hp.com/. |