Recovering a node and returning it to the cluster using the CLI

After a node or an I/O group fails, you can use the command-line interface (CLI) to recover a node and return it to the cluster.

1. Issue the following command to verify that the node is offline:

   svcinfo lsnode

2. Issue the following command to remove the old instance of the offline node from the cluster:

   svctask rmnode nodename/id

   Where nodename/id is the name or ID of the node.
3. Issue the following command to verify that the node can be seen on the fabric:

   svcinfo lsnodecandidate

   Note: Remember the worldwide node names (WWNNs) for each node because you will need them in the following step.
4. If the nodes are repaired by replacing the service controller, or the node is replaced, be sure to follow the replacement instructions for the specific node or controller. You will be instructed to reset the WWNN of the node to that of the original node. If you do not do that, you may need to reconfigure your SAN fabric, your hosts, and your storage systems.
   Attention: If more than one I/O group is affected, ensure that you are adding the node to the same I/O group from which it was removed. Failure to do this can result in data corruption. Use the information that was recorded when the node was originally added to the cluster. This can avoid a possible data corruption exposure if the node must be removed from and re-added to the cluster. If you do not have access to this information, call the IBM® Support Center to add the node back into the cluster without corrupting the data. If you are adding the node into the cluster for the first time, you must record the following information:

   • Node serial number
   • WWNN
   • All WWPNs
   • I/O group that contains the node
5. Issue the following command to add the node back into the cluster:

   svctask addnode -wwnodename WWNN -iogrp
    IOGRPNAME/ID [-name NODENAME]

   Where WWNN is the worldwide node name, IOGRPNAME/ID is the I/O group name or ID and NODENAME is the name of the node.

   In a service situation, a node should normally be added back into a cluster using the original node name. As long as the partner node in the I/O group has not been deleted too, this is the default name used if -name is not specified.

6. Issue the following command to verify that the node is online:

   svcinfo lsnode