Installing Linux over the SAN with an IBM subsystem device driver

Use this process to install Linux over the SAN with an IBM subsystem device driver (SDD).

This procedure assumes that you have correctly configured the bootloader to boot from the single-pathed SAN device. It also assumes that the SDD rpm is installed on your system. This procedure describes how to copy the SDD files into the initial ramdisk (initrd) and edit the linuxrc script, which is processed when the kernel mounts the initial ramdisk at boot time. For current information about the SDD rpm, see the IBM^® System Storage™ Multipath Subsystem Device Driver User's Guide, at http://www-03.ibm.com/servers/storage/support/software/sdd.

Perform the following steps to install Red Hat and SUSE with an IBM SDD:

Make a backup of the existing _initrd_file_ by typing the following commands:
```
cd /boot
cp _initrd_file_ _initrd_file.bak
```
Uncompress the image by typing the following command:
```
zcat _initrd_file_.bak > _initrd_file_
```
Set up a loopback device for the image by typing the following command:
```
losetup /dev/loop0 /path/to/your/_initrd_file_
```
Fix any errors that might exist on the file system by typing the following command:
```
e2fsck -f /dev/loop0
```
Determine the size of the initrd file system by typing the following command:
```
df /dev/loop0
```
Ensure that you have sufficient space in the /boot directory (or other home of your initrd files) to store considerably larger initrd files (for example, files with a size of 32 MB each). If there is not sufficient space in your /boot directory, you can perform the following steps in a temporary directory and then copy the compressed initrd file (a few megabytes instead of 32 MB) back into the /boot directory.

If the file system is not 32 MB or larger or does not have much free space, you must enlarge it to 32 MB by typing the following command:
```
losetup -d /dev/loop0
dd if=/dev/zero of=_initrd_file_ seek=33554432 count=1bs=1
```
Note: If the file is already 32 MB or larger, do not perform this step because it is unneeded and can corrupt the initial ramdisk file system.
On SUSE, you might need to create an even larger initial ramdisk (for example, 48 MB or 48×1024×1024 would result in a seek=50331648). If the initrd file is sufficiently large, skip ahead to mount the loopback device. See step 9.
Set up a loopback device for the image by typing the following command:
```
losetup /dev/loop0 /path/to/your/_initrd_file_
```
Ensure that you have a clean file system by again typing the following command:
```
e2fsck -f /dev/loop0
```
If you still have errors on the file system, the previous dd step was not performed correctly and it corrupted the initrd. You now must delete the loopback device by typing losetup -d /dev/loop0 and restart the procedure from the beginning.
Resize the file system by typing the following command:
```
resize2fs /dev/loop0
```
Note: Resizing automatically expands the file system so that it uses all available space.
Mount the loopback device by typing the following command:
```
mount /dev/loop0 /mnt/tmp
```
You now have an initrd file system that is 32 MB. You can add additional files by typing the following command:
```
cd /mnt/tmp
```
If you have not already added your host adapter driver to the initrd file using the standard mk_initrd or mkinitrd process (depending on your distribution), you must manually copy the module files for the host adapter driver. You also must manually copy the SCSI core and SCSI disk drivers into the initrd file system and add the appropriate insmod command to the linuxrc script.
On SUSE, you must create the etc/, proc/, and sysroot/ directories in the initrd file system.
You can also add echo commands into the linuxrc script after the host adapter load and mounting /proc to force the addition of LUNs through /proc/scsi/scsi if the device discovery is not occurring automatically.
Create the SDD directories in the initrd file system by typing the following commands:
```
mkdir -p opt/IBMsdd/bin
chmod -R 640 opt/IBMsdd/
```

For the SDD, you must copy the following files to the initrd file system.

Note: Ensure that you are in the /mnt/tmp directory when you perform the copies.

File Names	Target Location
/etc/vpath.conf	etc/
/etc/group	etc/
/etc/passwd	etc/
/etc/nsswitch.conf	etc/
/opt/IBMsdd/sdd-mod.o-CORRECT_VERSION	lib/sdd-mod.o
/opt/IBMsdd/bin/*	opt/IBMsdd/bin/
/lib/libc.so.6	lib/
/lib/ld-linux.so.2	lib/
/lib/libacl.so.1	lib/
/lib/libattr.so.1	lib/
/lib/libdl.so.2	lib/
/lib/libm.so.6	lib/
/lib/libpthread.so.0	lib/
/lib/libnss_files.so.2	lib/
/lib/librt.so.1	lib/
/bin/awk, chmod, chown, cp, date, grep, ls, mknod, mount, ps, rm, sed, sh, tar, unmount	bin/
/dev/sd[a-z], sd[a-z][a-z] For example, tar cps /dev/sd[a-z] /dev/sd[a-z][a-z]\| tar xps	dev/

For Red Hat, you must copy the following additional files to the file system:
File Names Target Location

/lib/libproc.so.2.0.7 lib/

/lib/libpcre.so.0 lib/

/lib/libtermcap.so.2 lib/

/bin/ash.static bin/ash

File Names	Target Location
/lib/libproc.so.2.0.7	lib/
/lib/libpcre.so.0	lib/
/lib/libtermcap.so.2	lib/
/bin/ash.static	bin/ash

For SUSE, you must copy the following additional files to the file system:

File Names	Target Location
/lib/libreadline.so.4	lib/
/lib/libhistory.so.4	lib/
/lib/libncurses.so.5	lib/
etc/nsswitch.conf Note: The etc/nsswitch.conf file must have its password and group entries changed to point to `files` instead of `compat`.	N/A

The following changes must be make to the initrd linuxrc script:
1. For Red Hat, remove the following block of commands from the end of the file:
```
echo Creating block devices
mkdevices /dev
echo Creating root device
mkroot dev /dev/root
echo 0x0100 > /proc/sys/kernel/real-root-dev
echo Mounting root filesystem
mount -o defaults -ro - -t ext2 /dev/root /sysroot
pivot_root /sysroot /sysroot/initrd
unmount /initrd/proc
```
  You must change the first line of the linuxrc script to invoke the ash shell instead of the nash shell.
2. If the /proc file system is not already explicitly mounted in the linuxrc script, append the following mount command:
```
mount -n -tproc /proc /proc
```
3. To configure SDD, append the following commands to the end of the linuxrc script:
```
insmod /lib/sdd-mod.o
/opt/IBMsdd/bin/cfgvpath
```
  Mount the systems root file system so that you can copy configuration information to it. For example, if you have an ext3 root file system on /dev/vpatha3, enter /bin/mount -o rw -t ext3, or /dev/vpatha3 /sysroot, or for a reiserfs root file system on /dev/vpatha3, enter /bin/mount -o rw -t reiserfs /dev/vpatha3 /sysroot
  To copy the dynamically created device special files onto the systems root file system, enter the following commands:
```
tar cps /dev/IBMsdd /dev/vpath* | (cd /sysroot && tar xps)
/bin/umount /sysroot
```
  You must define the root file system to the kernel. Traditionally, this information is passed to the bootloader as a string, for example /dev/vpatha3 and translated to a hexadecimal representation of the major and the minor numbers of the device. If the major and minor numbers equals 254,3, these numbers are represented in hex as 0xFE03. The linuxrc script passes the hexadecimal value into /proc with the following commands:
```
echo 0xFE03 > /proc/sys/kernel/real-root-dev
/bin/umount /proc
```
Edit the system fstab and change all the system mount points from LABEL or /dev/sd mount sources to their equivalent /dev/vpath. Refer to step 23 for dangers of booting by label in a multipath configuration.
Copy the system fstab to the initrd etc/ directory.
Unmount the image and remove the loopback binding by typing the following commands:
```
umount /mnt/tmp
losetup -d /dev/loop0
```

Compress the image by typing the following commands:

gzip -9 _initrd_file_
mv _initrd_file_.gz _initrd_file_

Append the following to your boot parameters (for example, in lilo.conf, grub.conf, or menu.lst):
```
ramdisk_size=34000
```
If you created a larger initrd file system, make this value large enough to cover the size.
For completeness, change the bootloader append for the root parameter of the kernel to the appropriate vpath device. For example, root=/dev/vpatha5. However, the previous steps override this value by passing the corresponding hex major, minor into the /proc file system within the initrd linuxrc script.
Note: If you boot by LABEL, there is a risk that the first device that is found in the fabric with the correct label could be the wrong device or that it is an sd single-pathed device instead of a vpath multipathed device.
Reboot the server. It will boot with the root file system on a vpath device instead of on an sd device.

Figure 1 and Figure 2 illustrate a complete linuxrc file for Red Hat and for SUSE.

Figure 1. Example of a complete linuxrc file for Red Hat

#!/bin/ash

echo "Loading scsi_mod.o module"
insmod /lib/scsi_mod.o 
echo "Loading sd_mod.o module"
insmod /lib/sd_mod.o 
echo "Loading qla2310.o module"
insmod /lib/qla2310.o 
echo "Loading jbd.o module"
insmod /lib/jbd.o 
echo "Loading ext3.o module"
insmod /lib/ext3.o 
echo Mounting /proc filesystem
mount -t proc /proc /proc
insmod /lib/sdd-mod.o
/opt/IBMsdd/bin/cfgvpath
/bin/mount -o rw -t ext3 /dev/vpatha3 /sysroot
tar cps /dev/IBMsdd /dev/vpath* | (cd /sysroot && tar xps)
/bin/umount /sysroot
echo 0xFE03 > /proc/sys/kernel/real-root-dev
/bin/umount /proc

Figure 2. Example of a complete linuxrc file for SUSE

#! /bin/ash

export PATH=/sbin:/bin:/usr/bin

# check for SCSI parameters in /proc/cmdline
mount -n -tproc none /proc
for p in `cat /proc/cmdline` ; do
  case $p in
    scsi*|*_scsi_*|llun_blklst=*|max_report_luns=*)
      extra_scsi_params="$extra_scsi_params $p"
      ;;
  esac
done
umount -n /proc

echo "Loading kernel/drivers/scsi/scsi_mod.o $extra_scsi_params"
insmod /lib/modules/2.4.21-190-smp/kernel/drivers/scsi/scsi_mod.o $extra_scsi_params

echo "Loading kernel/drivers/scsi/sd_mod.o"
insmod /lib/modules/2.4.21-190-smp/kernel/drivers/scsi/sd_mod.o

echo "Loading kernel/drivers/scsi/qla2310_conf.o"
insmod /lib/modules/2.4.21-190-smp/kernel/drivers/scsi/qla2310_conf.o

echo "Loading kernel/drivers/scsi/qla2310.o"
insmod /lib/modules/2.4.21-190-smp/kernel/drivers/scsi/qla2310.o

echo "Loading kernel/drivers/scsi/aic7xxx/aic7xxx.o"
insmod /lib/modules/2.4.21-190-smp/kernel/drivers/scsi/aic7xxx/aic7xxx.o

echo "Loading kernel/fs/reiserfs/reiserfs.o"
insmod /lib/modules/2.4.21-190-smp/kernel/fs/reiserfs/reiserfs.o
mount -t proc /proc /proc
insmod /lib/sdd-mod.o
/opt/IBMsdd/bin/cfgvpath
/bin/mount -o rw -t reiserfs /dev/vpatha3 /sysroot
tar cps /dev/IBMsdd /dev/vpath* | (cd /sysroot && tar xps)
/bin/umount /sysroot
echo 0xFE03 > /proc/sys/kernel/real-root-dev
/bin/umount /proc