SCSI disk limitations: Defining the number of disk devices on Linux

This section provides information to help you understand SCSI disk limitations.

The Linux kernel uses a static major and minor number address mechanism. There are a limited number of major numbers that the kernel reserves for SCSI disk devices. Thus, there are limitations to the number of SCSI disk devices that can exist depending on the major numbers available. The number of major numbers made available for SCSI disk devices varies depending on the distribution kernel level.

For Linux 2.4 kernels, the maximum number of configurable devices that are available on supported Linux host systems is either 128, 256, or 2304. Eight bits are reserved for minor numbers, thus there are 28, or 256, minor numbers available for each major number. By convention, for SCSI disks 16 minor numbers are reserved for each major number, thus there are up to 15 allowable partitions per disk. With 16 minor numbers used to describe each disk and its partitions, and 256 possible minor numbers, then there are 256 ÷ 16 = 16 possible disks that can be described by a single major number.

Traditionally, there have been eight major numbers reserved for SCSI devices. Those major numbers are 8, 65, 66, 67, 68, 79, 70, and 71. RHEL 2.1 and RHEL3 use the 8 reserved major numbers. Thus, with 8 reserved major numbers, and 16 possible disks per major number, these kernels can describe up to 8 x 16 = 128 possible disks. The earlier SuSE SLES8 kernels can support up to 256 devices because they reserve 16 major numbers. The additional major numbers are 128, 129, 130, 131, 132, 133, 134, and 135.

Newer kernels are moving to a dynamic configuration method that will use any available major numbers after the first 16 reserved major numbers have been used. The latest SuSE SLES 8 kernel errata includes this change and imposes a limit of no more than 144 major numbers allocated to SCSI disk devices. Thus these kernels can support up to 2304 disk devices.

In general, the following formula provides the maximum number of devices for Linux host systems: Number of devices = (number of major numbers) x (number of minor numbers) ÷ (number of partitions). For example, Number of devices = 8 x 256 ÷ 16 = 128 or Number of devices = 144 x 256 ÷ 16 = 2304.

For the Linux 2.6 kernels the number of major and minor bits has been increased to 12 and 20 bits respectively, thus Linux 2.6 kernels can support thousands of disks. There is still the limitation of only up to 15 partitions per disk.

Other factors that limit the number of disk devices

If you load a host adapter driver as a module in Linux, there is a kernel limit to the total number of disks that you can configure. This limit might be less than the total supported by the kernel (usually 128 or 256). The first module that is loaded on the system might configure discovered disks up to the maximum number of disks supported by the kernel. Subsequent drivers are limited to configuring a fixed, smaller number of drives. These drivers all share a pool of device structures that are statically allocated after the first host adapter driver is loaded. The number of allocated device structures is set with the CONFIG_SD_EXTRA_DEVS parameter when the kernel compiles. Traditionally, this value has been 40, but newer Red Hat kernels have it set to 128 or 256.

For example, consider a system with an internal SCSI Adaptec host adapter and one disk. The system contains fibre connected disks and a CONFIG_SD_EXTRA_DEVS parameter set to 40. The Adaptec host adapter driver loads first and configures the single attached disk. At this point, the system is limited to a total of 41 SCSI disks instead of the full 128 or 256. If there are more than 40 disks visible in the fibre channel fabric, the system configures and uses only the first 40. All of the fabric devices might be listed in the /proc/scsi/scsi directory, but only the first 40 are configured with the SCSI disk driver, scsi_mod. You are more likely to encounter this constraint in multipath fabric situations.

For the later SuSE SLES8 kernels that support 2304 SCSI disks, the CONFIG_SD_EXTRA_DEVS parameter was replaced with the CONFIG_SD_MAX_MAJORS parameter with default value of 144. For Linux 2.6 kernels, both these parameters are obsolete and are no longer needed.

Library | Support | Terms of use | Feedback
© Copyright IBM Corporation 2004, 2007. All Rights Reserved.