Installation and Reference Guide Device Mapper Multipath Enablement Kit for HP StorageWorks Disk Arrays

Version 4.0.0

Part number: AA-RW8RD-TE First edition: February 2008
Copyright 2008 Hewlett-Packard Development Company, L.P. Condential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.21 and 1 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendors standard commercial license. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Red Hat is the registered trademark of Red Hat, Inc. SuSE is the registered trademark of Novell, Inc.

Contents

About this guide
Intended audience. Document conventions and symbols. HP technical support. Helpful websites.
1 Introduction to HPDM Multipath.
Overview. Features. Whats New. Software Components. HPDM Multipath Operation..
2 Installing HPDM Multipath
HPDM Multipath Support Matrix. Prerequisites for HPDM Multipath Installation. Installing/Upgrading HPDM Multipath Tools. Setting up HPDM Multipath. Conguring QLogic HBA Parameters Conguring Emulex HBA Parameters

9 10. 1 1. 1 1. 12

3 Conguring HPDM Multipath.
Using /etc/multipath.conf File. Starting HPDM Multipath.

4 Using HPDM Multipath.

Creating an Alias using /etc/multipath.conf le. Using Multipath Map. Path Grouping. Clustering Solutions with HPDM Multipath. Conguring HP Service Guard with HPDM Multipath Devices. Conguring SteelEye LifeKeeper with HPDM Multipath Devices.
5 Removing HPDM Multipath Tools
Removing HPDM Multipath Tools.
6 Troubleshooting. 7 Abbreviations.
Device Mapper Multipath Enablement Kit for HP StorageWorks Disk Arrays

Tables

1 Document conventions. 2 Hardware and Software Prerequisites. 3 HP Recommended conguration parameters 5 Abbreviations 29
4 Basic operations of HPDM Multipath.
HP Device Mapper Multipath (HPDM Multipath) software kit is an HP released Device Mapper multipath binary based on Device Mapper included in Linux operating system distributions with enablement for HP StorageWorks Disk Arrays. This guide describes the Device Mapper multipath enablement for HP StorageWorks Disk Arrays (HPDM Multipath) and provides information to help you: Determine hardware and software prerequisites Install HPDM Multipath Congure HPDM Multipath Use HPDM Multipath Troubleshoot HPDM Multipath
NOTE: This version of HPDM Multipath supports all the features available in the earlier versions. It also supports a range of new features and OS. For more information, see What's New.

Intended audience

This document is intended for users who install, congure, and manage HPDM Multipath in their Linux server environment. Readers must be familiar with Linux system administration, including hardware and software installation.

Document conventions and symbols
Table 1 Document conventions Convention
Medium blue, underlined text (http://www.hp.com) Bold font Italic font

Element

Website addresses Key names Text typed into a GUI element, such as into a box GUI elements that are clicked or selected, such as menu and list items, buttons, and check boxes Text emphasis File and directory names System output Code Text typed at the command line

Monospace font

Monospace, italic font Monospace, bold font
Code variables Command-line variables Emphasis of le and directory names, system output, code, and text typed at the command line
IMPORTANT: Provides clarifying information or specic instructions.
NOTE: Provides additional information.

HP technical support

Telephone numbers for worldwide technical support are listed on the HP support website: http://www.hp.com/support/ Collect the following information before calling: Technical support registration number (if applicable) Product serial numbers Product model names and numbers Applicable error messages Operating system type and revision level Detailed, specic questions
For continuous quality improvement, calls may be recorded or monitored. HP recommends that customers sign up online using the Subscribers choice website: http://www.hp.com/go/e-updates Subscribing to this service provides you with e-mail updates on the latest product enhancements, newer versions of drivers, and rmware documentation updates as well as instant access to numerous other product resources. After signing up, you can locate your products by selecting Business support and then Storage under Product Category.

Helpful websites

For additional product information, see the following websites: http://www.hp.com/country/us/eng/prodserv/storage.html http://www.hp.com/support/ http://www.docs.hp.com http://sourceware.org/dm/ http://download.qlogic.com/drivers/56660/readme_driver_80107.html#driver_parameters
1 Introduction to HPDM Multipath
This chapter addresses the following topics: Overview Features What's New Software Components HPDM Multipath Operation

Overview

HPDM Multipath software kit is an HP released Device Mapper multipath binary, based on Device Mapper Multipath included in Linux Operating System distributions, with enablement for HP StorageWorks Disk Arrays. Device Mapper is an infrastructure in the Linux kernel. It provides a generic way to create virtual layers of block devices. It supports striping, mirroring, snapshots, concatenation, and multipathing. The multipath feature is provided with combination of DM Multipath kernel modules and multipath-tools user-space package.

Features

HPDM Multipath supports the following features: Detects path failure and automatically reroutes (failover) I/O to an available alternate path, if an existing connection fails. Provides an option to perform failback of the LUN to the repaired paths. Implements failover or failback actions transparently without disrupting applications. Monitors each path and noties if the path status changes. Facilitates I/O load balancing among the multiple paths. Provides CLI with display options to congure and manage Multipath features. Provides an option to customize names for the HPDM Multipath devices. Provides device name persistence across reboots or conguration SAN changes. Provides policy-based path grouping for the user to customize the I/O ow through specic set of paths. Provides online LUN addition and deletion support. Provides support for the HP StorageWorks Arrays (EVA-AA, MSA-AA, and XP) with latest rmware revisions. Provides multipathing in clustered environment. Provides support for multipathing devices to be congured under Logical Volume Manager.

Whats New

HPDM Multipath provides the following additional features as part of this version: Provides support for RHEL4.5/RHEL4.6/RHEL5.1/SLES9 SP4 and SLES10 SP1 along with RHEL5/SLES9 SP3/SLES10 OS distributions. Provides support for HP StorageWorks EVA4400 Disk Arrays. Provides support for HP StorageWorks MSA1000/1500 Disk Arrays. Provides support for HP StorageWorks MSA2000 Product family Disk Arrays (MSA2012fc/MSA2212fc/MSA2012i).
Provides support for XP20000/XP24000 Disk Arrays. Provides support for EVA iSCSI devices on RHEL5/RHEL5.1/SLES9 SP3/SLES9 SP4/SLES10/SLES10 SP1.

Software Components

This section discusses the following software components of Device Mapper Multipath: dm-multipath kernel module -- Routes I/O and provides failover to paths and path groups. multipath conguration tool -- Provides commands to congure, list, and ush Multipath devices. multipathd daemon -- Monitors path status. When paths revert, multipathd daemon may also initiate path group switches to ensure that the optimal path group is used. kpartx utility -- Reads partition tables on the specied device and creates device maps over the detected partitions. The kpartx utility is called from hotplug whenever device maps are created and deleted. devmap-name -- Provides a meaningful device name to udev for device maps (devmaps).

HPDM Multipath Operation

HPDM Multipath enables hosts to route I/O over the multiple paths available to an end storage unit (LUN). A path refers to the connection from an HBA port to a storage controller port. When an active path through which I/O happens fails, HPDM Multipath reroutes the I/O over other available paths. On a Linux host, when there are multiple paths to a storage controller, each path appears as a separate block device and hence results in multiple block devices for single LUN. HPDM Multipath creates a new Multipath block device for those devices that have the same LUN WWN. For example, a host with two HBAs when attached to a storage controller with two ports through a single FC switch provides four block devices: /dev/sda, /dev/sdb, /dev/sdc, and /dev/sdd. HPDM Multipath creates a single block device, /dev/mapper/mpath1, that reroutes I/O through these four underlying block devices.
Introduction to HPDM Multipath
This chapter addresses the following topics: HPDM Multipath Support Matrix Installing HPDM Multipath Tools Setting Up HPDM Multipath
HPDM Multipath Support Matrix
Table 2 lists the hardware and software prerequisites for installing HPDM Multipath. Table 2 Hardware and Software Prerequisites
System Feature Supported hardware and software Red Hat Enterprise Linux 4 Update 5 Red Hat Enterprise Linux 4 Update 6 Red Hat Enterprise Linux 5 errata 2.6.188.1.6.eI5 or later Red Hat Enterprise Linux 5 Update 1 SuSE LINUX Enterprise Server 9 - Service Pack 3 errata 2.6.57.283 or later SuSE LINUX Enterprise Server 9 - Service Pack 4 SuSE LINUX Enterprise Server 10 SuSE LINUX Enterprise Server 10 - Service Pack 1 Qlogic: FCA2214 FCA2214DC A7538A A6826A FC1 142SR FC1242SR FCHost Bus Adapters (HBA) FC1243 AB379A QMH 2462 Emulex: A8002A/LPeA8003A/LPeAD167A AD168A Servers PCI-E PCI-E PCI-X 2.0 PCI-X 2.0 1-port 4Gbps 2-port 4Gbps 1-port 4Gbps 2-port 4Gbps Fibre Channel Fibre Channel Fibre Channel Fibre Channel PCI-X PCI-X PCI-X PCI-X PCI-E PCI-E PCI-X PCI-X PCI-X 1-port 2Gbps 2-port 2Gbps 1-port 2Gbps 2-port 2Gbps 1-port 4Gbps 2-port 4Gbps 1-port 4Gbps 2-port 4Gbps 2-port 4Gbps Fibre Channel Fibre Channel Fibre Channel Fibre Channel Fibre Channel Fibre Channel Fibre Channel Fibre Channel Fibre Channel
Operating system versions
4 Gbps Mezzanine for HP Blade Servers
HP BladeSystem c-Class Server Blades, ProLiant x86, ProLiant AMD64, ProLiant EM64T Servers, Integrity Servers

Supported arrays

EVA 3000 (HSV101) VCS 4.007 or later EVA 5000 (HSV1 1) VCS 4.007 or later 1 EVA 4000 (HSV200) XCS 5.1 10/6.100 or later EVA 6000 (HSV200) XCS 5.1 10/6.100 or later EVA 8000 (HSV210) XCS 5.1 10/6.100 or later EVA 4100 (HSV200) XCS 6.1 or later 10 EVA 6100 (HSV200) XCS 6.1 or later 10 EVA 8100 (HSV210) XCS 6.1 or later 10 EVA 4400 (HSV300) XCS 0900 or later XP128* fw rev 21-14-18-00/00 or later XP1024* fw rev 21-14-18-00/00 or later XP10000 fw rev 50-07-30-00/00 or later XP12000 fw rev 50-09-34-00/00 or later XP 20000 fw rev 60-02-04-00/00 or later XP 24000 fw rev 60-02-04-00/00 or later MSA 1000/MSA 1500 fw rev 7.0.0 or later MSA 2000 Storage product family (MSA2012fc/MSA2212fc/MSA2012i) *If you use XP128/1024, the system modes 140 and 293 must be switched on, otherwise the XP does not respond to a SCSI inquiry to code page 0x83 with a unique serial number (scsi_id tool). This prevents the HPDM Multipath from detecting the redundant paths. Provides support for iSCSI devices presented from supported EVA disk arrays. Emulex: 8.0.16.32 or later (for SLES9/RHEL 4.5/RHEL 4.6), 8.1.6.8 or later (for SLES10), 8.1.10.1 or later (for RHEL5/RHEL5.1/SLES10 SP1) available at: 1 http://h18006.www1.hp.com/products/storageworks/4gbpciehba/index.html Qlogic: Version 8.01.07.25 or later for all OS distributions available at: http://h18006.www1.hp.com/products/storageworks/fca2214/index.html

HBA drivers

NOTE: On any single host, the HBAs must be from the same HBA vendor. Does not support coexistence with other multipath products. Does not support Active-Passive Storage Arrays. EVA iSCSI devices are not supported on RHEL 4U5 and RHEL 4U6.
Prerequisites for HPDM Multipath Installation
Ensure the following RPMs are installed on the system before proceeding with HPDM Multipath installation: For RHEL 5/RHEL 5.1: libsysfs-devel, device-mapper >= 1.02.02-2, libselinux-devel, libsepol-devel, readline-devel, ncurses-devel For RHEL 4.6: sysfsutils-devel, device-mapper, libaio-devel For RHEL 4.5: sysfsutils-devel, device-mapper For SLES10 SP1: device-mapper, device-mapper-devel, readline-devel, sysfsutils For SLES10: device-mapper, readline-devel, sysfsutils For SLES9 SP3 /SLES9 SP4: readline-devel, device-mapper, ncurses-devel
Installing HPDM Multipath
For all OS: aaa_base acl attr bash bind-utils bison bzip2 coreutils cpio cpp cracklib cvs cyrus-sasl db devs diffutils e2fsprogs file filesystem fillup findutils flex gawk gdbm-devel glibc glibc-devel glibc-locale gpm grep groff gzip info insserv kbd less libacl libattr libgcc libselinux libstdc++ libxcrypt m4 make man mktemp module-init-tools ncurses net-tools netcfg openldap2-client openssl pam pam-modules patch permissions popt procinfo procps psmisc pwdutils rcs readline sed strace syslogd sysvinit tar tcpd texinfo timezone unzip util-linux vim zlib zlib-devel autoconf automake binutils gcc gdbm gettext libtool perl rpm termcap udev
Installing/Upgrading HPDM Multipath Tools
To install or upgrade HPDM Multipath tools, complete the following steps: 1. Download the HPDM Multipath tools installation package from the following HP website at: http://www.hp.com 2. Log in as root to the host system. 3. Copy the installation tar package to a temporary directory (for example, /tmp/HPDMmultipath). 4. To unbundle the package, enter the following commands: # cd /tmp/HPDMmultipath # tar -xvzf HPDMmultipath-4.0.0.tar.gz # cd HPDMmultipath-4.0.0 5. Verify that the directory contains the INSTALL.sh shell script, the SRPMS, README.txt, and the docs directories. 6. To install or upgrade HPDM Multipath tools software on the server, enter the following command: #./INSTALL.sh 7. Follow the on-screen instructions to complete the installation. 8. When the installation is complete, enter the following command: #rpm -qa |grep multipath 9. Verify that the output lists the HPDMmultipath-tools-4.0.0-<OS> package.
Setting up HPDM Multipath
Setting up HPDM Multipath involves the following: Conguring QLogic HBA Parameters Conguring Emulex HBA parameters
Conguring QLogic HBA Parameters
You must congure the QLogic HBA parameters for HPDM Multipath. The HBA time outs are typically set up for non HPDM Multipath environments, where longer time outs are necessary. This is because the only alternative is to send an error to the application. However, with Multipath, errors such as cable failures must be intimated at the earliest so that the Multipath layer can quickly take action and redirect the I/O to another path. To congure the QLogic HBA parameter, complete the following steps: 1. For QLogic 2xxx family of HBAs, edit the /etc/modprobe.conf le in RHEL4 or RHEL5 hosts and /etc/modprobe.conf.local le in SLES9/SLES10 hosts with the following values:

options qla2xxx qlport_down_retry=10 ql2xfailover=0
2. Rebuild the initrd by executing the following script:
/opt/hp/src/hp_qla2x00src/make_initrd

3. Reboot the host.

Conguring Emulex HBA Parameters
To congure the Emulex HBA parameter, complete the following steps: 1. For Emulex lpfc family of HBAs: In RHEL4 hosts, edit the /etc/modprobe.conf le with the following values:
options lpfc lpfc_nodev_tmo=14 lpfc_lun_queue_depth=16 lpfc_discovery_threads=32
In SLES9 hosts, edit the /etc/modprobe.conf.local le with the following values:
In SLES10 hosts, edit the /etc/modprobe.conf le with the following values:
options lpfc lpfc_nodev_tmo=28 lpfc_lun_queue_depth=16 lpfc_discovery_threads=32
In RHEL5 hosts edit the /etc/modprobe.conf le with the following values:
/opt/hp/hp-lpfc/make_initrd
3. Reboot the host. NOTE: Ensure you have a HBA driver without multipathing capability installed. For more information, see the installation and reference guides for the respective HBA drivers.
3 Conguring HPDM Multipath
This chapter addresses the following topics: Using /etc/multipath.conf File Starting HPDM Multipath
Using /etc/multipath.conf File
The rst step of conguring HPDM Multipath involves editing the /etc/multipath.conf conguration le, which is distributed as part of the HPDM Multipath tools user space package. This le also enables you to customize HPDM Multipath for a variety of SAN storage subsystems. The multipath.conf le provides the following options to congure the attributes of a multipath device: I/O grouping policy polling interval for subsequent path status verication path selector algorithm program used by multipath to obtain a unique path identier The HPDM Multipath package comes with a template le with the recommended conguration for HP supported arrays. To use this template le: For RHEL systems: 1. Copy the /usr/share/doc/HPDMmultipath-tools-4.0.0/multipath.conf.HPTemplate le as /etc/multipath.conf Copy the /usr/share/doc/packages/HPDMmultipath-tools-4.0.0/ multipath.conf.HPTemplate le as /etc/multipath.conf

For SLES systems: 1.

The /etc/multipath.conf le consists of the following sections, to congure the attributes of a multipath device: System defaults (defaults) Black-listed devices (devnode_blacklist/blacklist) Storage array model settings (devices) Multipath device settings (multipaths) Blacklist exceptions (blacklist_exceptions)
The defaults section denes default values for attributes which are used whenever required setting is unavailable. The blacklist section denes which devices should be excluded from the multipath topology discovery. The blacklist_exceptions section denes which devices should be included in the multipath topology discovery, despite being listed in the blacklist section. The multipaths section denes the multipath topologies. They are indexed by a World Wide Identier (wwid). The devices section denes the device-specic settings based on vendor and product values. Table 3 lists the important attributes of HP supported arrays.

Table 3 HP Recommended conguration parameters
Attribute Description Supported values multibus: All valid paths in one priority group group_by_prio: One priority group per path priority value tur round-robin 0 immediate /bin/true /sbin/mpath_prio_alua %n uniform n=12
path_grouping_policy path_checker path_selector failback prio_callout rr_weight no_path_retry
Used for applying the policy to the multipath device hosted by this storage controller Used for determining the state of the path Used to select the path selector algorithm to be used for mpath. These algorithms are offered by the kernel mpath target Used to manage the time during path group failback Executable to obtain a path weight for a block device. Weights are summed for each path group to determine the next path group to be used in case of path failure Used to assign weights to the path (n =12) indicates the number of retries until queuing is disabled (queues till n number of polling), fail indicates immediate failure (no queuing), or queue indicates never stop queuing (queue forever till the path comes alive)
Example 1. Recommended Values for HP Arrays The product string of the storage controller contains 16 characters (including spaces) and the vendor string contains 8 characters (including spaces) in the /etc/multipath.conf le. To ease the task of manually editing the devices section of the multipath.conf le, the wild card, regular expression support for the product and vendor string is provided. For example, in /etc/multipath.conf le, the product string for the HSV101 arrays is given the value of HSV101*, instead of HSV101 \(C\)COMPAQ. For EVA3000-(HSV101)
device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } "HP" "HSV101 \(C\)COMPAQ" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua %n" uniform 100 immediate 12

Conguring HPDM Multipath

For EVA4000/6000 - (HSV200)
device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } "HP" "HSV200" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua %n" uniform 100 immediate 12

For EVA5000-(HSV1 1) 1

device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } "COMPAQ" "HSV111 \(C\)COMPAQ" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua /dev/%n" uniform 100 immediate 12

For EVA8000 (HSV210)

device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } "HP "HSV210" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua /dev/%n" uniform 100 immediate 12

For EVA4400 (HSV300)

device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } "HP "HSV300" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua /dev/%n" uniform 100 immediate 12

For MSA A/A arrays

device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight failback hardware_handler no_path_retry rr_min_io } "HP "MSA VOLUME*" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua /dev/%n" uniform immediate "0" 12 100
For MSA2000 Storage Product Family arrays
device { vendor product path_grouping_policy getuid_callout path_selector rr_weight prio_callout path_checker hardware_handler failback no_path_retry rr_min_io } "HP "MSA2[02]*" multibus "/sbin/scsi_id -g -u -s /block/%n" "round-robin 0" uniform "/bin/true" tur "0" immediate 12 100

For XP arrays

device { vendor product path_grouping_policy getuid_callout rr_weight rr_min_io path_checker failback no_path_retry } "HP "OPEN-*" multibus "/sbin/scsi_id -g -u -s /block/%n" uniform 100 tur immediate 12
NOTE: For MSA 2000 Storage Product Family arrays (MSA2012fc/MSA2212fc/MSA2012i, add the device section in /etc/multipath.conf le. For HP recommended device parameters, see For MSA2000 Storage Product Family arrays.
NOTE: In XP arrays, there are different LUNs, such as OPEN-<x>, 3390-3A, 3390-3B, OP-C:3390-3C, 3380KA, 3380-KB, and OP-C:3380-KC where x = {3,8,9,K,T,E,V}. The product strings for XP LUNs are based on these emulation types. A new device section must be added for each emulation type, because each product string requires a new device subsection. OPEN-* is sufcient for the product string for all the XP LUNs with different OPEN emulations, because regular expressions are supported in the /etc/multipath.conf le.

IMPORTANT: After editing the /etc/multipath.conf le for HP StorageWorks Arrays, you must restart multipathd daemon and run multipath command. For more information, see Starting HPDM Multipath.

Starting HPDM Multipath

Whenever an attribute for the LUNs of any of the HP supported storage arrays changes, the multipathd daemon must be restarted and the conguration must be reloaded by executing the following commands: # /etc/init.d/multipathd restart # /sbin/multipath These steps are required only if the attributes are modied. To view the status of the multipath devices, complete the following steps: 1. Run multipath utility. Use the # /sbin/multipath -ll option. For information on status maps in multipath -ll for HPDM Multipath devices, see Using HPDM Multipath.

4 Using HPDM Multipath

This chapter addresses the following topics: Creating an Alias using /etc/multipath.conf le Using Multipath Map Supporting Clustering Solutions
Creating an Alias using /etc/multipath.conf le
This section describes various parameters available for updating /etc/multipath.conf le. The multipath devices are created in the /dev/mapper directory in the hosts. These devices are similar to any other block devices present in the host, and are used for any block or le level I/O operations, such as creating the le system. You must use the devices under /dev/mapper/. You can create a new device alias by using the alias and the WWID attributes of the multipath device present in the multipath subsection of the/etc/mutipath.conf le. Example 2. Updating /etc/multipath.conf le For example, when the following subsection is added in the /etc/multipath.conf le for the LUN with WWID 3600508b30090f5d0d2a9d64590490022, a multipath device, mydatadisk1, is created under /dev/mapper:
multipaths { multipath wwid alias path_grouping_policy path_checker path_selector 3600508b30090f5d0d2a9d64590490022 mydatadisk1 multibus tur "round-robin 0"
You can use /dev/mapper/mydatadisk1 like any other block device.

Using Multipath Map

This section describes a multipath map obtained by executing the multipathll command, based on a sample multipath.conf le. Following is a sample multipath.conf le (for RHEL4) with parameters to support EVA 4000 as a storage controller:

defaults {

udev_dir polling_interval selector path_grouping_policy getuid_callout prio_callout path_checker rr_min_io rr_weight failback no_path_retry user_friendly_names } multipaths { multipath { } multipath { } multipath { } } devices { device { vendor product path_grouping_policy getuid_callout path_checker path_selector prio_callout rr_weight rr_min_io failback no_path_retry } device { } }. wwid path_grouping_policy path_checker path_selector
/dev 5 "round-robin 0" failover "/sbin/scsi_id -g -u -s /block/%n" "/bin/true" tur 1000 uniform immediate 12 no
3600508b30090f5d0d2a9d64590490022 multibus tur "round-robin 0"

for other target

"HP "HSV200" group_by_prio "/sbin/scsi_id -g -u -s /block/%n" tur "round-robin 0" "/sbin/mpath_prio_alua %n" uniform 100 immediate 60
for targets from other storage controllers

Using HPDM Multipath

For example, in a typical conguration, a Linux host with a dual port HBA is connected to an EVA 4000 through two switches. In this case, if all the paths are available, the host has four I/O paths for any LUN presented from the EVA 4000. The multipath.conf le enables the host to view the multipath map, as follows:
3600508b30090f5d0cf46bd1a4c420023 [size=1 GB][features="1 \_ round-robin \_ 3:0:1:3 sdad \_ 2:0:2:3 sdo \_ round-robin \_ 2:0:1:3 sdj \_ 3:0:0:3 sdy 3600508b30090f5d0d2a9d64590490022 [size=3 GB][features="1 \_ round-robin \_ 3:0:2:4 sdaj \_ 3:0:3:4 sdao \_ 2:0:0:4 sdf \_ 2:0:3:4 sdu queue_if_no_path"][hwhandler="0"] 0 [prio=100][active] 65:208 [active][ready] 8:224 [active][ready] 0 [prio=20][enabled] 8:144 [active][ready] 65:128 [active][ready] queue_if_no_path"][hwhandler="0"] 0 [prio=120][active] 66:48 [active][ready] 66:128 [active][ready] 8:80 [active][ready] 65:64 [active][ready]

Path Grouping

The information in the map is presented by grouping the paths for a LUN with unique identiers, such as UID/WWN.The size, features, and the corresponding hwhandlers are displayed followed by the unique LUN identier. The grouping of paths is done and presented in a map based on the I/O load balancing policy. In the above example, for the LUN 3600508b30090f5d0d2a9d64590490022, the grouping is done with the policy as multibus and the devices, such as sdaj, sdao, sdf, and sdu belong to same path group. The state of the group is either active or enabled depending on whether I/O is active or not. For the LUN 3600508b30090f5d0cf46bd1a4c420023, the grouping is done with the policy as group_by_prio. The devices sdad and sdo belong to one path group, and the devices sdj and sdy belong to a different path group, because they are from two controllers with different serial numbers. The I/O always happens in the path group which is active. If all paths in the active group fails, the failover occurs to the other path group that is enabled on changing that path group to active. When the paths are up again and the failback parameter is set as immediate, the failback occurs to the earlier group and I/O occurs through the earlier group.

Example 3. Failover and Failback If group A has sdad and sdo, and group B has sdj and sdy, the failover occurs from group A to group B and the failback occurs from group B to group A. The state of the path is given as [ready] [active] if the path is up, and ready for I/O. If the path is down, this state is shown as [faulty] [failed]. The path states are updated periodically based on the polling interval set in the /etc/multipath.conf le. The path states are updated by the multipathd daemon. Table 4 lists the basic operations supported by the multipath CLI utility provided with HPDM Multipath. Table 4 Basic operations of HPDM Multipath
Command # multipath F # multipath d # multipath # multipath -l # multipath -ll # multipath v2 <device> # multipath v3 Description Deletes all HPDM Multipath devices. Displays potential paths, but does not create any device. Creates HPDM Multipath devices. Displays the list of device status. Displays the detailed list of device status. Congures and displays multipath map information. Congures and displays multipath map information.
Clustering Solutions with HPDM Multipath
HPDM Multipath supports the following clustering solutions: HP Serviceguard for Linux version 1 1.18 or later SteelEye Lifekeeper for Linux version 6.1.4 or later
Conguring HP Service Guard with HPDM Multipath Devices
HPDM Multipath devices can be used for data storage under Serviceguard protection. To congure HPDM Multipath devices under Serviceguard, complete the following steps: 1. Congure the SAN such that each host has multiple paths to the array. Present the LUNs to all the hosts. 2. See Conguring HPDM Multipath for conguring the LUNs for HPDM Multipath 3. Use the HPDM Multipath device to create the logical volumes that are used by Serviceguard for Linux. Please refer the HP ServiceGuard Compatibility matrix for the supported Operating Systems and the Arrays. NOTE: HPDM Multipath devices can be used with HP Serviceguard A.1 1.18 for Linux. The LVM volume groups must be congured on the HPDM Multipath devices. In HP Service Guard package conguration, the disk monitor must be congured using the HPDM Multipath device in the format /dev/dm-x. Please ensure that the device names (/dev/dm-x) are same across all the nodes. Conguration requirements HP Serviceguard for Linux are available in the certication matrix available at the following website: http://www.hp.com/info/sglx
Conguring SteelEye LifeKeeper with HPDM Multipath Devices
HPDM Multipath devices in the clustered environment can be used for le systems having Lifekeeper as the clustering solution. To congure HPDM Multipath devices, complete the following steps: 1. Congure the SAN such that each host has multiple paths to the array. Present the LUNs to all the hosts. 2. Any le systems on the LUNs must be mounted using the HPDM Multipath device. 3. See Conguring HPDM Multipath for conguring the LUNs for HPDM Multipath 4. Install and start the LifeKeeper for Linux with HPDM Multipath Recovery kit installed on all the hosts. 5. Create a Resource Hierarchy on LifeKeeper using the HPDM Multipath device. Please refer to SteelEye LifeKeeper documentation for the supported Operating Systems and the Arrays. http://licensing.steeleye.com/support/docm.php

This chapter addresses the following topic: Removing HPDM Multipath Tools
Removing HPDM Multipath Tools
You can remove HPDM Multipath tools by using the INSTALL.sh shell script. To remove HPDM Multipath tools from your system, complete the following steps: 1. Log in to the system as a root user. 2. From the directory containing the script INSTALL.sh, execute the #./INSTALL.sh command. 3. Follow the on-screen instructions. 4. Alternatively, you can also use rpm e HPDMmultipath-tools-4.0.0-<OS> to remove Multipath tools.

6 Troubleshooting

Following are the troubleshooting steps for issues that you may encounter while using HPDM Multipath: Enter the following commands whenever a new LUN is added or deleted at the Linux host, to update the multipath maps in the kernel: 1. /etc/init.d/multipathd restart 2. multipath -v <X> Enter the following commands whenever multipath devices are deleted using multipath -F, to create the multipath devices and to create the same for the partitions on the multipath devices: 1. multipath -v <X> 2. /sbin/dmsetup ls --target multipath --exec "/sbin/kpartx -a p part" (SLES10/ SLES10 SP1) /sbin/dmsetup ls --target multipath --exec "/sbin/kpartx a" (RHEL 4U5/RHEL 4U6/RHEL 5/RHEL 5U1) If SELinux is enabled, device maps cannot be created and SELinux denies access requested by /sbin/multipathd. NOTE: In some cases, labelling problems can cause SELinux denials. Try to restore the default system le context for bin by running the following commends: restorecon -v bin If this does not work, there is currently no automatic way to allow this access. Instead, you can generate a local policy module to allow this access - For more information, see the following website:http://fedora.redhat.com/docs/selinux-faq-fc5/#id2961385 You can also disable SELinux protection altogether. However, disabling SELinux protection is not recommended. If an existing LUN is deleted and a new LUN is presented in the same SCSI slot, LUN collision may occur. This results in the creation of a new LUN through old device special les. This may lead to data corruption. To recover from this error state, run following commands: 1. multipath -F 2. hp_rescan -a or echo "- - -" > /sys/class/scsi_host/<host instance>/scan 3. /etc/init.d/multipathd restart 4. /sbin/multipath -v3 Path status may not be reected immediately when it is recoved from FAILED to ACTIVE state. To reect the correct path state run the following commands. 1. /etc/init.d/multipathd restart 2. /sbin/multipath -v3 On hosts running SLES OS, if you do not enable boot.multipath to start at boot time, multipath maps are not created. To ensure the creation of device maps across reboots, enable boot.multipath using the following command: # chkconfig boot.multipath on

Troubleshooting

7 Abbreviations
Table 5 lists the abbreviations or terms used in this document. Table 5 Abbreviations
Abbreviations/Acronyms CLI HPDM GUI HBA I/O LUN LVM OS RAID RHEL SLES UID WWID WWN FC SCSI SAN Denition Command-Line Interpreter/Interface HP Device Mapper Graphical User Interface Host Bus Adapter Input/Output Logical Unit Number Logical Volume Manager Operating System Redundant Array of Independent (or Inexpensive) Disks Red Hat Enterprise Linux SuSE LINUX Enterprise Server Unique Identier World-Wide Identier World-Wide Name Fibre Channel Small Computer System Interface Storage Area Network

HP Smart Array Multipath Software User Guide
December 2005 (Fourth Edition) Part Number 354907-004
Copyright 2005 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendors standard commercial license. Microsoft, Windows, and Windows NT are U.S. registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group. Linux is a U.S. registered trademark of Linus Torvalds.

Audience assumptions

This document is for the person who installs, administers, and troubleshoots servers and storage systems. HP assumes you are qualified in the servicing of computer equipment and trained in recognizing hazards in products with hazardous energy levels.

Contents

Overview..... 5
Software overview..... 5 Software distribution..... 5 Minimum requirements.... 5 Supported operating systems..... 6 Microsoft feature support.... 6 Linux feature support.... 6 Multiple operating system support.... 6 Supported four-node configurations.... 6 Two-server external boot configuration.... 6 Two-server internal boot configuration... 7 Two-server mixed boot configuration... 7 Two-server cluster configuration.... 7 Three-server external boot configuration... 7 Three-server internal boot configuration.... 8 Three-server mixed boot configuration... 8
Installation and operation in Microsoft environments... 9
Pre-installation hardware setup for Microsoft.... 9 Uninstalling the Smart Array Multipath software driver... 10 Installation for an existing 2-node cluster with Microsoft Windows 2000... 10 Multipath installation for Microsoft.... 11 Post-installation hardware setup for Microsoft.... 11 Microsoft browser security settings.... 11 Static LUN balancing.... 12
Installation and operation in Red Hat Enterprise Linux 2.1 environments.. 14
Pre-installation configuration for Red Hat Enterprise Linux 2.1... 14 Editing the active kernel..... 14 Modifying the RAIDTAB configuration file.... 16 Multipath installation for Red Hat Enterprise Linux 2.1.... 19 Static load (read-only) balancing in Red Hat Enterprise Linux 2.1... 20
Installation and operation in Red Hat Enterprise Linux 3.0 and 4.0 environments. 21
Pre-installation hardware setup for Red Hat Enterprise Linux 3.0 and 4.0... 21 Multipath installation for Red Hat Enterprise Linux 3.0 and 4.0... 21 Post-installation configuration for Red Hat Enterprise Linux 3.0 and 4.0... 22 Static load (read-only) balancing in Red Hat Enterprise Linux 3.0 and 4.0.. 23
Installation and operation in UnitedLinux environments... 24
Pre-installation hardware setup for UnitedLinux.... 24 Multipath installation for UnitedLinux.... 24 Post-installation configuration for UnitedLinux.... 25 Static load (read-only) balancing in UnitedLinux... 26

Troubleshooting.... 27

Problem diagnosis..... 27 Problems during installation..... 27 HBA failure..... 28 Cable failure..... 28
Acronyms and abbreviations.... 29

Index..... 30

Overview

In this section

Software overview.... 5 Software distribution.... 5 Minimum requirements.... 5 Supported operating systems.... 6 Supported four-node configurations... 6

Software overview

Smart Array Multipath software delivers dual path functionality and supports failover capability for a redundant HBA in a server with redundant cabling to an HP StorageWorks Modular Smart Array 500 Generation 2 storage system. This fault-tolerant tool enhances availability in clustering and DAS solutions with up to two servers managing up to 2 TB of stored data. The software supports multiple I/O paths to the same logical volumes. When a path fails, the software moves the logical drives from the failed path to the path of the redundant HBA in the same server. Smart Array Multipath software works differently with Microsoft and Linux products to provide specific features that are compatible with operating system functionality.

Software distribution

Smart Array Multipath software is included with the 4-port U320 Shared Storage Module option. Smart Array Multipath software version 2.0B and later only contains Multipath I/O deliverables for Microsoft Windows operating systems and is only available for web download from the HP website (http://h18007.www1.hp.com/support/files/server). For Linux operating systems, refer to the procedures in this guide to configure the standard Linux drivers for multipath usage.

Minimum requirements

Be sure that your solution consists of the following hardware:
At least one server with two supported HBAs (Smart Array 642 adapters or future supported adapters) with updated firmware A supported operating system installed on the server or MSA500 G2 storage system An MSA500 G2 storage system with the following:
At least one HP StorageWorks Modular Smart Array 500 Generation 2 Controller A 2-Port Shared Storage Module and two SCSI cables or A 4-Port Shared Storage Module and four SCSI cables
For the latest versions of firmware, refer to the HP website (http://www.hp.com/support/proliantstorage).

Overview 5

Supported operating systems
Supported operating systems include:

Microsoft

Windows 2000 Windows Server 2003 Red Hat Enterprise Linux UnitedLinux
For information about the most current operating system support, errata kernels, and service pack requirements, refer to the HP website (http://www.hp.com/products/sharedstorage).
Microsoft feature support
Static LUN balancing enables the administrator to optimize I/O by assigning specific volumes to specific paths in the storage system. If one path fails, the storage system moves volumes to the path for the redundant HBA for I/O management. Server-based PCI Hot Plug functionality supports online HBA addition and replacement. Logical volume hot-add functionality supports online array expansion/extension. ProLiant Storage Manager with the ProLiant Multipath Manager software and device manager extension provide point-and-click multipath management. Configuration flexibility enables boot volumes for DAS solutions to reside on the storage system.

Linux feature support

Static load (read-only) balancing enables the HBAs to optimize I/O with dual paths to the same volumes in the storage system. If one HBA fails, the storage system moves volumes to the path for the redundant HBA for I/O management. Depending on the operating system, servers maintain boot volumes and enable greater storage capacity on the storage system.
Multiple operating system support
For multipath configurations with multiple operating systems, use SSP to restrict HBAs from accessing logical volumes containing data from a different operating system. For more information about SSP, refer to the documentation that ships with the 4-Port Shared Storage Module.

Supported four-node configurations
The following sections identify supported multipath configurations with their respective 4-Port Shared Storage Module connections and locations for boot and data volumes. These configurations require two or three servers attached with four SCSI cables to the MSA500 G2 storage system.
Two-server external boot configuration
All boot and data volumes reside on the MSA500 G2 storage system.

Overview 6

Server 1 2

Port (bus) A1 B1 A2 B2

Boot volume location Storage system* Storage system*
Data volume location Storage system Storage system
*Red Hat Enterprise Linux operating systems support boot volumes on servers only. For a mixed or external boot configuration in a multiple operating system environment, always locate Linux boot volumes on servers.
Two-server internal boot configuration
Boot volumes reside on their respective servers, while data volumes reside on the MSA500 G2 storage system.
Server Port (bus) A1 B1 A2 B2 Server 2 Storage system Boot volume location Server 1 Data volume location Storage system
Two-server mixed boot configuration
One boot volume resides on its respective server, while the other boot volume resides on the MSA500 G2 storage system. Both data volumes reside on the MSA500 G2 storage system.
Server Port (bus) A1 B1 A2 B2 *Red Hat Enterprise Linux operating systems support boot volumes on servers only. For a mixed or external boot configuration in a multiple operating system environment, always locate Linux boot volumes on servers. Server 2 Storage system Boot volume location Storage system* Data volume location Storage system
Two-server cluster configuration
Identical servers (hardware and software configuration) connect to the MSA500 G2 storage system. Boot volumes reside on their respective servers, while both data volumes reside on the storage system.
Three-server external boot configuration
All boot and data volumes reside on the MSA500 G2 storage system. One server connects to two ports.

Overview 7

Server 3
Boot volume location Storage system* Storage system* Storage system*
Data volume location Storage system Storage system Storage system
Three-server internal boot configuration
Boot volumes reside on their respective servers, while data volumes reside on the MSA500 G2 storage system. One server connects to two ports.

Server 3 Port (bus) A1 B1 A2 B2 Server 2 Server 3 Storage system Storage system Boot volume location Server 1 Data volume location Storage system
Three-server mixed boot configuration
Two boot volumes reside on their respective servers, while one boot volume resides on the MSA500 G2 storage system. All data volumes reside on the MSA500 G2 storage system. One server connects to two ports.
Server 3 Port (bus) A1 B1 A2 B2 Server 2 Server 3 Storage system Storage system Boot volume location Storage system* Data volume location Storage system

Overview 8

Installation and operation in Microsoft environments
Pre-installation hardware setup for Microsoft... 9 Uninstalling the Smart Array Multipath software driver... 10 Installation for an existing 2-node cluster with Microsoft Windows 2000.. 10 Multipath installation for Microsoft... 11 Post-installation hardware setup for Microsoft.... 11 Microsoft browser security settings.... 11 Static LUN balancing.... 12
Pre-installation hardware setup for Microsoft
To prepare for software installation:
Complete the MSA500 G2 storage system hardware installation:
NOTE: For hardware and option installation procedures, refer to the storage system user guide or the documentation that ships with the option.
a. Install the MSA500 G2 storage system. b. Install the 4-Port Shared Storage Module. c. Install the MSA500 G2 controllers. d. Update the system firmware, if needed. 2.
Complete the server installation:
a. Install all hardware, including a second supported HBA. Refer to the server user guide.
IMPORTANT: The server only supports one SCSI path during installation. If the server is already cabled for a multipath configuration, disconnect the SCSI cable connected from the redundant HBA to the storage system.
b. Install one of the supported operating systems. c. Reboot the server. 3. 4.
Install multipath cabling for one of the supported four-node configurations (on page 6). Be sure that only one SCSI cable is connected from the server to the MSA500 G2 storage system. During installation, the Smart Array Multipath software supports only one SCSI path per Microsoftbased server. Power up the storage system. Wait for the controller display to provide a Startup Complete message. Power up one of the servers.

Installation and operation in Microsoft environments 9
Uninstalling the Smart Array Multipath software driver
CAUTION: Before uninstalling the software driver in a Microsoft Windows Server 2003 clustering environment, use the Cluster Administrator to stop the cluster service on the appropriate node.

1. 2. 3. 4. 5. 6.

Power down the server. Disconnect all SCSI cables from any controller in the storage system, unless the boot drive is attached. Power up the server. Select Start>Settings>Control Panel. Select Add/Remove Programs. From the list of components, select HP Smart Array Multipath Software 2.0C.
NOTE: For Microsoft Windows Server 2003 only, the selection is HP Smart Array Multipath Driver.
Click Change/Remove. Click OK.
NOTE: A Microsoft Windows Server 2003 system does not indicate progress during the removal. Wait at least 10 minutes to allow the system to complete the removal.

Click Cancel.

10. Shut down the server. 11. Disconnect the SCSI cable from the redundant HBA.
Installation for an existing 2-node cluster with Microsoft Windows 2000
1. 2. 3. 4. 5. 6. 7. 8. 9.
Log on to one node of the cluster with an account having domain administrator privileges. Open the Cluster Administrator and move all cluster groups to the second node. On the first node, stop the cluster service and set the startup type to Manual. Install the HP Smart Array Multipath Software. Refer to "Multipath installation for Microsoft (on page 11)." Reboot as instructed. Log on to Windows 2000 with an account having domain administrator privileges. Click Yes or OK for any new hardware found or installed. Shut down the system. Do not reboot. Install the second HBA and connect the cable from second HBA to the storage system.
10. Power up the server. 11. Log in to Windows 2000 with an account having domain administrator privileges after the startup

completes.

12. Start the cluster service and set the startup type to Automatic. 13. Open the Cluster Administrator. 14. Move the disk groups from the second node to the first node and verify that all groups come online. 15. Repeat all steps for the second node.
Installation and operation in Microsoft environments 10
Multipath installation for Microsoft
CAUTION: Before installing Smart Array Multipath Software, be sure to uninstall any previous versions of the software and update the HBA and MSA500 G2 controller drivers.
Insert the Smart Array Multipath Software CD into the CD-ROM drive of the server.
IMPORTANT: If autorun is disabled, run setup.exe from the CD.

2. 3. 4.

To agree to the software license agreement, click Agree. When the Main screen displays, select Install Multipath Software. Click OK.
IMPORTANT: If HP web-based applications have not been installed on the system, the Web-based Management Setup Wizard displays. The wizard prompts you to provide an administrator login and password. Follow the on-screen instructions. The CD installs the Smart Array Multipath Manager, the ProLiant Storage Manager, and the redundancy driver.

5. 6. 7. 8. 9.

When the system prompts for a reboot, remove the CD then click Cancel. Shut down the server. Connect the SCSI cable to the redundant HBA. Power up the server. After the server boots, open a browser window and enter: https://localhost:2381 The system prompts you with a security certificate.
10. To accept the certificate and proceed to the login, select Yes. 11. Enter the administrator login and password.
The System Management Homepage displays.
12. Click the Settings tab. 13. Click Options. 14. In the Configuration Options section, select Local Access. 15. Click Save Configuration. 16. Repeat steps 1 through 15 for each server connected to the MSA500 G2 storage system.
Post-installation hardware setup for Microsoft
To prepare for software operation:

1. 2. 3. 4.

Reconnect the SCSI cable from the redundant HBA to the MSA500 G2 storage system. Cycle the server power. Repeat steps 1 and 2 for each server. Use the appropriate software for multipath operations.
Microsoft browser security settings
For Windows Server 2003, the modified version of Internet Explorer 6.0 defaults all servers to the Internet zone, which enables the highest level of security. At this security level, browsing to HP Management agents generates many security messages.
Installation and operation in Microsoft environments 11
To avoid these security messages:

1. 2. 3. 4. 5.

Open Internet Explorer 6.0. Select Tools>Internet Options. Select the Security tab. Click Local intranet. Manually set the servers to the Local intranet zone by entering: http://<servername or IP address>:2301 https://<servername or IP address>:2381

Static LUN balancing

The Smart Array Multipath software and Microsoft operating system support static LUN balancing. The movement of drives to specific paths can be done two ways:
Locally with a DME Remotely with the Smart Array Multipath found on the Smart Array Multipath Software CD From the desktop, right-click My Computer. Click Manage. The Device Manager window opens. Click SCSI and RAID Controllers. Right-click one of the array controller icons and click Properties. When the Properties window opens, click the Redundancy Configuration tab. From the Redundancy Configuration tab, the administrator can view the following controller information:
To balance LUNs with the DME:
Name Slot number Status Slot number of redundant controller Controller tree
In the controller tree, right-click one of the logical volumes and then click Switch Path. The system moves the logical volume to the other path. First-time users must install the software. Install the ProLiant Storage Manager software and Smart Array Multipath Manager software ("Multipath installation for Microsoft" on page 11). Browse to the System Management Homepage by entering: https:\\<name of server>:2381

To manage your redundant configuration:
IMPORTANT: If you encounter numerous security messages on a Microsoft Windows Server 2003 platform, review the Microsoft browser security settings (on page 11).

3. 4. 5. 6. 7.

To accept the security certificate, click Yes. The system may prompt for the administrator password established during the original software installation. The System Management Homepage opens. Click the Home tab. Select ProLiant Storage Manager. A new window opens. Click the appropriate controller on the Device View tab. Click Multipath Management. The Multipath Management and Configuration screen displays.
Installation and operation in Microsoft environments 12
From the Multipath Management and Configuration screen, do one of the following:
To move all drives from the current path to the redundant path, click the link for the controller and select Switch All Paths. The system moves all drives to the redundant path. To move a drive from one path to the redundant path, click the link for the drive and select Switch Path.
IMPORTANT: After establishing a redundant configuration and adding a logical volume, use Windows Disk Manager to detect the new volume.
Installation and operation in Microsoft environments 13
Installation and operation in Red Hat Enterprise Linux 2.1 environments
Pre-installation configuration for Red Hat Enterprise Linux 2.1... 14 Multipath installation for Red Hat Enterprise Linux 2.1... 19 Static load (read-only) balancing in Red Hat Enterprise Linux 2.1... 20
Pre-installation configuration for Red Hat Enterprise Linux 2.1
Before installing Smart Array Multipath software on a server with a Linux Red Hat operating system:
Create a new Linux kernel that recognizes the multipath hardware configuration. Modify the raidtab configuration file to be sure it contains the appropriate information.
Editing the active kernel
To configure the system for multipath operations, HP recommends the following steps:
Modify the makefile. Overwrite the active config file. Edit the.config and make the kernel. Insert the new kernel into the GRUB or LILO boot-loader.
The procedures in this section provide examples for all these basic steps. Alternatively, you can install the modules without rebuilding the kernel. IMPORTANT: Modules not installed in a kernel are not loaded at boot time, requiring you to remake modules after every power cycle or reboot. HP recommends that you install the modules into the kernel using the steps in this section.

Modifying the Makefile

IMPORTANT: The following procedure is for Linux Red Hat only. To modify the Makefile:
Identify the name of the active kernel:
a. At the root level of the kernel source tree (usually /usr/src/linux-xxx), enter:

uname -a

directory names or display possible options). Name the boot image vmlinuz-xxxx where xxxx is the kernel version number with the name you used in the very first step (ex., vmlinuz-2.4.9Emultipath). cp /usr/src/linux-xxx/arch/ixxx/boot/bzImage /boot/vmlinuz-xxxxx
b. Make the boot image bootable:
mkinitrd vmlinuz-xxxx xxxxx
c. Update the GRUB or LILO configuration file to include the new kernel, by doing one of the

following:

To update GRUB, edit /boot/grub/menu.lst in a text editor, copy one of the entries and change the values to match those of the new kernel boot image and the title GRUB will use to display. For LILO, edit the /etc/lilo.conf by copying one of the entries and changing the values to match those of the new kernel boot image and the title LILO will use to display. To apply the new configuration, run the following command: /sbin/lilo The new kernel is ready for use.
11. Reboot and select the new kernel at the GRUB or LILO prompt. If the new kernel does not work
properly, select the original kernel to boot from and remake the kernel or reconfigure the operating system.
Installing the module without modifying the kernel
Load the module. Enter: insmod multipath Verify that the module is installed and loaded by listing all modules. Enter: lsmod
The kernel is now ready for multipath operations, but to support multipath, you must enter these commands each time the server is restarted.
Modifying the RAIDTAB configuration file
To perform multipath operations with a Linux operating system, a raidtab configuration file must be present. Use the instructions in the following sections to be sure the raidtab configuration file contains the appropriate information.

RAIDTAB terms

Observe the following definitions for raidtab terms:
raiddevintroduces the configuration for a particular device nr-raid-disksthe number of raid disks in the array; specified in the /dev/cciss directory persistent-superblockmust be set to 1 if an md device is to be mounted/used during a reboot chunk-sizesets the stripe size in bytes, in the power of 2 devicethe physical device that makes up the RAID array raid-diskthe index of the device in the RAID array
For more information, refer to the operating system documentation.
Installation and operation in Red Hat Enterprise Linux 2.1 environments 16
Sample RAIDTAB configuration file
The following raidtab sample file is for a multipath configuration with three partitions (/dev/md0, /dev/md1, and /dev/md2) where each md device represents a partition on the device. If another partition is needed, add it as /dev/md3 with the same characteristics as the preceding partitions. The limit for number of partitions is four primary partitions. However, creating one of the four partitions as a type type 0xf Win 95 Extd (LBA) partition enables you to create extended logical partitions inside of the primary partition. Inside of the extended partition, you can create additional Linux partitions. The raidtab configuration file must reside in the /etc directory. The contents of /etc/raidtab should resemble the following:

raiddev /dev/md0 raid-level nr-raid-disks persistent-superblock chunk-size device raid-disk device raid-disk raiddev /dev/md1 raid-level nr-raid-disks persistent-superblock chunk-size device raid-disk device raid-disk raiddev /dev/md1 raid-level nr-raid-disks persistent-superblock chunk-size device raid-disk device raid-disk multipath 8 /dev/cciss/c0d2p/dev/cciss/c1d2pmultipath 8 /dev/cciss/c0d2p/dev/cciss/c1d2pmultipath 8 /dev/cciss/c0d2p/dev/cciss/c1d2p1 1
Completing the Linux multipath setup
After a valid raidtab file is created in the /etc directory, verify that the correct drivers and modules are loaded.
Verify that the md and multipath drivers are loaded, either statically or by a module load. Enter: lsmod
Installation and operation in Red Hat Enterprise Linux 2.1 environments 17
This command reveals which modules are running. Examining the boot-time messages reveals the static loaded drivers. IMPORTANT: If you have made configuration changes to the kernel (including the multipath or /dev/md driver) and have not rebooted from the new kernel, you must reboot before continuing. Red Hat typically includes the md driver in the kernel.
Verify that the md driver is in the kernel. Enter: dmesg | grep md | more This command shows the boot-time messages. If the md driver was loaded, several messages should appear, similar to the following messages: md: md driver version X.X md: Autodetecting RAID arrays md: autorun md: autorun DONE
If the multipath driver was built as a module but not loaded, load the module. Enter: insmod multipath This command loads the driver. If the driver does not load the multipath module, rebuild the module.
After verifying loaded drivers, make the raid files. For the sample partition table in the "Sample RAIDTAB Configuration File (on page 17) section, enter: mkraid /dev/md0 mkraid /dev/md1 mkraid /dev/md2 mkraid /dev/md3
CAUTION: Do not use Fdisk or mkraid to create any other partitions on this logical drive. This action results in data corruption and data loss.
If mkraid responds that the partition contains existing data, do the following:
a. Force the mkraid to run. Enter:

mkraid f

b. Make filesystems on the raid devices. Enter:
mkfs t ext2 /dev/md0 mkfs t ext2 /dev/md1 mkfs t ext2 /dev/md2 mkfs t ext2 /dev/md3 mkfs t ext2 /dev/md4 IMPORTANT: The example in step 5b uses ext2 because known bugs in the ext3 filesystem can cause data corruption.
The multipath RAID device should be functioning. To mount the device /dev/md0, enter: mount /dev/md0 /spare_filesystem If necessary, you can stop or start the RAID device with the following commands:
To stop the RAID device, enter: raidstop /dev/md0 This command may need to start the md device after a reboot. Place the command in the rc /etc configuration files for mounted filesystems at boot time.

Installation and operation in UnitedLinux environments
Pre-installation hardware setup for UnitedLinux... 24 Multipath installation for UnitedLinux.... 24 Post-installation configuration for UnitedLinux... 25 Static load (read-only) balancing in UnitedLinux.... 26
Pre-installation hardware setup for UnitedLinux
Multipath installation for UnitedLinux
Use the following procedure for supported versions of UnitedLinux operating systems. To use the Smart Array Multipath Software with a Linux operating system, download the required driver RPM. To download the driver RPM:

1. 2. 3.

Insert the Smart Array Multipath Software CD into the CD-ROM drive of the server. Open a console prompt. Mount the CD-ROM.
Installation and operation in UnitedLinux environments 24

4. 5. 6. 7. 8.

Change to the Linux directory. List the contents of the directory and select the proper RPM for the current operating system. Install the proper RPM using the following command: #rpm_-Uhv --replacefiles cpq_cciss_<os version>.i586.rpm After successful installation, reboot the server. Repeat steps 1 through 7 for each server connected to the MSA500 G2 storage system.
Post-installation configuration for UnitedLinux
After installing Smart Array Multipath software on a server with a UnitedLinux operating system, configure for multipath operations. Review the following sample configuration procedure. This sample assumes that the user wants to set /dev/cciss/c1d0p2 and /dev/cciss/c2d0p2 to point to the same volume as a multipath device. To configure the system for multipath use:

Number 1 105

Major 2 2

Minor 0 1

RaidDevice active sync active sync
State /dev/cciss/c2d0p2 /dev/cciss/c1d0p2
IMPORTANT: You can also review status by running:
Installation and operation in UnitedLinux environments 25
cat /proc/mdstat Results would look similar to the following: Personalities : [multipath] read_ahead 1024 sectors md0 : active multipath cciss/c2d0p2[0] cciss/c1d0p2[1] 14683328 blocks [2/2] [UU] unused devices : <none>
To format the volume, run: mkfs t ext2 /dev/md0 To mount the volume, run: mount /dev/md0 /spare_filesystem
Static load (read-only) balancing in UnitedLinux
Installation and operation in UnitedLinux environments 26

Troubleshooting

Problem diagnosis..... 27 Problems during installation.... 27 HBA failure..... 28 Cable failure..... 28

Problem diagnosis

When a problem exists with the multipath I/O, consult the Insight Management Agents and do one of the following:
For Microsoft operating systems:
Use the ProLiant Storage Manager or ProLiant Multipath Manager FAQs. Use the ProLiant Storage Manager with ProLiant Multipath Manager software and the DME to identify I/O path status.
For Linux operating systems, use the following command to identify HBA status, multipath status, and I/O block transfer size: cat /proc/mdstatus
Problems during installation
This section provides systematic instructions on what to try and where to go for help for the most common problems encountered during software installation. If you encounter problems during installation:

1. 2. 3. 4. 5. 6. 7.

In a Microsoft environment, be sure to uninstall any previous version of the multipath software first. Be sure that the firmware is upgraded for all HBAs and both MSA500 G2 controllers, if installed. Be sure that the hardware installation meets the minimum requirements (on page 5). Be sure that the SCSI cabling matches one of the supported four-node configurations (on page 6). Be sure that the HBA is functioning properly. Refer to the appropriate server and HBA documentation. If the HBA has failed, refer to "HBA failure (on page 28)." Be sure that the SCSI cables are functioning properly. If a SCSI cable fails, refer to "Cable failure (on page 28)." Be sure that you followed operating system-specific instructions:
If the server has a Red Hat operating system, be sure that the active kernel is edited properly. If the server has a Microsoft operating system, be sure that the SCSI cable from the redundant HBA was disconnected before Smart Array Multipath software installation.
If these steps do not solve the installation problem, contact an authorized reseller.

Troubleshooting 27

HBA failure
If one of the HBAs in the server fails, the redundant HBA performs all I/O operations. To replace the failed HBA:
Power down the HBA in one of the following ways:
For a PCI-based HBA in a Microsoft operating system environment, use the PCI Hot Plug functionality in the server to power down the HBA while the server is online. For an embedded HBA in a Microsoft operating system environment, power down the server. For an HBA in a Linux operating system environment, power down the server.

2. 3. 4. 5. 6.

Disconnect the SCSI cable from the connector on the HBA or from the external VHDCI connector on the server. Remove the failed HBA. Refer to the server maintenance and service guide. Install a replacement HBA. Refer to the documentation that ships with the server or HBA. Connect the SCSI cable to the connector on the HBA or to the external VHDCI connector on the server. Power up the HBA in one of the following ways:
Use PCI Hot Plug functionality to restore power. Power up the server.
IMPORTANT: In a Microsoft environment, the software recognizes the replacement HBA and automatically restores redundancy.
If operating in a Linux environment, reconfigure the I/O for multipath operation.

Cable failure

To replace the failed cable:
Disconnect the SCSI cable:
a. Disconnect the cable from the MSA500 G2 storage system. b. Disconnect the cable from the connector on the HBA or from the external VHDCI connector on the

server.

Connect the replacement cable:
a. Connect the cable to the connector on the HBA or to the external VHDCI connector on the server. b. Connect the cable to the MSA500 G2 storage system.
Power up the HBA in one of the following ways:

Troubleshooting 28

Acronyms and abbreviations

direct attached storage

device manager extension

host bus adapter

logical block addressing

logical unit number

Red Hat Package Manager
Selective Storage Presentation
very high density cable interconnect
2-Port Shared Storage Module 5
Linux procedures 14 Linux support 6
4-Port Shared Storage Module 5
Management Agents 27 management pack overview 5 Microsoft browser security settings 11 Microsoft procedures 9 Microsoft support 6 minimum hardware configuration 5 multipath operations 12, 20, 23 multipath software, overview 5
ACU (Array Configuration Utility) 20, 23
batteries, replacing 27 boot configurations 6 boot volume locations 6 browser settings 11
overview, management pack 5 overview, product 5 overview, web interface 5
cable failure 28 cluster configurations 7 configuration parameters 6 controller installation 9, 19
PCI Hot Plug functionality 6, 28 powering up 9, 19 problem diagnosis 27 product overview 5 ProLiant Storage Manager 6, 12, 27

data volume locations 6 device manager extension 12
features 6 firmware, updating 9, 19 four-node configurations 6
raidtab configuration file 16 raidtab terms 16 Red Hat procedures 14

HBA failure 28

SCSI cabling 9, 28 security settings 11 Selective Storage Presentation 6 software installation 11, 19, 21 static LUN balancing 12
installing software 11, 19, 21 internal boot configurations 7, 8

Index 30

three-server configurations 8 troubleshooting 27 two-server configurations 7

Index 31