Top
PRIMECLUSTERGlobal Disk Services Configuration and AdministrationGuide 4.5
FUJITSU Software

A.2.18 Swapping Disks (SAS)

When swapping the following disks, follow the steps described in this section.

In this section, these disks are called SAS disks.

If an SAS disk is swapped, the device number of the new disk may change. For this reason, an SAS disk cannot be swapped by [Swap Physical Disk] or [Restore Physical Disk] of the GDS Management View, or by the procedure to swap the physical disk by using the sdxswap command. For swapping an SAS disk, remove it from the class once, and then swap the disk device. After that, register the new disk to the class again.

The following example shows the procedures for swapping the SAS disk connected to a mirror group. In this example, the disk is mirrored and you do not need to remove volumes from the disk prior to swapping. If the disk to be swapped is not a mirrored disk, such as a single disk and a disk whose highest level group is not a mirror group, volumes must be removed from the disk prior to swapping.

In this case, first back up volume data if necessary, and then remove the volume. Next, remove the disk from the class, and after registering the new disk to the class, re-create volume and restore the data.

  1. Identify a raw device name (devname) of the disk to be swapped by referring to the GDS I/O error messages output in the /var/adm/messages file.

    To search the GDS I/O error messages, use the string "sfdsk" as a key. In the following example, the raw device name is "disk@g5000c5001d4809ff."

    WARNING: sfdsk: write error on slice:
               volume info:    devno(maj,min)=0x13300000002(307,2)
               device info:    devno(maj,min)=0x2000000038(32,56)
               devname=/scsi_vhci/disk@g5000c5001d4809ff (sd7)
               error info:     blk in disk=17308981, blk in slice=5232448
                               length=8192, resid=8192, errno=5
  2. Check the device name of the disk to be swapped.

    Specify the raw device name identified in step 1 as an argument of the grep command.

    In the following example, the device name of the disk to be swapped is "c0t5000C5001D4809FFd0."

    # ls -l /dev/rdsk | grep disk@g5000c5001d4809ff
    lrwxrwxrwx   1 root   root  52  Apr 25  13:46 c0t5000C5001D4809FFd0s0 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:a,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s1 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:b,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s2 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:c,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s3 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:d,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s4 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:e,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s5 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:f,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s6 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:g,raw
    lrwxrwxrwx   1 root   root   52  Apr 25  13:46 c0t5000C5001D4809FFd0s7 -> ../../devices/scsi_vhci/disk@g5000c5001d4809ff:h,raw
  3. Check the disk name, class name, and group name of the disk to be swapped.

    Execute the sdxinfo -D command and find the line where the device name confirmed in step 2 is shown in the DEVNAM field.

    In the following example, the disk name, class name, and the group name for the device name "c0t5000C5001D4809FFd0" are "rootDisk0002," "RootClass," and "rootGroup" respectively.

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- -------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   rootDisk0002 mirror RootClass rootGroup c0t5000C5001D4809FFd0 585912500 *          ENABLE
    disk   dataDisk0001 mirror RootClass dataGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    disk   dataDisk0002 mirror RootClass dataGroup c0t5000CCA0150F96F0d0 585912500 *          ENABLE
  4. Disconnect the disk to be swapped from the group.

    Specify the class name, the group name, and the disk name confirmed in step 3 for the -c, -g, and -d options of the sdxdisk -D command.

    # sdxdisk -D -c RootClass -g rootGroup -d rootDisk0002

    Note

    If a spare disk is connected

    When you try to disconnect the disk to be swapped while the spare disk is connected, the following error will be displayed.

    SDX:sdxdisk: ERROR: spare disk connected for disk_name

    In this case, you should disconnect the spare disk first.

    The following example shows the procedure to disconnect the spare disk (disk0001) before disconnecting the disk to be swapped (rootDisk0002).

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- ------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   rootDisk0002 mirror RootClass rootGroup c0t5000C5001D4809FFd0 585912500 *          ENABLE
    disk   disk0001     spare  RootClass rootGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    # sdxdisk -D -c RootClass -g rootGroup -d disk0001
    SDX:sdxdisk: INFO: disk0001: disconnected disk from group rootGroup
    # sdxdisk -D -c RootClass -g rootGroup -d rootDisk0002
    SDX:sdxdisk: INFO: rootDisk0002: disconnected disk from group rootGroup
  5. Check if the disk to be swapped is disconnected from the group.

    Execute the sdxinfo -D command and check if an asterisk (*) is displayed in the GROUP field of the disconnected disk (rootDisk0002).

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- -------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   dataDisk0001 mirror RootClass dataGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    disk   dataDisk0002 mirror RootClass dataGroup c0t5000CCA0150F96F0d0 585912500 *          ENABLE
    disk   rootDisk0002 undef  RootClass *         c0t5000C5001D4809FFd0 585912500 *          ENABLE
  6. Remove the disk to be swapped from the class.

    Specify the class name and the disk name confirmed in step 3 for the -c and -d options of the sdxdisk -R command.

    # sdxdisk -R -c RootClass -d rootDisk0002
  7. Check if the disk to be swapped has been removed from the class.

    Execute the sdxinfo -D command to make sure the information of the deleted disk (rootDisk0002) is not displayed.

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- ------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   dataDisk0001 mirror RootClass dataGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    disk   dataDisk0002 mirror RootClass dataGroup c0t5000CCA0150F96F0d0 585912500 *          ENABLE
  8. Identify the identifier specific to the hardware to which the disk to be swapped is connected.

    To find the identifier specific to the hardware, check the "Ap_Id" field of the information in which the device name of the disk to be swapped is displayed.

    In the following example, the identifier specific to the hardware of the physical disk "c0t5000C5001D4809FFd0" is "c5::w5000c5001d4809fd."

    # cfgadm -av
    Ap_Id                     Receptacle  Occupant      Condition   Information
    When         Type       Busy     Phys_Id
    ...
    c5::w5000c5001d4809fd,0  connected     configured    unknown  \
            Client Device: /dev/dsk/c0t5000C5001D4809FFd0s0(sd7)
    unavailable disk-path   n  \
            /devices/pci@400/pci@2/pci@0/pci@e/scsi@0/iport@4:scsi::w5000c5001d4809fd,0
    ...
  9. Swap disks.

    Ask field engineers to swap your disk unit.

    Both hot swap and non-hot swap are available.

  10. Check the device name of the new disk.

    For the information of identifier specific to the hardware identified in step 8, check if "unconfigured" is displayed to the Occupant and check the information displayed in "iport@X.scsi" format to the Phys_Id.

    The device of the new disk is displayed as "configured" to the Occupant and "iport@X.scsi" that is checked with the above procedure is displayed to the Phys_Id.

    In the following example, the device name of the new disk is "c0t5000C5001D4806BFd0."

    # cfgadm -av
    Ap_Id Receptacle Occupant Condition Information When Type Busy Phys_Id ... c5::w5000c5001d4809fd,0 connected unconfigured unknown (sd7) unavailable disk-path n \ /devices/pci@400/pci@2/pci@0/pci@e/scsi@0/iport@4:scsi::w5000c5001d4809fd,0 c5:: w5000c5001d4806bd,0 connected configured unknown \ Client Device: /dev/dsk/c0t5000C5001D4806BFd0s0(sd8) unavailable disk-path n \ /devices/pci@400/pci@2/pci@0/pci@e/scsi@0/iport@4:scsi:: w5000c5001d4806bd,0 ...
  11. Confirm that the device special file (cCtWWNdDs{0-7}) of the new disk exists.

    Append an asterisk (*) to the end of the device name confirmed in step 10 and specify it as an argument of the ls command.

    In the following example, you can see that the device special files from "c0t5000C5001D4806BFd0s0" to "c0t5000C5001D4806BFd0s7" exist.

    # ls -lL /dev/rdsk/c0t5000C5001D4806BFd0*
    crw-r-----   1 root     sys  32, 16 May 16 17:26 /dev/rdsk/c0t5000C5001D4806BFd0s0
    crw-r-----   1 root     sys  32, 17 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s1
    crw-r-----   1 root     sys  32, 18 May 17 09:44 /dev/rdsk/c0t5000C5001D4806BFd0s2
    crw-r-----   1 root     sys  32, 19 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s3
    crw-r-----   1 root     sys  32, 20 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s4
    crw-r-----   1 root     sys  32, 21 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s5
    crw-r-----   1 root     sys  32, 22 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s6
    crw-r-----   1 root     sys  32, 23 May 17 12:10 /dev/rdsk/c0t5000C5001D4806BFd0s7
  12. If the relevant system is a cluster system, register the new disk resources with PRIMECLUSTER according to the following steps. If not, skip to step 13.

    1. Identify the node ID of the cluster managed by PRIMECLUSTER.

      In the following example, the node ID is "0."

      # /etc/opt/FJSVcluster/bin/clgetnode
      RID 3
      KEY TRC89
      RNAME TRC89
      NODEID 0
    2. Confirm the PRIMECLUSTER resource ID of the device (resource ID before swapping disks).

      Specify the device name confirmed in step 2 above by the -k option and the node ID confirmed in step 12-1 by the -s option of the clgetrid command.

      In the following example, the resource ID is "25."

      # /etc/opt/FJSVcluster/sys/clgetrid -c DISK -k c0t5000C5001D4809FFd0 -s 0
      25
    3. Delete the disk resource of the original disk.

      In the following example, delete the resource whose resource ID is 25.

      # /etc/opt/FJSVcluster/bin/cldeldevice -r 25
    4. Confirm that the resource of the original disk has been deleted.

      Confirm that the deleted resource is not displayed.

      # /etc/opt/FJSVcluster/sys/clgetrid -c DISK -k c0t5000C5001D4809FFd0 -s 0
    5. Register the new disk with PRIMECLUSTER.

      # /etc/opt/FJSVcluster/sys/clautoconfig -r
    6. Confirm that the device name of the new disk is registered.

      If the device name "c0t5000C5001D4806BFd0" is displayed, the new disk has successfully been registered with PRIMECLUSTER.

      # /etc/opt/FJSVcluster/bin/clgettree
      Cluster 1 cluster
          Domain 2 cluster
              Shared 7 SHD_cluster
                  SHD_DISK 3417 SHD_Disk3417 UNKNOWN
                      DISK 3418 c3t46554A4954535520333030303030383530303043d0 UNKNOWN TRC89
                      DISK 3477 c3t46554A4954535520333030303030383530303043d0 UNKNOWN ryuta
                  SHD_DISK 3419 SHD_Disk3419 UNKNOWN
                      DISK 3420 c3t46554A4954535520333030303030383530303042d0 UNKNOWN TRC89
                      DISK 3478 c3t46554A4954535520333030303030383530303042d0 UNKNOWN ryuta
              Node 3 TRC89 ON
                  Ethernet 105 igb3 UNKNOWN
                  SDX_DC 3519 RootClassI UNKNOWN
                  DISK 71 c0t5000CCA0150FEA10d0 UNKNOWN
                  DISK 72 c0t5000CCA00AC1C874d0 UNKNOWN
                  DISK 73 c0t5000CCA0150F96F0d0 UNKNOWN
                  DISK 3418 c3t46554A4954535520333030303030383530303043d0 UNKNOWN
                  DISK 3420 c3t46554A4954535520333030303030383530303042d0 UNKNOWN
                  DISK 3518 c0t5000C5001D4806BFd0 UNKNOWN
              Node 5 ryuta  ON
                  Ethernet 106 igb3 UNKNOWN
                  DISK 101 c0t5000C5001D480F6Fd0 UNKNOWN
                  DISK 3477 c3t46554A4954535520333030303030383530303043d0 UNKNOWN
                  DISK 3478 c3t46554A4954535520333030303030383530303042d0 UNKNOWN
  13. Reload the GDS disk information.

    # sdxinfo -x Refresh
  14. Register the new disk with GDS.

    Specify the class name of the original disk for the -c option of the sdxdisk -M command. Specify the device name of the new disk and the disk name of the original disk for the -d option.

    # sdxdisk -M -c RootClass -d c0t5000C5001D4806BFd0=rootDisk0002
  15. Confirm that the new disk is registered with GDS.

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- -------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   dataDisk0001 mirror RootClass dataGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    disk   dataDisk0002 mirror RootClass dataGroup c0t5000CCA0150F96F0d0 585912500 *          ENABLE
    disk   rootDisk0002 undef  RootClass *         c0t5000C5001D4806BFd0 585912500 *          ENABLE
  16. Connect the new disk to the group.

    Specify the class name, group name, and disk name confirmed in step 3 for the -c, -g, and -d options of the sdxdisk -C command.

    # sdxdisk -C -c RootClass -g rootGroup -d rootDisk0002
  17. Confirm that the new disk is connected to the group.

    # sdxinfo -D
    OBJ    NAME         TYPE   CLASS     GROUP     DEVNAM                DEVBLKS   DEVCONNECT STATUS
    ------ ------------ ------ --------- --------- --------------------- --------- ---------- -------
    disk   rootDisk0001 mirror RootClass rootGroup c0t5000CCA0150FEA10d0 585912500 *          ENABLE
    disk   rootDisk0002 mirror RootClass rootGroup c0t5000C5001D4806BFd0 585912500 *          ENABLE
    disk   dataDisk0001 mirror RootClass dataGroup c0t5000CCA00AC1C874d0 585912500 *          ENABLE
    disk   dataDisk0002 mirror RootClass dataGroup c0t5000CCA0150F96F0d0 585912500 *          ENABLE
  18. When the new disk is connected to the group, Just Resynchronization of the new disk will be performed.

    # sdxinfo -S
    OBJ    CLASS     GROUP     DISK         VOLUME     STATUS
    ------ --------- --------- ------------ ---------- -------
    slice  RootClass rootGroup rootDisk0001 rootVolume ACTIVE
    slice  RootClass rootGroup rootDisk0002 rootVolume COPY
    slice  RootClass rootGroup rootDisk0001 swapVolume ACTIVE
    slice  RootClass rootGroup rootDisk0002 swapVolume COPY
    slice  RootClass rootGroup rootDisk0001 homeVolume ACTIVE
    slice  RootClass rootGroup rootDisk0002 homeVolume COPY
    slice  RootClass group0001 disk000001   volume0001 ACTIVE
    slice  RootClass group0001 disk000002   volume0001 ACTIVE