Top
PRIMECLUSTERGlobal Disk Services Configuration and AdministrationGuide 4.5
FUJITSU Software

F.1.5 System Disk Abnormality

The following description shows resolutions to problems relevant to system disks on which the file systems / (root), /usr, and /var operates.

In one of the following circumstances, take action as indicated for the relevant situation.

(1) Create backup of root file system.

Explanation

Mirroring the system disk will protect data in case one of the physical disks crashes. However, to restore data damaged due to a critical fault as a result of multiple failures and so on or by an operational mistake, create backup data in advance.

Resolution

See "Chapter 6 Backing Up and Restoring."


(2) System can be booted, but the system disk data is invalid.

Explanation

For some reason, the system disk data is invalid. You must restore data from the backup data created in advance.

Resolution

For ZFS boot environment, see "6.2.2 Restoring." For UFS boot environment, see "6.3.2 Restoring (When the System Can Be Booted)."


(3) System cannot be booted (Failure of original boot disk).

Explanation

Even when you have the system disk mirrored, you may experience unsuccessful booting. For example, you may encounter such a problem when the file accessed during the booting process in the boot disk is physically damaged, or when a disk is swapped improperly.
If the console message of the unsuccessful boot disk indicates such is the case, try booting from the other boot disk, i.e. the mirror disk.

Resolution

1) Confirm the name of the mirror disk. Use the OpenBoot ok prompt shown below.

ok printenv boot-device
/pci@1f,4000/scsi@3/disk@0,0:a /pci@1f,4000/scsi@3/disk@1,0:a

In this example, the original disk name is /pci@1f,4000/scsi@3/disk@0,0:a, which is indicated on the left. The mirror disk is /pci@1f,4000/scsi@3/disk@1,0:a. When mirroring is configured with three-way or more multiplexing, multiple mirror disks exist.


2) Boot from the mirror disk.

ok boot /pci@1f,4000/scsi@3/disk@1,0:a

3) If booting is successful, find out what caused the original disk to fail, and recover by swap the disk.

See

For details on disk swap, see "D.8 sdxswap - Swap disk" and resolutions described in "5.3.4 Disk Swap."

When the procedures above cannot resolve the problem, causes other than data errors in the original boot disk unit may exist, or data errors may also exist in the mirror disk.
If it would appear that data errors also exist in the mirror disk, see "(5) System cannot be booted (Boot disk data damage).."


(4) System cannot be booted (Failure in all boot disk devices).

Note

This troubleshooting is intended for the UFS boot environment. If a system cannot be booted in the ZFS boot environment, see the following:

Explanation

Booting may be failed even a system disk has been mirrored. If you want to check the restoring procedure when all the boot disks are physically failed (or assumed to be failed), perform the following procedure for restoring as all the data saved in a disk will be deleted.

Resolution

[Before the operation]

Before the operation and at the time of configuring a system, perform the following tasks.


1) Check the partition configuration and the cylinder size of a system disk. Then make a note of them.

This task must be performed before the system disk mirroring configuration. Execute the following command and take note of the underlined information.

Example: When the system disk is c4t50000394882261A0d0.

# format c4t50000394882261A0d0
format> verify

Primary label contents:

Volume name = <00000000>
ascii name  = <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668>
pcyl        = 64988
ncyl = 64986
acyl = 2
nhead = 27
nsect = 668
Part Tag Flag Cylinders Size Blocks
0 root wm 1861 - 9302 64.00GB (7442/0/0) 134223912
1 swap wu 0 - 1860 16.01GB (1861/0/0) 33564996
2 backup wm 0 - 64985 558.89GB (64986/0/0) 1172087496
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 0 0 (0/0/0) 0

2) Check if MPxIO is enabled for the system disk and record the result.

This step must be completed before setting up mirroring for the system disk.

MPxIO is enabled, if the /scsi_vhci directory is in the logical path of the displayed boot disk.

Example: If the system disk is c0t50000394882261A0d0

# format c0t50000394882261A0d0
format> current Current Disk = c0t50000394882261A0d0: 00000000 <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668>
/scsi_vhci
/disk@g50000394882261a0 /dev/chassis/SYS/HDD0/disk

3) Collect backups of root file system and so on.

It is necessary to perform this operation while system disks are in mirroring state.
For the method of collecting backups, see "6.3.1 Backing Up."

Note

  • To prepare for unexpected failures, make backups regularly.

  • For Solaris 10, it is necessary to choose the OS to be booted when restoring depending on the kernel patch number. Therefore, when collecting backups, make a note of the patch application status by using the following command.

    # /usr/bin/showrev -p

4) Check the name of a root class.

Names of root classes will be used in the subsequent procedures. Check names in beforehand because it is difficult to check them when the system cannot be booted due to a failure in all boot disk devices.

Execute sdxinfo -C. The NAME field in the line where "root" is output in the TYPE field is the root class name. In the following example, the root class name is "RootClass."

# sdxinfo -C
OBJ NAME TYPE SCOPE SPARE ------- ---------- -------- ------------ ----- class RootClass root (local) 0 class class0001 local node1 0 class class0002 shared node1:node2 0


5) Check the physical disk name of root class.

Physical disk names of root class will be used in the subsequent procedures. Check the physical disk names beforehand because it is difficult to check them when the system cannot be booted due to a failure in all boot disk devices.

Execute sdxinfo -D. Specify the name of the root class that was checked using step 4) with -c option. The DEVNAM field is the physical disk name. In the following example, the physical disk names of root class are "c4t50000394882261A0d0" and "c4t5000039488226254d0."

# sdxinfo -D -c RootClass
OBJ NAME TYPE CLASS GROUP DEVNAM DEVBLKS DEVCONNECT STATUS
------ ------- ------ ------- ------- ------- -------- ---------------- -------
disk rootDisk0001 mirror RootClass rootGroup c4t50000394882261A0d0 1172087496 node1 ENABLE
disk rootDisk0002 mirror RootClass rootGroup c4t5000039488226254d0 1172087496 node1 ENABLE

[Restoring]

1) Swap all the physical disks registered in the root class.

Swapping operation is performed according to a defined procedure for each device.

Note

  • When restoring to the disk that was used in GDS before without swapping a physical disk, boot the system from the CD-ROM device (or network) and restore all the physical disks registered in a root class to the initial status. For the method of booting from the CD-ROM device (or network), see step 2).

    # format c0t0d0
    format> type AVAILABLE DRIVE TYPES: 0. Auto configure ... Specify disk type (enter its number)[14]: 0 c0t0d0: configured with capacity of 4.00GB <SUN4.2G cyl 3880 alt 2 hd 16 sec 135> selecting c0t0d0 [disk formatted] format> label Ready to label disk, continue? y format> quit

2) Boot the system from a CD-ROM device (or the network).

For the procedure, see the Solaris manual.

Note

  • To restore Solaris 10, to which the kernel patch 137111-08 or earlier has been applied, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS, to which the kernel patch 137111-08 or earlier has been applied, you are restoring. To boot from a CD-ROM device, use the medium of Solaris 10 5/08 or earlier.

  • To restore Solaris 10, to which the kernel patch 137137-09 or later has been applied, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS, to which the kernel patch 137137-09 or later has been applied, you are restoring. To boot from a CD-ROM device, use the medium of Solaris 10 10/08 or later.

  • To restore Solaris 11, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS you are restoring.

  • In a system where mirroring is conducted by GDS, booting from a CD-ROM device (also network) and performing an illegal operation may disrupt the mirroring environment. Be sure to follow the procedure described here and to avoid operations not provided in this procedure.

  • When mounting the file system on a physical slice at the time of recovery, mounting as read-only is recommended to write-lock.


See

For details on step 2) to 5), see the Solaris manual.

3) Restore the system disk to the partition configuration confirmed in step 1) in [Before the operation].

Note

For the disk of Expansion File Unit connected to a 6Gbps SAS card, or the internal disks of SPARC M12/M10 and SPARC T4-4/T4-2/T4-1/T3-4/T3-2/T3-1 servers, the device name that is displayed in the format command differs with the device name before swapping because the physical disk was swapped in step 1). Execute format command to set the partition of the device that was output.

# format
Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t50000394882261A2d0 <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668> 00000000 /pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000394882261a2,0 1. c0t50000394980066DEd0 <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668> 00000002 /pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000394980066de,0 2. c0t500003948822616Ed0 <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668> 00000008 /pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w500003948822616e,0 : 10. c1t500000E0D02C6286d6 <FUJITSU-ETERNUS_DXL-0000 cyl 2046 alt 2 hd 64 sec 256> 00E7002F /pci@8000/pci@4/pci@0/pci@8/QLGC,qlc@0/fp@0,0/ssd@w500000e0d02c6286,6format> partition Specify disk (enter its number): 2 selecting c0t500003948822616Ed0: 00000008 format> partition : partition> 0 Part Tag Flag Cylinders Size Blocks 0 root wm 0 0 (0/0/0) 0 Enter partition id tag[root]: root Enter partition permission flags[wm]: wm Enter new starting cyl[0]: 1861 Enter partition size[0b, 0c, 1861e, 0.00mb, 0.00gb]: 134223912b partition> 1 Part Tag Flag Cylinders Size Blocks 1 swap wu 0 0 (0/0/0) 0 Enter partition id tag[swap]: swap Enter partition permission flags[wu]: wu Enter new starting cyl[0]: 0 Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 33564996b partition> label Ready to label disk, continue? y partition> quit format> verify Primary label contents: Volume name = <00000002> ascii name = <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668> pcyl = 64988 ncyl = 64986 acyl = 2 nhead = 27 nsect = 668 Part Tag Flag Cylinders Size Blocks 0 root wm 1861 - 9302 64.00GB (7442/0/0) 134223912 1 swap wu 0 - 1860 16.01GB (1861/0/0) 33564996 2 backup wu 0 - 64985 558.89GB (64986/0/0) 1172087496 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

4) Restore backup data collected to a tape medium and so on to each physical slice.

In the following example, restore the data of each file system based on the backup data collected by using the ufsdump(1M) command in step 3) in [Before the operation].

Restore the data from the tape medium of tape device /dev/rmt/0 by using the ufsrestore(1M) command.

The following is a restore example of s0 slice. If other slices exist other than s0, restore also those slices.

# newfs /dev/rdsk/c0t50000394980066DEd0s0
# mount -F ufs /dev/dsk/c0t50000394980066DEd0s0 /mnt
# cd /mnt
# ufsrestore rvf /dev/rmt/0
# rm restoresymtable
# cd /
# umount /mnt
# fsck /dev/rdsk/ c0t50000394980066DEd0s0

5) Install the boot block to the root partition of the system disk.

When executing installboot command, the slice which includes /usr file system needs to be mounted.

6) Mount the new system disk. When there is /etc in the other partition, also mount the file system under the mount point of the root file system as /etc.

# mount -F ufs /dev/dsk/c0t50000394980066DEd0s0 /mnt
# mount -F ufs /dev/dsk/c0t50000394980066DEd0s5 /mnt/etc

7) Cancel the system disk mirroring.

Update /mnt/etc/system, /mnt/etc/vfstab, /mnt/kernel/drv/sfdsk.conf, and /mnt/etc/opt/FJSVsdx/dev/devname.db.

7-1) Check if MPxIO is enabled for the system disk and record the result.

Execute the current subcommand of the format(1M) command. MPxIO is enabled, if the /scsi_vhci directory is in the logical path of the displayed boot disk.

Example: If the system disk is c0t50000394882261A0d0

# format c0t50000394882261A0d0
format> current
Current Disk = c0t50000394882261A0d0: 00000000
<TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668>
/scsi_vhci/disk@g50000394882261a0
/dev/chassis/SYS/HDD0/disk

7-2) Edit /mnt/etc/system.

An example is shown below.

# vi /mnt/etc/system
... forceload: drv/sd forceload: drv/sfdsk rootdev:/pseudo/sfdsk@0:2,blk <- Delete this line.

7-3) Edit the mount setting lines of the root class partitions in /mnt/etc/vfstab according to the partitions of the new system disk.

An example is shown below.

Note

The current physical disk name (cXtXdX) is not the same as the one when starting from the system disk as in the step 7-1) of [Restoring] if MPxIO is enabled in the step 2) of [Before the operation] and MPxIO is disabled in the step 7-1) of [Restoring]. In this case, describe the current physical disk name in /mnt/etc/vfstab.

7-4) Delete all the lines where sysvol-db and SDX_DEV_TREE is stated from /mnt/kernel/drv/sfdsk.conf.

An example is shown as below.

# vi /mnt/kernel/drv/sfdsk.conf
# # Copyright (c) 1998, by Fujitsu, Ltd. # All rights reserved. # name="sfdsk" parent="pseudo"; sysvol-db0="sd","2"; <-Delete this line. sysvol-db1="sd","10"; <-Delete this line. sfdsk_root_volume=1; SDX_DEV_TREE_32_2="/scsi_vhci/disk@g50000394882261a0 (sd0)"; <-Delete this line. SDX_DEV_TREE_32_10="/scsi_vhci/disk@g5000039488226254 (sd1)"; <-Delete this line. ...

7-5) Delete the lines which include physical disk names of root class that was checked in step 5) of [Before the operation] from /mnt/etc/opt/FJSVsdx/dev/devname.db.

An example is show as below:

# vi /mnt/etc/opt/FJSVsdx/dev/devname.db
device_32_2="/dev/rdsk/c4t50000394882261A0d0s2" <-Delete this line. device_32_10="/dev/rdsk/c4t5000039488226254d0s2" <-Delete this line. device_118_2="/dev/rdsk/c4t600000E00D00000000002D1F00470000d0s2" ...

8) Edit /mnt/kernel/drv/mpt_sas.conf so that MPxIO is not enabled.

This step is necessary only if MPxIO is enabled in the step 2) of [Before the operation] and MPxIO is disabled in the step 7-1) of [Restoring].


9) Apply the device name of the disk.

This operation is necessary to the disk of Expansion File Unit connected to a 6Gbps SAS card, or the internal disks of SPARC M12/M10 and SPARC T4-4/T4-2/T4-1/T3-4/T3-2/T3-1 servers.

# cd /devices ; tar cf - . | ( cd /mnt/devices ; tar xf - )
# devfsadm -r /mnt -p /mnt/etc/path_to_inst

10) Update the boot-archive file.

# cd /
# /sbin/bootadm update-archive -R /mnt

11) Unmount the file system.

# umount /mnt/etc
# umount /mnt

12) Reboot the system from the new boot disk by the following procedure.

The procedure to reboot from a boot disk is different for the disk of Expansion File Unit connected to a 6Gbps SAS card or the internal disks of SPARC M12/M10 and SPARC T4-4/T4-2/T4-1/T3-4/T3-2/T3-1 servers, and the other cases.

For the procedure for SAN Boot environment, see SAN Boot manual.


The disk of Expansion File Unit connected to a 6Gbps SAS card, or the internal disks of SPARC M12/M10 and SPARC T4-4/T4-2/T4-1/T3-4/T3-2/T3-1 servers

12a-1) Check the root partition's slice names of the new boot disk and make a note of them.

# ls -l /dev/dsk |grep c0t50000394980066DEd0s0
lrwxrwxrwx 1 root root 82 Sep 3 19:12 c0t50000394980066DEd0s0 -> ../../devices/pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@f/disk@w50000394980066de,0:a

12a-2) Check and record the boot disk information and make a note of them.

How to check information depends on whether or not MPxIO is enabled in step 7-1).

12a-3) Enter the OpenBoot environment.

# halt

Note

If the system is started from an emergency startup disk for restoration, enter the OpenBoot environment using the shutdown(1M) command.

12a-4) Set the boot-device property in OpenBoot.

Combine the obp-path parameter output in step 12a-2) and the slice name (:a in this example) output in step 12a-1), and then set the boot-device property.

ok setenv boot-device /pci@8000/pci@4/pci@0/pci@0/scsi@0/disk@w50000394980066de,0:a

12a-5) Set the boot-device-index property of OpenBoot to 0.

ok setenv boot-device-index 0

12a-6) Boot and re-configure the system.

ok boot -r

Note

The following warning messages may be output during the booting process, but it does not affect the system. The current swap area is set as a dump device. If you want to change the setting of dump device, use dumpadm(1M).

savecore: open("/dev/sfdsk/System/dsk/swap"): No such file or directory
dumpadm: cannot use /dev/sfdsk/System/dsk/swap as dump device:
                                           No such file or directory

Other than the above boot disks

12b-1) Check the root partition's device path of swapped boot disks and make a note of them.

# ls -l /dev/dsk | grep c0t0d0s0
lrwxrwxrwx 1 root root 63 Aug 20 12:00 c0t0d0s0 -> ../../devices/pci@1f,4000/scsi@3/sd@0,0:a

12b-2) Enter the OpenBoot environment.

# halt

Note

If the system is booted from an emergency startup disk for restoration, enter the OpenBoot environment using the shutdown(1M) command.

12b-3) Execute the show-devs command in the OpenBoot environment and display the device name.

ok show-devs /pci@1f,4000/scsi@3
... /pci@1f,4000/scsi@3/disk ...

12b-4) Set the boot-device property in OpenBoot.

12b-5) Check if there is the boot-device-index property in OpenBoot. If there is, set the boot-device-index property to 0.

ok printenv boot-device-index

If there is boot-device-index property, "boot-device-index = number"is displayed. In this case, execute the following command.

If "boot-device-index = number"is not displayed, do not execute the following command.

ok setenv boot-device-index 0

12b-6) Boot the system.

ok boot

Note

The following warning messages may be output during the booting process, but it does not affect the system. The current swap area is set as a dumpdevice. If you want to change the setting of dump device, use dumpadm(1M).

savecore: open("/dev/sfdsk/System/dsk/swap"): No such file or directory
dumpadm: cannot use /dev/sfdsk/System/dsk/swap as dump device:
                                          No such file or directory

13) Reset the content of /kernel/drv/mpt_sas.conf file that was edited in step 8) to its original content.

This step is necessary only after you do the step 8).


14) Execute the following command to enable MPxIO and reboot system.

This step is necessary only after you do the step 13).

# stmsboot -u

15) If it is the cluster system, delete the cluster resource of the root class.

The target is the only root class. Do not delete the cluster resource of local classes and shared classes.

The following examples show the case where the following classes exist:

Check whether the class resource of the root class exists.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -I
NAME SCOPE RID(RDB) RID(CDB) --------------- -------------------- --------------- --------------- RootClass node1 368 1 class0001 node1 369 2 class0002 node1,node2 371,372(370) 4,5(3) #

If the class resource of the root class is not displayed, go to the next step 16).

If the class resource of the root class is displayed, delete the class resource of the root class.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -R -c RootClass
sdxdcrsc: RootClass removed #

Confirm that the class resource of the root class has been deleted.

Execute the following command and check that the target class name is not displayed.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -I
NAME SCOPE RID(RDB) RID(CDB) --------------- -------------------- --------------- --------------- class0001 node1 369 2 class0002 node1,node2 371,372(370) 4,5(3) #

16) Re-configure mirroring of the system disk.

Note

When this procedure is performed by assuming that a system disk is broken, that is a disk was not swapped in step 1), the following messages may be output at the time of re-setting the mirroring.

# sdxdisk -M -c System -a type=root -d c0t0d0=Root1:keep,c1t0d0=Root2:undef
SDX:sdxdisk: INFO: System: created class SDX:sdxdisk: ERROR: c1t0d0: already exists in ... SDX:sdxdisk: INFO: System: removed class #

When the above errors are output, re-boot the system and set the mirroring again.

Information

If there are missing parts and errors during this procedure, the mirroring status may be unstable and the subsequent re-settings may be failed.

You can check the unstable parts by using the sdxrootck command after performing this procedure or failing the re-setting.

See


(5) System cannot be booted (Boot disk data damage).

Explanation

For some reason, the system disk data is invalid. You must restore data from the backup data created in advance.
If restoring data from the mirror disk was unsuccessful, follow the procedures shown below.

Resolution

For the ZFS boot environment, see "6.2.2 Restoring." For the UFS boot environment, see "6.3.3 Restoring (When the System Cannot Be Booted)."


(6) System cannot be booted (Incorrect boot disk device name).

Explanation

By mirroring system disks, the system's boot disk device name (boot-device parameter) is automatically set and updated. However, in some systems GDS cannot recognize correct device names and thus sets incorrect device names.
In that event, the following console message is output and booting the system fails.

ok boot
Rebooting with command: boot
Boot device: /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@0,0:a File and args:
Evaluating: boot
Can't open boot device

In this example the boot disk device name is "/ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@0,0:a", but the correct device name is "/ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0:a" and thus booting the system fails.

Resolution

Create the bootdevtab file (boot disk device name file) for proper device name configuration.

1) Specify the correct boot device name and boot the system.

ok boot /ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0:a

For the correct boot disk device name, perform the following.

1-1) View device names of disks connected to the system.
For the following procedures, make a copy of the displayed results.

ok show-disks
a) /ssm@0,0/pci@18,700000/pci@1/scsi@2/disk b) /ssm@0,0/pci@1c,700000/pci@1/scsi@2/disk c) /ssm@0,0/pci@18,700000/pci@2/SUNW,isptwo@4/sd d) /ssm@0,0/pci@1c,700000/pci@2/SUNW,isptwo@4/sd q) NO SELECTION Enter Selection, q to quit: q ok

1-2) Based on a) in the displayed results, "/ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@0,0:a" should be replaced with "/ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0:a" as that is the correct boot disk device name.


2) Create the bootdevtab file.

2-1) Create a template.

# sdxinfo -x BootDev > /var/tmp/bootdevtab

2-2) Edit the template.

# vi /var/tmp/bootdevtab
Before edit : c0t0d0 /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@0,0 c1t0d0 /ssm@0,0/pci@1c,700000/pci@1/scsi@2/sd@0,0 After edit : c0t0d0 /ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0 c1t0d0 /ssm@0,0/pci@1c,700000/pci@1/scsi@2/disk@0,0

Specify the physical disk name in the first field.
Specify the corresponding boot disk device name in the second field. Neither field requires partition assignment, such as s0 and :a.

To the template a boot disk device name assumed by GDS will be output. If the output boot disk name is incorrect, replace it with the correct device name. The correct device name can be checked as described in 1-1) through 1-2). Note that two boot disk device names may be output. In that event, delete one of the names and replace the remaining one with the correct device name.

Note

  • Separate one field from another in space or tab delimited format. Lines starting with the number sign (#) are assumed as comments.
    All disks registered with a root class are output to the template.
    Specify disks that may be used as the boot disk in the bootdevtab file. The relevant disks are:

    • Original boot disk

    • Mirror disk

    • Spare disk connected to a boot disk's group

    • Boot and spare disks in an alternative boot environment
      (When using the system volume snapshot function of GDS Snapshot)

    Do not specify the following disks in the bootdevtab file. Delete them from the bootdevtab file or comment out the relevant lines. Leaving them unedited will not affect the operation.

    • Disks not used as the boot disk

    • Disks for which GDS correctly assumed the boot disk device name and no assignment in the bootdevtab file is required

  • When FC-AL internal disks are swapped, the boot disk device name changes.
    Declare as follows so that recreation of the bootdevtab file is not required every time disks are swapped. Put the asterisk (*) after the last at mark (@). GDS will then complement the asterisk portion automatically.

    Before edit :
    c0t0d0  /ssm@0,0/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w50020f230000798c,0
    
    After edit :
    c0t0d0  /ssm@0,0/pci@1d,700000/SUNW,qlc@1/fp@0,0/disk@*
  • For MPLB disks, specify the boot disk unit names of their paths.

    Before edit :
    # mplb0
    c12t16d0 /pci@1f,4000/fibre-channel@4/disk@10,0
    c13t16d0 /pci@1f,4000/fibre-channel@5/disk@10,0
    # mplb1
    c12t17d0 /pci@1f,4000/fibre-channel@4/disk@11,0
    c13t17d0 /pci@1f,4000/fibre-channel@5/dsik@11,0
    After edit :
    # mplb0
    c12t16d0 /pci@1f,4000/fibre-channel@4/sd@10,0
    c13t16d0 /pci@1f,4000/fibre-channel@5/sd@10,0
    # mplb1
    c12t17d0 /pci@1f,4000/fibre-channel@4/sd@11,0
    c13t17d0 /pci@1f,4000/fibre-channel@5/sd@11,0

2-3) Place the template in the /etc/opt/FJSVsdx/lib directory.

# cd /etc/opt/FJSVsdx/lib
# mv bootdevtab bootdevtab.old (when there is an existing bootdevtab file)
# cp /var/tmp/bootdevtab bootdevtab
# chmod 600 bootdevtab

2-4) Confirm that the bootdevtab file was created normally.

# sdxinfo -x BootDev
c0t0d0 /ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0 c1t0d0 /ssm@0,0/pci@1c,700000/pci@1/scsi@2/disk@0,0

Check whether the boot disk device name is output normally.


3) Check whether the system's boot disk device name (boot-device parameter) is set normally.

3-1) Stop the system and specify the correct boot disk device name.

# init 0
... ok boot /ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0:a

3-2) Check the boot-device parameter values.

# eeprom boot-device
boot-device=/ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@0,0:a \ /ssm@0,0/pci@1c,700000/pci@1/scsi@2/disk@0,0:a

In the boot-device parameter, the device name of the boot device where synchronization copying is complete is set. If synchronization copying is incomplete, perform the check after it is complete.

To add a disk to the root class and specify the disk in the bootdevtab file, before connecting to the disk to a group, re-create the bootdevtab file according to the procedures described in 2).

When the disk is connected to a group and synchronization copying is complete, the boot disk device name is added to the boot-device parameter.

When the bootdevtab file is not re-created before synchronization copying is complete, a boot disk device name assumed by GDS is added to the boot-device parameter when the synchronization copying ends. In that event, re-create the bootdevtab file and set the correct boot disk device name as follows. Alternatively, reboot the system for proper boot disk device name configuration.

# eeprom boot-device
boot-device=... /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@1,0:a Incorrect boot disk device name
# eeprom boot-device="... 
/ssm@0,0/pci@18,700000/pci@1/scsi@2/disk@1,0:a"
                          Correct boot disk device name

(7) System cannot be booted in a ZFS Boot Environment (Abnormality of root class private slice).

Explanation

Data of the private slice in a root class may become incorrect and the system cannot be booted by some trouble.

Resolution

See

For details of Solaris commands, see the Solaris manual.

1) Boot the system from a CD-ROM device (or the network).

For the procedure, see the Solaris manual.

Note

  • To restore Solaris 10, to which the kernel patch 137111-08 or earlier has been applied, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS, to which the kernel patch 137111-08 or earlier has been applied, you are restoring. To boot from a CD-ROM device, use the medium of Solaris 10 5/08 or earlier.

  • To restore Solaris 10, to which the kernel patch 137137-09 or later has been applied, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS, to which the kernel patch 137137-09 or later has been applied, you are restoring. To boot from a CD-ROM device, use the medium of Solaris 10 10/08 or later.

  • To restore Solaris 11, the version of an OS booted from a CD-ROM device (also network or emergency startup disk for restoration) must correspond to the version of the OS you are restoring.

  • In a system where mirroring is conducted by GDS, booting from a CD-ROM device (also network) and performing an illegal operation may disrupt the mirroring environment. Be sure to follow the procedure described here and to avoid operations not provided in this procedure.

  • When mounting the file system on a physical slice at the time of recovery, mounting as read-only is recommended to write-lock.


2) Check the disk to configure the ZFS root pool that can be imported.

# zpool import
pool: rpool <-ZFS root pool name id: ... state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://www.sun.com/msg/ZFS-8000-6X config: rpool UNAVAIL missing device c0t0d0s2 ONLINE Additional devices are known to be part of this pool, though their exact configuration cannot be determined.

3) Create a symbolic link of the s0 slice for the disk where the ZFS root pool for import exists.

For the disk that there is the ZFS root pool checked in step 2), create the symbolic link of the device special file (/dev/dsk/c0t0d0s0) of s0 slice.

In the following example, use /tmp/work as a work directory.

# mkdir /tmp/work
# ln -s /dev/dsk/c0t0d0s0 /tmp/work/c0t0d0s0

4) Import the ZFS root pool.

# zpool import -d /tmp/work rpool

Note

The messages "cannot mount" and "failed to create mountpoint" may be output. If you can confirm that it has been imported normally in the next step 5), there is no problem.

5) Check the status of imported ZFS root pool.

Check the symbolic link path created in step 3) is displayed, ONLINE is displayed in the state field and no error messages are displayed.

# zpool status rpool
pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 /tmp/work/c0t0d0s0 ONLINE 0 0 0 errors: No known data errors

Note

If ZFS mirror is configured, the following may be displayed.

# zpool status rpool
pool: rpool state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-2Q scrub: none requested config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror DEGRADED 0 0 0 /tmp/work/c0t0d0s0 ONLINE 0 0 0 980382089098729109 UNAVAIL 0 0 0 errors: No known data errors

In this case, after detaching devices other than devices selected in step 3), check the symbolic link path created in step 3) is displayed in the config field, ONLINE is displayed in the state field and, no error messages are displayed.

# zpool detach rpool 980382089098729109
# zpool status rpool
pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 /tmp/work/c0t0d0s0 ONLINE 0 0 0 errors: No known data errors

6) Mount the root file system to restore to /mnt.

6-1) Change the mountpoint property of the root file system to restore to /mnt.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT rpool 6.89G 881M 94K /rpool rpool/ROOT 5.53G 881M 18K legacy rpool/ROOT/s10s_u6wos_07b 5.53G 881M 5.39G / Check the file system name of which MOUNTPOINT is / # zfs set mountpoint=/mnt rpool/ROOT/s10s_u6wos_07b # zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 6.89G 882M 94K /rpool rpool/ROOT 5.53G 882M 18K legacy rpool/ROOT/s10s_u6wos_07b 5.53G 882M 5.39G /mnt Check MOUNTPOINT is changed to /mnt ...

6-2) Mount the root file system to restore.

# zfs mount rpool/ROOT/s10s_u6was_07b

7) Delete all the lines where sysvo1-db is stated from /mnt/kernel/drv/sfdsk.conf.

An example is shown below.

# vi /mnt/kernel/drv/sfdsk.conf
# # Copyright (c) 1998, by Fujitsu, Ltd. # All rights reserved. # name="sfdsk" parent="pseudo"; sysvol-db0="sd","2"; <-Delete this line. sysvol-db1="sd","18"; <-Delete this line. sfdsk_root_volume=1;

8) Check if there is the boot-archive file under /mnt/platform. If there is, execute the bootadm(1M) command and update the boot-archive file.

# ls /mnt/platform/`uname -m`/boot_archive
/mnt/platform/sun4u/boot_archive

If there is the boot-archive file, the file name is displayed as shown above.

In this case, execute the following command. In other cases, do not execute the following command.

# /sbin/bootadm update-archive -R /mnt

9) Unmount the root file system mounted in step 6) and restore the mountpoint property to /, then export the ZFS root pool.

# zfs unmount /mnt
# zfs set mountpoint=/ rpool/ROOT/s10s_u6wos_07b
# zpool export -f rpool

10) Start the system from the disk, confirmed in step 2, where the imported ZFS root pool exists.

The procedure to reboot from a boot disk is different for the following three cases:

For the procedure for SAN Boot environment, see SAN Boot manual.


The disk connected to a 12Gbps SAS card

10a-1) Check the device path of a slice of the restore destination.

Execute the following command to check the device path of a slice. Note and save this path for later reference.

# ls -l /dev/dsk | grep c0t600000E00D0000000001060300040000d0s0
lrwxrwxrwx 1 root root 64 Feb 2 13:57 c0t600000E00D0000000001060300040000d0s0 -> ../../devices/scsi_vhci/disk@g600000e00d0000000001060300040000:a

10a-2) Check the boot disk information.

Execute the following command to check the device path of a slice. Note and save this path for later reference.

# prtconf -v /dev/rdsk/c0t600000E00D0000000001060300040000d0s0
disk, instance #0 Driver properties: ... Hardware properties: ... Paths from multipath bus adapters: Path 33: /pci@8100/pci@4/pci@0/pci@0/LSI,sas@0/iport@f0/disk@w500000e0d0460306,0 (*1) (*2) lsc#3(online) ...

(*1) Disk node name (disk@w500000e0d0460306,0 in this example)
(*2) SAS Address (500000e0d0460306 in this example)

Note

If MPxIO is disabled, the line "Paths from multipath bus adapters:" is not displayed.


10a-3) Enter the OpenBoot environment.

# shutdown -y -g0 -i0

10a-4) Check the device path of SAS HBA that is connected to the disk of the restore destination.

Execute the following command to check the device path of SAS HBA of the restore destination.

ok probe-scsi-all
/pci@8100/pci@4/pci@0/pci@0/LSI,sas@0 (*1) FCode Version 1.00.65, MPT Version 2.05, Firmware Version 4.00.00.00 Target a Unit 0 Disk TOSHIBA MBF2300RC 3706 585937500 Blocks, 300 GB SASDeviceName 50000393d82956d4 SASAddress 500000e0d0460306 PhyNum 0 Target b (*2) Unit 0 Disk TOSHIBA MBF2300RC 3706 585937500 Blocks, 300 GB SASDeviceName 50000393d828bbfc SASAddress 500000e0da0cc620 PhyNum b ... ok

(*1) Device path
(*2) SAS Address (SAS Address checked in *2 in step 10a-2)


10a-5) Set boot-device property.

Concatenate the device path checked in step 10a-4), the disk node name checked in step 10a-2), and the slice name checked in step 10a-1). Set this concatenated value to the boot-device property of OBP.

ok setenv boot-device /pci@8100/pci@4/pci@0/pci@0/LSI,sas@0/disk@w500000e0d0460306,0:a

10a-6) Check if there is boot-device-index property in OpenBoot. If there is, set boot-device-index property to "0."

ok printenv boot-device-index

If there is boot-device-index property, "boot-device-index = number" is displayed. In this case, execute the following command.

If "boot-device-index = number" is not displayed, do not execute the following command.

ok setenv boot-device-index 0

10a-7) Boot the system.

ok boot

The disk of Expansion File Unit connected to a 6Gbps SAS card, or the internal disks of SPARC M12/M10 and SPARC T4-4/T4-2/T4-1/T3-4/T3-2/T3-1 servers

10b-1) Check the slice name of the root partition of the new boot disk. Note and save this slice name for later reference.

[Example for Solaris 11.3 or earlier]

# ls -l /dev/dsk | grep c0t5000CCA00AC113CCd0s0
lrwxrwxrwx 1 root root 48 Apr 19 14:21 c0t5000CCA0150FEA10d0s0 -> ../../devices/scsi_vhci/disk@g5000cca0150fea10:a

[Example for Solaris 11.4 or later]

# ls -l /dev/dsk | grep c0t50000397683346A9d0s0
lrwxrwxrwx 1 root root 48 Jul 27 19:23 c0t50000397683346A9d0s0 -> ../../ devices/scsi_vhci/disk@g50000397683346a9:a

10b-2) Check the boot disk information. Note and save it for later reference.

How you check the information depends on the OS version and whether or not MPxIO is currently enabled.

You can check whether or not MPxIO is enabled by using the current subcommand of the format(1M) command. MPxIO is enabled if the logical path of the boot disk has the /scsi_vhci directory inside.

Example: when the boot disk is c0t50000394882261A0d0

# format c0t50000394882261A0d0
format> current Current Disk = c0t50000394882261A0d0: 00000000 <TOSHIBA-MBF2600RC-3706 cyl 64986 alt 2 hd 27 sec 668> /scsi_vhci/disk@g50000394882261a0 /dev/chassis/SYS/HDD0/disk

[Solaris 11.3 or earlier]

[Solaris 11.4 or later]

10b-3) Enter the OpenBoot environment.

# halt

Note

If the system is started from an emergency startup disk for restoration, enter the OpenBoot environment using the shutdown(1M) command.

10b-4) Set the boot-device property in OpenBoot.

[Solaris 11.3 or earlier]

Concatenate the obp-path parameter output in step 10b-2) and the slice name (:a in this example) output in step 10b-1). Set this concatenated value to the boot-device property.

ok setenv boot-device /pci@400/pci@2/pci@0/pci@e/scsi@0/disk@w5000cca0150fea11,0:a

[Solaris 11.4 or later]

Concatenate the ddi-boot-path parameter output in step 10b-2) and the slice name (:a in this example) output in step 10b-1). Set this concatenated value to the boot-device property.

ok setenv boot-device /pci@8000/pci@4/pci@0/pci@0/scsi@0/disk@w50000397683346aa,0:a

10b-5) Check if there is boot-device-index property in OpenBoot. If there is, set the boot-device-index property to 0.

ok printenv boot-device-index

If there is the boot-device-index property, "boot-device-index = number" is displayed. In this case, execute the following command.

If "boot-device-index = number" is not displayed, do not execute the following command.

ok setenv boot-device-index 0

10b-6) Boot the system.

ok boot

Note

The following warning messages may be output during the booting process, but there is no problem. The current swap area is set as a dump device. If you want to change the setting of a dump device, use dumpadm(1M).

savecore: open("/dev/sfdsk/System/dsk/swap"): No such file or directory
dumpadm: cannot use /dev/sfdsk/System/dsk/swap as dump device:
                           No such file or directory.

Other than the above boot disks

10c-1) Check the device path of the root partition of the new boot disk. Note and save this device path for later reference.

# ls -l /dev/dsk | grep c0t0d0s0
lrwxrwxrwx 1 root root 63 Aug 20 12:00 c0t0d0s0 -> ../../devices/pci@1f,4000/scsi@3/sd@0,0:a

10c-2) Enter the OpenBoot environment.

# halt

Note

If the system is started from an emergency startup disk for restoration, enter the OpenBoot environment using the shutdown(1M) command.

10c-3) Execute the show-devs command in the OpenBoot environment and display the device name.

ok show-devs /pci@1f,4000/scsi@3
... /pci@1f,4000/scsi@3/disk ...

10c-4) Set the boot-device property in OpenBoot.

10c-5) Check if there is the boot-device-index property in OpenBoot. If there is, set the boot-device-index property to 0.

ok printenv boot-device-index

If there is boot-device-index property, "boot-device-index = number" is displayed. In this case, execute the following command.

If "boot-device-index = number" is not displayed, do not execute the following command.

ok setenv boot-device-index 0

10c-6) Boot the system.

ok boot

Note

The following warning messages may be output during the booting process, but it does not affect the system. The current swap area is set as a dump device. If you want to change the setting of a dump device, use dumpadm(1M).

savecore: open("/dev/sfdsk/System/dsk/swap"): No such file or directory
dumpadm: cannot use /dev/sfdsk/System/dsk/swap as dump device:
                           No such file or directory.

11) Delete the private slice of the system disk.

Check the partition configuration of the system disk using the prtvtoc(1M) command and if the private slice exists (the slice which the tag is 33), delete it using the sdxsystem Format command.

# prtvtoc /dev/rdsk/c0t0d0s2
# /etc/opt/FJSVsdx/bin/sdxsystem Format -d c0t0d0

12) Initialize the disk which was the mirror destination of the system disk.

# dd if=/dev/zero of=/dev/rdsk/c1t0d0s0 bs=1024k
# format -e

For the format -e command, select the disk (c1t0d0) which was the mirror destination, and input "y" to the inquiry "Disk not labeled. Label it now?"


13) If it is the cluster system, delete the cluster resource of the root class.

The target is the only root class. Do not delete the cluster resource of local classes and shared classes.

The following examples show the case where the following classes exist:

Check whether the class resource of the root class exists.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -I
NAME SCOPE RID(RDB) RID(CDB) --------------- -------------------- --------------- --------------- System node1 368 1 class0001 node1 369 2 class0002 node1,node2 371,372(370) 4,5(3) #

If the class resource of the root class is not displayed, go to the next step 14).

If it is displayed, delete the class resource of the root class.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -R -c System
sdxdcrsc: System removed #

Confirm that the class resource of the root class has been deleted.

Execute the following command and check that the target class name is not displayed.

# /etc/opt/FJSVsdx/bin/sdxdcrsc -I
NAME SCOPE RID(RDB) RID(CDB) --------------- -------------------- --------------- --------------- class0001 node1 369 2 class0002 node1,node2 371,372(370) 4,5(3) #

14) Re-configure mirroring of the system disk.

For details, see "7.1.1 System Disk Mirroring Settings in a ZFS Boot Environment (GDS Mirroring)" or "7.2.1 System Disk Mirroring Settings in a ZFS Boot Environment (ZFS Mirroring)."

Note

The following may be output at the time of re-setting the mirroring.

# sdxdisk -M -c System -a type=root -d c0t0d0=Root1:keep
SDX:sdxdisk: INFO: System: created class SDX:sdxdisk: ERROR: c0t0d0: already exists in ... SDX:sdxdisk: INFO: System: removed class #

When the above errors are output, re-boot the system and set the mirroring again.


(8) The system cannot be booted (ufs logging errors).

Explanation

When backing up a ufs file system on a system disk using the procedure described in "a) When booting the system from a CD-ROM device (or the network) and creating backups" in "6.3.1 Backing Up," if step a1) is not performed for disk disconnection, the following errors may be output on OS startup and starting the OS may fail.

WARNING: Error processing ufs log data during scan 
WARNING: ufs log for /var changed state to Error 
WARNING: Please umount(1M) /var and run fsck(1M) 
WARNING: Error accessing ufs log for /var; Please run fsck(1M) 
mount: I/O error
mount: /dev/sfdsk/RootClass/dsk/varVolume cannot be mounted.
mount: mount-point /var/run does not exist.

Resolution

Restore the backed up file system.

For the restore methods, see "6.3.2 Restoring (When the System Can Be Booted)" or "6.3.3 Restoring (When the System Cannot Be Booted)."

Even if operation is performed as described in [Explanation], an error may not occur immediately.
Additionally, if the fsck(1M) command is used according to the messages shown in [Explanation], it may be possible to start the OS and it appears that the file system was restored. However, in these situations, file system data and the synchronized status of mirrors are not ensured.

When performing operation as described in [Explanation], be sure to restore the file system.


(9) The system cannot be booted (Alias name error of the boot device).

Explanation

After mirroring the system disk, aliases of the boot devices are registered with the OpenBoot nvramrc variable. The alias names may not be correctly registered because the number of bytes in nvramrc is restricted. In this case, if you specify the alias name to boot the system, the following message is output and the system cannot be booted.

{0} ok boot sdx-boot-alt-device-32-2b
Boot device: /pci@8600/pci@4/pci@0/pci@9/LSI,sas@  File and args:
Boot load failed
ERROR: boot-read fail

Resolution

Specify either another boot device or a path of the boot device to boot the system.

Example 1: To boot the system by specifying another boot device

{0} ok printenv nvramrc
nvramrc =  devalias net /pci@8600/pci@4/pci@0/pci@11/network@0
           devalias disk /pci@8600/pci@4/pci@0/pci@9/LSI,sas@0/disk
           devalias sdx-root-slice-32-1
 /devices/scsi_vhci/disk@g600000e00d1100000010003101800000:a,raw
           devalias sdx-boot-device-32-1
 /pci@8600/pci@4/pci@0/pci@9/LSI,sas@0/disk@w500000e0d4003185,1:a
           ...

{0} ok boot sdx-boot-device-32-1

Example 2: To boot the system by specifying a path of the boot device

{0} ok boot /pci@8600/pci@4/pci@0/pci@9/LSI,sas@0/disk@w500000e0d4003195,0:a