Top
PRIMECLUSTER Global Disk Services  Configuration and AdministrationGuide 4.6
FUJITSU Software
Appendix D Troubleshooting > D.1 Resolving Problems > D.1.5 System Disk Abnormality [EFI]
PreviousNext

D.1.5 System Disk Abnormality [EFI]

The following description shows resolutions to problems relevant to system disks on which the file systems / (root), /usr, and /var operates.

In one of the following circumstances, take the actions as indicated for the relevant situation.

(1) Create backup of root file system.

Explanation

Mirroring the system disk will protect data in case one of the physical disks crashes. However, to restore data damaged due to a critical fault as a result of multiple failures and so on or by an operational mistake, create backup data in advance.

Resolution

See "7.4.2 Backing Up [EFI]."


(2) System can be booted, but the system disk data is invalid.

Explanation

For some reason, the system disk data is invalid. You must restore data from the backup data created in advance.

Resolution

See "7.4.5 Restoring (When System Can Be Booted) [EFI]."


(3) System cannot be booted. (Failure of original boot disk)

Explanation

Even when you have the system disk mirrored, you may experience unsuccessful booting. For example, you may encounter such a problem when the file accessed during the booting process in the boot disk is physically damaged, or when a disk is swapped improperly.
If the console message of the unsuccessful boot disk indicates such is the case, try booting from the other boot disk, i.e. the mirror disk.

Resolution

1) From boot disk units displayed in the EFI boot manager's boot option selection screen, select a mirror disk and boot.
Among the SDX disk names, the top one is the original boot disk unit name and the others are the mirror disk names. When mirroring is configured with three-way or more multiplexing, multiple mirror disks exist. In the following example, Root1 is the original boot disk name and Root2 and Root3 are the mirror disk names.

Root1
Root2
Root3

2) If booting is successful, find out what caused the original disk to fail, and recover the disk by swapping disks.

See

For details on disk swap, see "B.1.8 sdxswap - Swap Disk" and resolutions described in "7.3.1.2 Operation Procedure."

When the procedures above cannot resolve the problem, causes other than data errors in the original boot disk unit may exist, or data errors may also exist in the mirror disk.
If it would appear that data errors also exist in the mirror disk, see "(5) System cannot be booted. (Boot disk data damage)."


(4) System cannot be booted. (Failure of all boot disks)

Explanation

Even when you have the system disk mirrored, you may experience unsuccessful booting.

The following describes the restoration procedure when all the boot disks physically fail simultaneously.

For the following procedure, swap all the physical disks registered in a root class and restore the system disk data based on collected backups. Then, re-construct the mirroring configuration.

Also, this section explains how to perform the mirroring cancellation to boot the system without swapping a physical disk or restoring data of the system disk by assuming that booting cannot be performed by some abnormality other than a physical disk failure.

Note

Cancel the mirroring of the normal system disk by the procedure in "9.4 System Disk Mirroring Cancellation [EFI]."

Note

To perform restoration according to this procedure, you must collect the following information beforehand when setting the system disk mirroring:

  • Partition configuration of the system disk

  • Backup of the system disk

For details, see "6.5 System Disk Mirroring Settings [EFI]."

Resolution

Perform restoration according to the following procedure.

1) Swap all the physical disks registered in the root class. You must swap them according to the procedure defined for each device.

Skip this step if the disks do not have a failure.

2) Boot the system from a CD-ROM device.

3) Set the new system disk to the original partition configuration.

See

For details on the parted command, see the manual for the parted(8) command.

3-1) Start the parted command.

# parted /dev/sda

3-2) Create the disk label (partition table) of the GPT type with the mklabel subcommand.

(parted) mklabel gpt

3-3) Before setting the system disk mirroring, create a partition according to the partition configuration checked in step 1) in "6.5 System Disk Mirroring Settings [EFI]."

3-3-1) Create the partition with the mkpart subcommand.

3-3-2) Set the boot partition.

Example: When the file system type is FAT and the partition number is 1

(parted) set 1 boot on

3-4) Confirm that the partition configuration is the same as the one checked in step 1) in "6.5 System Disk Mirroring Settings [EFI]" with the print subcommand. 

(parted) print
Disk /dev/sdf: 34.4GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name                  Flags
 1      1049kB  630MB   629MB   fat32        EFI System Partition  boot, esp
 2      630MB   15.0GB  14.4GB  xfs
 3      15.0GB  17.0GB  2000MB  xfs
 4      17.0GB  19.0GB  2001MB  xfs
 5      19.0GB  21.0GB  2000MB  xfs
 6      21.0GB  23.0GB  2001MB  xfs
 7      23.0GB  25.0GB  2000MB  linux-swap(v1)                     swap

Note

If a disk was not swapped in step 1), the private slice of GDS remains. At this time, if you execute the pint subcommand, the private slice of GDS is displayed. Among the partitions displayed by the print subcommand, the partitions excluding the partitions that were checked in step 1) of "6.5 System Disk Mirroring Settings [EFI]" is the private slice of GDS. In this case, delete the private slice of GDS with the rm subcommand of the parted command.

Example: When the partition number (displayed in Minor filed) is 8

(parted) rm 8

Note

When a mirror destination disk is not replaced in step 1), remove all the partitions of the mirror destination disk.

Example: When the mirror destination disk is /dev/sdc 

parted /dev/sdc
(parted) rm 1
(parted) rm 2
(parted) rm 3
(parted) rm 4
(parted) rm 5
(parted) rm 6
(parted) rm 7
(parted) rm 8

3-5) Exit the parted command.

(parted) quit

4) Restore the backup data collected in a tape medium and so on to each physical slice.

Take this step if the physical disk is swapped in step 1) or if the system disk data becomes invalid.

See

For the restoration, see the manuals of the OS, file systems to be restored and used commands.

5) Cancel the setting of the system disk mirroring.

5-1) Mount the EFI system partition and the root partition.
In the following example, the /work/efi, /work/root, and /work/boot directories are used as temporary mount points.

# mkdir /work
# mkdir /work/efi
# mkdir /work/root
# mkdir /work/boot
# mount /dev/sda7 /work/efi
# mount /dev/sda5 /work/root
# mount /dev/sda6 /work/boot

Information

If you have never performed the operation to update the initrd file (such as updating kernels) after mirroring the system disk, you can cancel the setting easily and quickly by performing the procedure "Editing procedure of the grub.cfg file when the initrd file is not updated." in Information described later instead of steps 5a-2) and 5a-3).

Execute this procedure only when you can ensure that the operation to update the initrd file has not been performed. In other cases, perform the following steps 5a-2) through 5a-12).

5-2) Delete the information of GDS from the initrd file.

5-2-1) Check the entry name.
Check the entry name described in the set default entry in the /work/efi/EFI/redhat/grub.cfg file. 

# more /work/efi/EFI/redhat/grub.cfg
### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="GDS sysvol entry"
fi

5-2-2) Check the initrd file name.
Check the initrd file name of the entry checked in step 5a-2-1) among the entries in the grub.cfg file.

In the following example, the initrd file name is initramfs-3.10.0-123.el7.x86_64.img. 

# more /work/efi/EFI/redhat/grub.cfg
menuentry 'GDS sysvol entry' --class red --class gnu-linux --class gnu  --class os --unrestricted {
                       (*1)
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod xfs
        insmod regexp
        regexp -s device '(hd[^,]+)'
        set root="$device,gpt2"
        linuxefi /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/sfdsk/gdssys2
        ro vconsole.keymap=jp106 crashkernel=auto
        vconsole.font=latarcyrheb-sun16 rhgb quiet LANG=ja_JP.UTF-8
        rd.driver.post=sfdsksys
        initrdefi /initramfs-3.10.0-123.el7.x86_64.img 
                                                   (*2)
}
(*1) Entry name checked in step 5a-2-1)
(*2) initrd file name

5-2-3) Create two work directories in the /work/root directory.

Create directories with different names from existing ones.

# mkdir /work/root/work_directory1
# mkdir /work/root/work_directory2

5-2-4) Extract the initrd file to each work directory.

In the following procedure, extract the initrd file to the work directory created in step 5a-2-3.

If in the following procedure, the message "no space left on device" is output because the file system mounted on the /work/root does not have enough space, mount another file system with enough space. Also create work directories and then extract the initrd file.

1) Check that the initrd file contains Early CPIO image.

If "Early CPIO image" is output when the following command is executed, the initrd file contains Early CPIO image. 

# /work/root/usr/bin/lsinitrd /work/boot/initramfs-3.10.0-123.el7.x86_64.img \
                                                                    (*1)
2>/dev/null | grep -e "^Early CPIO image"
(*1) initrd file name checked in step 5a-2-2)

2) If the initrd file contains Early CPIO image, take the procedure below to extract the initrd file. 

# cd /work/root/work_directory1
# cpio -idm < /work/boot/initramfs-3.10.0-123.el7.x86_64.img 
                                                       (*1)
# cd /work/root/work_directory2
# /work/root/usr/lib/dracut/skipcpio /work/boot/initramfs-3.10.0-123.el7.x86_64.img \
| gunzip -c - | cpio -idm                                                     (*1)
(*1) initrd file name checked in step 5a-2-2)

3) If the initrd file does not contain Early CPIO image, take the procedure below to extract the initrd file. 

# cd /work/root/work_directory2
# gunzip -c /work/boot/initramfs-3.10.0-123.el7.x86_64.img | cpio -idm
                                                      (*1)
(*1) initrd file name checked in step 5a-2-2)

5-2-5) Remove GDS drivers from extracted initrd. 

# rm /work/root/work_directory2/lib/modules/3.10.0-123.el7.x86_64/extra/FJSVsdx-drvcore/sfdsksys.ko
# rm /work/root/work_directory2/lib/modules/3.10.0-123.el7.x86_64/extra/FJSVsdx-drvcore/sfdsk_lib.ko

5-2-6) Remove shell scripts to read GDS drivers from extracted initrd.

Delete the following files if they exist:

# rm /work/root/work_directory2/sbin/ins_sfdsksys.sh
# rm /work/root/work_directory2/usr/lib/dracut/hooks/cmdline/01-parse-ins_sfdsksys.sh

5-2-7) Compress edited initrd.

1) If the initrd file contains Early CPIO image in 1) of step 5a-2-4), take the procedure below to compress initrd.

# cd /work/root/work_directory1
# find . -print0 | cpio --null -H newc -o --quiet > /work/boot/initrd-new.img
# cd /work/root/work_directory2
# find . -print0 | cpio --null -H newc -o --quiet |  gzip -9 >> /work/boot/initrd-new.img

2) If the initrd file does not contain Early CPIO image in 1) of step 5a-2-4), take the procedure below to compress initrd.

# cd /work/root/work_directory2
# find . -print0 | cpio --null -H newc -o --quiet | gzip -9 > /work/boot/initrd-new.img

5-2-8) Delete the work directories created in the step 5a-2-3).

# cd /
# rm -rf /work/root/work_directory1
# rm -rf /work/root/work_directory2

5-2-9) Place the initrd file compressed in step 5a-2-7) in the EFI system partition.

Back up the initrd file checked in step 5a-2-2), and then place the initrd file compressed in step 5a-2-7) as the same name of the initrd file checked in step 5a-2-2). 

# cd /work/boot
# mv initramfs-3.10.0-123.el7.x86_64.img initramfs-3.10.0-123.el7.x86_64.img.gdsbak
# mv /work/boot/initrd-new.img /work/boot/initramfs-3.10.0-123.el7.x86_64.img

5-3) Edit the grub.cfg file.

# vim /work/efi/EFI/redhat/grub.cfg

Change the entry checked in step 5a-2-1) among the grub.cfg file entries as follows:

(*1) Entry name checked in step 5a-2-1) 
(*2) Change the root device name from the path of the GDS's logical volume to UUID.
     (Check UUID by executing the blkid command.)
(*3) Delete the description of rd.driver.post=sfdsksys.

Information

Editing procedure of the grub.cfg file when the initrd file is not updated.

If you have never performed the operation to update the initrd file (such as updating kernels) after mirroring the system disk, you can cancel the setting easily and quickly by performing the following procedure instead of steps 5-2) and 5-3).

  1. Check the entryname.
    Check the entryname described in the set default entry in the /work/efi/EFI/redhat/grub.cfg file. 

    ### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="GDS sysvol entry"
fi

  2. Edit the grub.cfg file.

    # vim /work/efi/EFI/redhat/grub.cfg

    Change the kernel image setting after setting the system disk mirroring defined in the grub.cfg file to a setting which does not mirror the system disk. The entry checkd in step 1 displayed is the kernel image setting after mirroring the system disk among entries in the grub.cfg file. Examples are shown below.

    • Example before edit 

      menuentry 'GDS sysvol entry' --class red --class gnu-linux --class gnu  --class os --unrestricted {
                       (*1)
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod xfs
        insmod regexp
        regexp -s device '(hd[^,]+)'
        set root="$device,gpt2"
        linuxefi /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/sfdsk/gdssys2
                                                                    (*2)
        ro vconsole.keymap=jp106 crashkernel=auto
        vconsole.font=latarcyrheb-sun16 rhgb quiet LANG=ja_JP.UTF-8
        rd.driver.post=sfdsksys
                            (*3)
        initrdefi /initramfs-3.10.0-123.el7.x86_64.img 
                                                   (*4)
}

    • Example after edit 

      menuentry 'GDS sysvol entry' --class red --class gnu-linux --class gnu  --class os --unrestricted {
                        (*1)
        load_video
        set gfxpayload=keep
              insmod gzio
        insmod part_gpt
              insmod xfs
        insmod regexp
        regexp -s device '(hd[^,]+)'
        set root="$device,gpt2"
        linuxefi /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=ef9cab98-5b1d-4fe2-89e6-4e1548778338
                                                                                           (*2)
        ro vconsole.keymap=jp106 crashkernel=auto
        vconsole.font=latarcyrheb-sun16 rhgb quiet LANG=ja_JP.UTF-8
        initrdefi /initramfs-3.10.0-123.el7.x86_64.img.gdsbak
                                                          (*4)
}
      (*1) Entry name checked in step 1
(*2) Change the root device name from the path of the GDS's logical volume to UUID.
     (Check UUID by executing the blkid command.)
(*3) Delete the description of rd.driver.post=sfdsksys.
(*4) Add .gdsbak for the final entry of the initrd file.

5-4) Edit the fstab file.

# vim /work/root/etc/fstab

Edit the mount setting lines of volumes in the root class described in the fstab file according to the partition configuration of the new system disk. Examples are shown below.

5-5) Edit the dracut.conf file. If the patch for PH15096 is applied, this procedure is not required.

# vim /work/root/etc/dracut.conf

Delete the description of the GDS driver from the dracut.conf file. 

(*1) Description of the GDS driver
(*2) Leave one space for a single-byte character and delete sfdsksys.

5-6) Edit the /etc/default/grub file.

Change the setting of GRUB_DEFAULT in the /etc/default/grub file.
Delete rd.driver.post=sfdsksys from the file as well.

# vim /work/root/etc/default/grub
(*1) Kernel version that is included in the initrd file name checked in step 5a-2-2)

5-7) Remove the following directories if they exist:

5-8) Change the permission of the root_mknod.sh file.

# chmod 400 /work/root/etc/opt/FJSVsdx/RootMirror/root_mknod.sh

5-9) Delete the information of GDS configuration databases set in the EFI environment variables. 

# cd /sys/firmware/efi/vars/
# cat sysvol-db2-*/raw_var > del_var

5-10) Unmount the EFI system partition and the root partition. 

# cd /
# umount /work/efi
# umount /work/root
# umount /work/boot
# rmdir /work/efi
# rmdir /work/root
# rmdir /work/boot
# rmdir /work

5-11) Restart the system. 

# shutdown -r now

5-12) After the system is restarted, execute the following command to update the grub.cfg file. 

# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

6) Add the new boot disk to a boot option of the EFI boot manager (BIOS for PRIMEQUEST 3000).
The procedure differs depending on the OS version.


6-1) Select [Boot Maintenance Manager] in the boot option selection screen of the EFI boot manager (Boot tab of BIOS for PRIMEQUEST 3000).

6-2) Select [Boot Option] in the Boot Maintenance Manager screen.

6-3) Select [Add Boot Option] in the Boot Options screen.

6-4) When the Add Boot Option screen is displayed, add the new boot disk to the boot option.

  1. Select the device corresponding to /dev/sda from the device list.

    For the method to recognize the device, see the manual of EFI.

  2. After the contents in the root directory of the selected disk are displayed, select "EFI."

  3. After the contents in the EFI directory are displayed, select "redhat."

  4. After the contents in the redhat directory are displayed, select "shim.efi" (for RHEL7) or "shimx64.efi" (for RHEL8).

  5. Select "Input the description."

  6. Input a character string used for a label.

    For example, input "New_sda" here.

  7. Select "Commit Changes and Exit."

  8. For PRIMEQUEST 3000, input [Esc] and exit [Boot Options].

  9. Input [Esc] to exit [Boot Maintenance Manager].

6-5) For PRIMEQUEST 3000, select [Boot Override] on the Exit tab of BIOS.
For other than PRIMEQUEST 3000, select [Boot Manager] in the boot option selection screen of the EFI boot manager.

6-6) Select the label name input in 6. of step 6-4) in the option selection screen (Boot Override screen for PRIMEQUEST 3000) and boot the system.


7) Delete the SDX disk items from the boot option of the EFI boot manager.

See

For details on the efibootmgr command, see the manual of efibootmgr command.

7-1) Check the current setting of the EFI boot manager.

# efibootmgr
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0001,0002,0000
Boot0000* sda
Boot0001* RootDisk1
Boot0002* RootDisk2

In this example, the SDX disk items are RootDisk1 of the number 0001 and RootDisk2 of the number 0002.

7-2) Delete the SDX disk items from the boot option.

# efibootmgr -B -b 0001
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0002,0000
Boot0000* sda
Boot0002* RootDisk2
# efibootmgr -B -b 0002
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0000
Boot0000* sda

8) Reconfigure the system disk mirroring.

See

For details on the setting methods, see "6.5 System Disk Mirroring Settings [EFI]."


(5) System cannot be booted. (Boot disk data damage)

Explanation

For some reason, the system disk data is invalid. You must restore data from the backup data created in advance. If restoring data from the mirror disk was unsuccessful, follow the procedures shown below.

Resolution

See "7.4.6 Restoring (When System Cannot Be Booted) [EFI]."

(6) System cannot be booted. (Removing errata kernel)

Explanation

After an errata kernel has been removed in an environment where the system disk is mirrored, and if the grub.cfg file is not updated by using the grub2-mkconfig command before the system is restarted, the system may not be started.

See

For notes on removing an errata kernel, see "7.18 Removing Errata Kernel."

Resolution

  1. To start the system, select the boot entry of "GDS sysvol entry for Red Hat Enterprise Linux Server, ..." format from the boot option selection screen of the EFI boot manager.

  2. After the system is started, execute the grub2-mkconfig command to update the grub.cfg file.

    # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

  3. If no boot entry exists in step 1 above, follow "(4) System cannot be booted. (Failure of all boot disks)" to restore the system. The disk is not necessary to be replaced at this time.

PreviousNext
Copyright FUJITSU LIMITED 2019-2022