Top
PRIMECLUSTER Global Disk Services  Configuration and AdministrationGuide 4.7
Appendix D Troubleshooting > D.1 Resolving Problems > D.1.14 OS Messages
PreviousNext

D.1.14 OS Messages

For OS messages, in one of the following circumstances, take the actions as indicated for the relevant situation.

(1) Messages are output at node startup.

Explanation

At node startup, the following message may be output.

sfdsksys: no version for "sfdskd_event_exist" found: kernel tainted.

Some or all of the following messages may be output to the console and the syslog during system startup. XXXXXXXX indicates 40 digit hexadecimal.

kernel: Request for unknown module key 'FUJITSU Software: Fujitsu BIOS DB FJMW Certificate: XXXXXXXX' err -11
kernel: sfdsk_lib: loading out-of-tree module taints kernel.
kernel: sfdsk_lib: module verification failed: signature and/or required key missing - tainting kernel

Resolution

Even if this message is output, the system is not affected. There is no need to work around.


(2) A SCSI ioctl related message is output at node startup or command execution.

Explanation

The message "program sdxservd is using a deprecated SCSI ioctl, please convert it to SG_IO" may be output when:

Resolution

Even if this message is output, the system is not affected. There is no need to work around.


(3) Stack information is output at node startup and command execution.

Explanation

Stack information starting with the "Call Trace:" message may be output on the console and the system log when:

Resolution

Even if this message is output, the system is not affected. There is no need to work around.


(4) A devlabel message is output at node startup or during collection of investigation material.

Explanation

At node startup or during collection of investigation material (when executing pclsnap, sdxsnap.sh, or FJQSS), the following message may be output.

devlabel: devlabel's temporary ignore list /etc/sysconfig/devlabel.d/ignore_list has been emptied due to a change in device configuration.

devlabel: The device /dev/sdXX is being put in devlabel's temporary ignore list /etc/sysconfig/devlabel.d/ignore_list to avoid errors.

devlabel: The device old_device is now known as new_device./etc/opt/FJSVsdx/.devlabel/class/sdx_dev_999_999 now points to the new name.

devlabel: The symlink /etc/opt/FJSVsdx/.devlabel/class/sdx_dev_999_999 -> device is being ignored in /etc/sysconfig/devlabel because the correct device cannot be found.

devlabel: devlabel service started/restarted

Resolution

Even if this message is output, the system is not affected. There is no need to work around.


(5) A message about the sfdsksys module is output when the errata kernel is applied.

Explanation

If an errata kernel is applied in an environment where the system disk is mirrored, the following OS error message may be output on the console.

Failed to install module sfdsksys

Resolution

Execute the following command to check if the installed errata kernel version is displayed.

# rpm -q kernel

When the installed errata kernel version is displayed correctly, the errata kernel is properly applied. No corrective action is required.

If the installed errata kernel version is not displayed, contact field engineers.


(6) Messages are output at node startup, and GDS cannot be started.

Explanation

If Secure Boot is enabled in the environment that is not capable of Secure Boot, the following messages may be output to the console or the system log. In this case, GDS may not be started. XXXXXXXX indicates 40 digit hexadecimal.

kernel: Request for unknown module key 'FUJITSU Software: Fujitsu BIOS DB FJMW Certificate: XXXXXXXX' err -11
bash: modprobe: ERROR: could not insert 'sfdsk_lib': Required key not available
bash: modprobe: ERROR: could not insert 'sfdsksys': Required key not available
bash: modprobe: ERROR: could not insert 'sfdsk': Required key not available
systemd: Failed to start PRIMECLUSTER Global Disk Services (fjsvsdx) service.

Resolution

If Secure Boot is enabled in the environment that is not capable of Secure Boot, disable it.


(7) A message is output at node startup, and the system cannot be started.

Explanation

If system disk mirroring is set in the environment that is not capable of Secure Boot and Secure Boot is enabled, the following message may be output to the console or the system log. In this case, the system may not be started.

bash: modprobe: ERROR: could not insert 'sfdsksys': Required key not available

Resolution

If Secure Boot is enabled in the environment that is not capable of Secure Boot, disable it.


(8) A message about the ext4 file system is output at node stop.

Explanation

At node stop, the following message may be output to the console.

end_request: I/O error, dev dev, sector NNNNNNNN EXT4-fs error (device device):
ext4_find_entry: reading directory #inode offset N

dev is the device name of the system disk. N is a number. device is the mount device of the file system that includes the /etc/opt directory. inode is the inode number of the directory where an error occurred.

Resolution

Execute the following command to check the inode number of the /etc/opt/FJSVsdx directory.

# ls -ldi /etc/opt/FJSVsdx
41391133 drwxr-xr-x. 4 root root 4096 Oct  2 14:03 /etc/opt/FJSVsdx
   inode number

If the inode number is consistent with inode, the output of this message means that the sdxmond daemon of GDS is referring a file in the /etc/opt/FJSVsdx directory at node stop. In this case, the system is not affected by this status. No resolution is required.

If the inode number is not consistent with inode, this message is unlikely to be considered as a message regarding GDS error. Contact field engineers.

(9) Messages of systemd or systemd-udevd are output when the system is started or the operations of GDS are executed.

Explanation

When the system is started or the operations of GDS are executed, the following messages of systemd or systemd-udevd are output to the console and the /var/log/messages file.

(a) systemd-udevd: error: device: Device or resource busy

device is the physical disk of the disk that is registered in the shared class or the local class (example: /dev/sdf).

This message is output in the following cases:

  • A disk is registered to the shared class or the local class.

  • The system is started.

(b) systemd-udevd: error: device: Device or resource busy

device is the partition on the physical disk of the disk that is registered in the shared class or the local class (example: /dev/sdf2).

This message is output in the following cases:

  • A volume of the shared class or the local class is created.

  • The system is started.

(c) systemd-udevd: error: device: No such device or address

device is the device of the GDS volume (example: /dev/sfdsk32, /dev/sfdsk/gdssys2, /dev/sfdsksys2).

This message is output in the following cases:

  • A volume of any class type is created.

  • The system is started.

(d) systemd-udevd: inotify_add_watch(N, device, M) failed: No such file or directory

device is the partition for a volume of the root class (example: /dev/sdd5).

N and M indicate numerical values.

This message is output when:

  • "System Disk Settings" is executed.

  • "Unmirror System Disk" is executed.

  • "Swap Physical Disk" is executed for the system disk.

  • "Restore Physical Disk "is executed for the system disk.

(e) systemd: Device device appeared twice with different sysfs paths device1 and device2

device, device1, and device2 are the text strings that indicate the partitions.

device1 and device2 are the partitions for the volume, or the volume devices.

[Example 1]

systemd: Device dev-disk-by\x2duuid-0028c72b\x2dad94\x2d4b27\x2d912b\x2dbe389de83cd2.device appeared twice with different sysfs paths
/sys/devices/pci0000:00/0000:00:10.0/host2/target2:0:0/2:0:0:0/block/sda/sda2 and
/sys/devices/pci0000:00/0000:00:10.0/host2/target2:0:1/2:0:1:0/block/sdb/sdb2

[Example 2]

systemd: Device dev-disk-by\x2duuid-75C6\x2d77DC.device appeared twice with different sysfs paths
/sys/devices/virtual/block/sfdsk!gdssys6 and
/sys/devices/pci0000:00/0000:00:15.0/0000:03:00.0/host2/target2:0:1/2:0:1:0/block/sdb/sdb1

This message is output when:

  • The system is started.

Resolution

Even if this message is output, the system is not affected. No resolution is required.

(10) Messages of multipathd are output when creating volumes.

Explanation

When creating volumes, the following messages may be output to the console and the system log. X is numerical value.

multipathd: sfdskX: triggering change event to reinitialize
multipathd: sfdskX: failed to get udev uid: Invalid argument
multipathd: failed to get path uid
multipathd: uevent trigger error

Resolution

Even if this message is output, the system is not affected. No resolution is required.

For inhibiting the message output, add the following description in the black list of the DM-MP setting file "/etc/multipath.conf."

devnode "^sfdsk[0-9]+$"

A description example:

blacklist {
        devnode "^sfdsk[0-9]+$"
}

(11) The message about SELinux is output.

Explanation

If the mirroring among servers is used, the following message about SELinux may be output after a node is restarted or a network error is restored.

setroubleshoot: failed to retrieve rpm info for /var/opt/FJSVsdx/log/file_name
setroubleshoot: Plugin Exception restorecon_source
setroubleshoot: SELinux is preventing /usr/sbin/iscsiadm from read access on the file /var/opt/FJSVsdx/log/file_name. For complete SELinux messages. run sealert -l ID
python: SELinux is preventing /usr/sbin/iscsiadm from read access on the file /var/opt/FJSVsdx/log/file_name.

Resolution

Even if this message is output, the system is not affected. No resolution is required.

(12) The device name of GDS volume is displayed in the sfdskX format, therefore the message for which volume is unknown.

Explanation

The device name of GDS volume may be displayed in the sfdskX format for the OS message.

Example) kernel: XFS (sfdsk33): Internal error ...

Resolution

The GDS volume corresponding to the sfdskX format device displayed for the OS message can be checked by the following procedure.

  1. Check the major and minor numbers of sfdskX format device.

    In the following example, the sfdsk33 major number is 487 and minor number is 33.

    Example) How to check the sfdsk33 major and minor numbers

    # ls -l /dev/sfdsk33
brw-rw----. 1 root disk 487, 33 Feb 21 16:32 /dev/sfdsk33

  2. Search for the volume coincident with the major and minor numbers checked in step 1.

    In the following example, the volume in which the major number is 487 and the minor number is 33

    is /dev/sfdsk/class0001/dsk/volume2. Therefore, the volume corresponding to sfdsk33 is prove to be

    /dev/sfdsk/class0001/dsk/volume2.

    Example)

    # ls -l /dev/sfdsk/*/dsk
/dev/sfdsk/class0001/dsk:
total 0
brw-------. 1 root root 487, 32 Feb 21 16:31 volume1
brw-------. 1 root root 487, 33 Feb 21 16:32 volume2
brw-------. 1 root root 487, 34 Feb 21 16:32 volume3
...

(13) A message is output from the standby node when the node is restarted, and the cluster cannot be started.

Explanation

When you reboot a node in a VMWare environment that uses I/O fencing, the cluster on the standby node may not start properly with the following message in the system log on the standby node.

kernel: sd X:X:X:X: reservation conflict

kernel: WARNING: sfdsk: write error on disk:

Resolution

Make sure that you have configured the I/O fencing feature of the GDS.

For details of the settings, refer to "Installation and Configuration of Related Software" under "Using PRIMECLUSTER in a VMware Environment" described in "PRIMECLUSTER Installation and Administration Guide."

(14) When the stop operation of the GDS volume is executed, an error message is output, and the stop operation of the GDS volume fails.

Explanation

In a RHEL 8.1 or later environment, if Device Mapper Multipath (DM-MP) is registered in the GDS class, the following message is output to the console when a GDS volume is stopped, and stopping a GDS volume may fail.

ERROR: volume: device busy on node node

In the above case, one of the following messages is output to the system log. X is a number.

multipathd: sfdskX: triggering change event to reinitialize

multipathd: sfdskX: failed to get udev uid: Invalid argument

multipathd: failed to get path uid

multipathd: uevent trigger error

multipathd: sfdskX: unusable path (wild) - checker failed

Resolution

The following procedure must be performed to avoid failure.

This procedure can be performed during the service.

  1. Add the following description in the black list of the DM-MP setting file "/etc/multipath.conf".

    [A description example]

        blacklist {
            devnode "^sfdsk[0-9]+$"
    }

  2. Reload the multipathd daemon.

    # systemctl reload multipathd.service

PreviousNext
Copyright Fujitsu Limited 2023