Top
PRIMECLUSTER Global Disk Services  Configuration and AdministrationGuide 4.3
FUJITSU Software

Preface

This manual describes the function, setting up, and managing of the following software:

Software (Feature)

Abbreviation

Description

Global Disk Services

GDS

Volume management software provided by PRIMECLUSTER.

Global Disk Services Snapshot

GDS Snapshot

Optional software of GDS provided by PRIMECLUSTER GDS Snapshot.


Purpose

This manual aims to help users understand GDS and GDS Snapshot, including environment configuration, operation and maintenance.


Target Reader

This manual is devoted to all users who will operate and manage GDS and GDS Snapshot and programmers who will create applications running in GDS environments.


Organization

This manual consists as follows:

Headers

Contents

Chapter 1
Functions

Explains the features and function overview of GDS and GDS Snapshot.

Chapter 2
Structure

Explains the concepts and rules of objects used by GDS and GDS Snapshot.

Chapter 3
Notes on Design

Explains notes when designing the system where GDS and GDS Snapshot are used.

Chapter 4
Installation

Explains how to install GDS and GDS Snapshot.

Chapter 5
GDS Management View

Explains how to start and exit GDS Management View and its screen configuration and functions.

Chapter 6
Settings

Explains how to set up GDS and GDS Snapshot.

Chapter 7
Operation and Maintenance

Explains how to operate and maintain GDS and GDS Snapshot.

Chapter 8
Configuration Change

Explains how to change the configuration of GDS and GDS Snapshot.

Chapter 9
Removing Configuration

Explains how to remove the configuration of GDS and GDS Snapshot.

Appendix A
General Notes

Explains general notes for using GDS and GDS Snapshot.

Appendix B
Command Reference

Explains the commands available in GDS and GDS Snapshot, and the PRIMECLUSTER commands used for registering and deleting the shared disk resources.

Appendix C
GDS Messages

Explains the contents, possible causes, and resolutions for GDS messages that appear when setting or operating GDS and GDS Snapshot.

Appendix D
Troubleshooting

Explains resolutions for abnormality of objects and physical disks managed by GDS.

Appendix E
Frequently Asked Questions (FAQ)

Lists frequently asked questions regarding GDS and GDS Snapshot.

Appendix F
Startup Scripts and Startup Daemons in GDS [RHEL 6 or earlier]

Explains the GDS startup script and startup daemon.

Appendix G
Systemd Services and Startup Daemons in GDS [RHEL 7]

Explains the GDS systemd service and startup daemon.

Appendix H
Release Information

Explains new features, manual update details, and changes in specifications depending on the versions of GDS and GDS Snapshot.

Glossary

Explains GDS and GDS Snapshot terminology. Please refer to it as necessary.

GDS: Global Disk Services
GDS Snapshot: Global Disk Services Snapshot


Related documentation

Please refer to the following documents according to need.

Manual Printing

Use the PDF file to print this manual.

Adobe Reader is required to read and print this PDF file. To get Adobe Reader, see Adobe Systems Incorporated's website.


Symbol

The following conventions are used in this manual:

Point

Main points are explained.

Note

Items that require attention are explained.

Information

Useful information is given.

See

Manual names and sections of this manual you should refer to are given.


Abbreviated name

Export Controls

Exportation/release of this document may require necessary procedures in accordance with the regulations of your resident country and/or US export control laws.


Trademarks

Microsoft, Windows, and Internet Explorer are registered trademarks or trademarks of Microsoft Corporation in the United States and other countries.
All other trademarks are property of their respective owners.

Linux is a registered trademark of Linus Torvalds.

Red Hat is a registered trademark of Red Hat, Inc. in the U.S. and other countries.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Dell EMC, EMC, PowerPath, SAN Manager, SRDF, Symmetrix, and TimeFinder are trademarks or registered trademarks of EMC Corporation.

VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions.

Other product names that appear in this manual are product names, trademarks, or registered trademarks of respective companies.


Notice

Date of publication and edition

November 2015, Seventh edition
December 2015, 7.1 edition
January 2016, 7.2 edition
September 2016, 7.3 edition
December 2016, 7.4 edition
May 2017, 7.5 edition
August 2017, 7.6 edition
April 2018, 7.7 edition
May 2018, 7.8 edition
August 2018, 7.9 edition
April 2019, 7.10 edition


All Rights Reserved, Copyright (C) FUJITSU LIMITED 2011-2019.

Revision History

Revision

Location

Edition

Added the descriptions of the systems and the operations suitable for the mirroring among servers.

1.2.1.4 Mirroring among Servers [4.3A40 or later] [RHEL6]

7.1

Added the descriptions to the note "Sharing the network with the cluster interconnect or the administrative LAN."

3.9.1 Network Configuration

Removed an unnecessary step of restarting the system.

7.11.3 Restoring from Backup Disks

Added the descriptions of the procedure when upgrading Dell EMC PowerPath.

8.8 Upgrading Procedure for DellEMC PowerPath

Changed the descriptions of the note "When You Cannot Unmirror the System Disk."

9.4.1.2 Operation Flow

Added the descriptions to the note "Definition of devices in the /etc/fstab file."

9.4.1.2 Operation Flow

Added the descriptions to explain that access to the physical special files are not prevented when the disk is registered to the root class.

A.1 Restraining Access to Physical Special File

Added the warning message 22016.

C.2.1 Warning Messages (22000-22099)

Added the descriptions to "Explanation" and "Resolution" of the internal error message 26000.

C.2.3 Internal Error Messages (26000)

Added the error messages 60426 to 60430.

C.4.1 Error Messages (60000-60499)

Added the warning message 62026.

C.4.2 Warning Messages (62000-62099)

Changed the resolution of "(4) System cannot be booted. (Failure of all boot disks)" for RHEL6 and RHEL7 environments.

D.1.5 System Disk Abnormality [EFI]

Added the description of how the symptom occurs to "Explanation" of "(8) System is not started after system disk is unmirrored. [RHEL6 or later]."

D.1.5 System Disk Abnormality [EFI]

Added the descriptions of the procedure when restoring the file systems on the volume.

D.1.11 File System Errors

Added "(9) Messages of systemd or systemd-udevd are output when the system is started or the operations of GDS are executed. [RHEL7]."

D.1.14 OS Messages

7.2

Specified that the shadow object is not supported.

1.6 Shadow Volume
3.8.7 Shadow Classes
7.3.4 Swapping Physical Disks Registered with Shadow Classes
B.2.2 sdxshadowdisk - Shadow Disk Operations
B.2.3 sdxshadowgroup - Shadow Group Operations
B.2.4 sdxshadowvolume - Shadow Volume Operations

7.3

Added the description for the root class to "Size of a private slice."

2.3.9 Disk Size

Added the description of the mirroring among servers.
Added the notes about the number of mpath devices.

3.2 DM-MP (Device Mapper Multipath)

Added the note when mirroring the system disk in RHEL6 environment.

3.5 System Disk Mirroring Preconditions

Specified that the hot spare function is not supported in the root class.

3.8.1 Number of Disks within Class
6.5.3.1.1 Operation Flow
6.5.3.1.2 Operation Procedure

Changed the formula to estimate the indication of the required time.

3.8.11 Required Time for Synchronization Copying Invoked by Hot Spare

Changed the description for using IP aliasing.

3.9.1 Network Configuration

Changed the description of "Setting shutdown facility."

3.9.4 Configuration and Settings of the Cluster System

Added the description of the mirroring among servers.

4.4 Setting Up DM-MP

Added " Inhibiting the unnecessary message output of multipathd."

4.4 Setting Up DM-MP

Added the notes when mirroring a partition.

4.6 Disk Setting for Performing Mirroring among Servers [4.3A40 or later] [RHEL6]

Added the procedure to check that the sg3_utils package is installed in RHEL6 environment.

4.6.2 Checking and Setting Required Packages

Added the description in the restart procedure of the SCSI target daemon (tgtd) in RHEL6 environment.

4.6.3 Creating iSCSI Target

Changed the procedure to specify the parameter of iSCSI session.

4.6.4 Establishing iSCSI Session

Changed the setting procedure in "Establishing iSCSI Session."

4.6.4 Establishing iSCSI Session

Added the description about the virtual disk on KVM guest.

4.8 Configuring KVM Environment [4.3A10 or later]

Added RDM in the virtual compatible mode to the explanation for the virtual disk.

4.9 Configuring VMware Environment

Changed the description about the description of the Excluded Device List in mirroring per partition.

6.1.4 To Use Mirroring among Servers [4.3A40 or later] [RHEL6]

Added the patch number for RHEL6(x86) environment.

6.2.1 Settings and Applying Patches [4.3A30]

Added the note "Backing up system disk."

6.5 System Disk Mirroring Settings [EFI]

Added the procedure for a rescanning of iSCSI session.

7.3.3 Swapping Disks of Netmirror Group

Changed the procedure to check the possibility of device name change.

7.3.5.5 Swapping Internal Disks Registered with Root Classes or Local Classes [RHEL6 or later]

Added the note when the system disk mirroring is configured again to "Note."

7.4 Backing Up and Restoring System Disk [EFI]

Added the description for RHEL6 or later environment in step b3).

7.4.1 Backing Up

Added the description to delete the iSCSI device from the resource database before taking the procedure.

9.6 Deleting iSCSI Settings Used for Mirroring among Servers

Changed EXAMPLES.

B.3.2 cldelrsc(8)-Delete resource

Added error messages 60433 to 60438.

C.4.1 Error Messages (60000-60499)

Changed the corrective action for disk failure in "(1) Disk is in DISABLE status."

D.1.2 Disk Status Abnormality

Changed the procedure to extract the initrd file in Resolution of "(4) System cannot be booted. (Failure of all boot disks)."

D.1.5 System Disk Abnormality [EFI]

Added the description when the cluster application becomes Inconsistent in "(2) The cluster application becomes Faulted or Inconsistent after the node is restarted."

D.1.6 Errors in the Mirroring Among Servers

Added "(7) The message regarding an unmount is output when both nodes stop."

D.1.6 Errors in the Mirroring Among Servers

Added "(6) The error message is displayed when starting GDS Management View."

D.1.7 GDS Management View Abnormality

Added "(10) Messages of multipathd are output when creating volumes."

D.1.14 OS Messages

Added "(11) The message about SELinux is output."

D.1.14 OS Messages

Added a description to "Information."

7.15.6 Restoration to 2-node Operation after Operation on Only One Node

7.4

Changed the description of restoration procedure.

7.15.7 Restoration when the Latest Disk Cannot Be Selected Automatically

Added the note to "-e force (when using -V)."

B.1.9 sdxfix - Restore Failed Object

Specified that internal disk is a target.

B.1.14 sdxdevinfo - Display Device Information of Internal Disk

Added error messages 42067 to 42070.

C.3.2 Error Messages (42000-42099)

Changed the description of warning message 44041.

C.3.3 Warning Messages (44000-44099)

Added error messages 60439 and 60440.

C.4.1 Error Messages (60000-60499)

Changed the resolution of "(5) The resource of the class where the netmirror volume exists becomes OFF-FAIL."

D.1.6 Errors in the Mirroring Among Servers

Changed the description of "(11) The message about SELinux is output."

D.1.14 OS Messages

Added "2.3.10 Group Size."

2.3.10 Group Size

7.5

Added the note "When using GDS on both host and guest in the KVM environment."

3.21 Usage in the Virtual Environment

Added a note.

4.6.2 Checking and Setting Required Packages

Added the information "Disk Size."

6.6.1.6 Volume Configuration

Added a description of file system expansion.

7.14.2 Volume Expansion Using Commands [EFI]

Added a description of restarting both nodes.

7.15.2 Stopping or Restarting the Node

Added a description about the status after both nodes are started.

7.15.4 Lock Volume of the Netmirror Volume

Added a condition of changing volume name to the note "Preconditions for Changing Volume Attributes."

8.2.1.4 Volume Configuration

Changed the note "Changes in Special File Path Name by Changing Volume Name."

8.2.1.4 Volume Configuration

Added the note when upgrading scsi-target-utils package.

8.9 Upgrading scsi-target-utils Package [4.3A40 or later] [RHEL6]

Added a description of the effect when restoration procedure 1) is not executed.

9.5 OS Re-installation

Added a description to the "Resolution" of warning message 22010.

C.2.1 Warning Messages (22000-22099)

Changed the explanation and resolution of error message 60037.

C.4.1 Error Messages (60000-60499)

Changed the explanation and resolution of warning message 5123.

C.5.2 Warning Messages (5000,5100-22099)

Changed the "Explanation" of "(1) Messages are output at node startup."

D.1.14 OS Messages

Changed the description of collecting investigation material.

D.2 Collecting Investigation Material

Deleted the unnecessary description designated for the KVM guest.

4.6.3 Creating iSCSI Target

7.6

Added "4.8.4 Setting up virtio Block Device Used for Mirroring among Servers [4.3A40 or later] [RHEL6]."

4.8.4 Setting up virtio Block Device Used for Mirroring among Servers [4.3A40 or later] [RHEL6]

Changed the description designated for the KVM guest.

7.3.3 Swapping Disks of Netmirror Group

Add a new word "by-id file".

Glossary

Changed the note about Symfoware.

1.2.1.4 Mirroring among Servers [4.3A40 or later] [RHEL6]

7.7

Added the disk size condition to "Conditions for system disk configuration and settings."

3.5 System Disk Mirroring Preconditions

Added the note for iSCSI boot environment to "Note."

3.5 System Disk Mirroring Preconditions

Added "Capacity Expansion of Disk Registered in the Class."

3.12 Online Volume Expansion

Added the reference destinations of notes to "See."

3.22 Disaster Recovery
8.6 Server Migration
8.7.2 Using Copy Function of Storage Units and sdxconfig Command

Change the procedures.

3.22.1 Building Disaster Recovery System
3.22.3 Switching to Disaster Recovery System
3.22.4 Restoration to Operation System from Disaster Recovery System

Added the explanation of procedure for data synchronization of operation system and disaster recovery system.

3.22.2 Data Synchronization of Operation System and Disaster Recovery System

Added the note for re-installing OS.

4.1 Notes on Installing OS

Changed the explanation of host name.

5.1.1 Starting Web-Based Admin View Operation Menu

Changed the note "When the Web-Based Admin View does not start."

5.1.1 Starting Web-Based Admin View Operation Menu

Added the note to "Update Now."

5.3.4 [View] Menu

Changed the explanation in the information "exdevtab.sh."

6.1.3 To Use Dell EMC Storage Units

Added the note "For RHEL7.4 or later environment."

6.5.3.1.2 Operation Procedure

Added the explanation of automatically unmounting when stopping OS to the information "For Local Classes."

6.7.3 File System Auto Mount

Changed how to change the access permission.

6.9 Volume's Block Special File Access Permission

Added the explanation of necessary settings before and after backing up and restoring system disk.

7.4 Backing Up and Restoring System Disk

Added "7.4.1 Settings Before Backing Up."

7.4.1 Settings Before Backing Up

Added the setting procedure of before and after backing up.

7.4.2 Backing Up [EFI]

Added "7.4.3 Settings After Backing Up."

7.4.3 Settings After Backing Up

Added "7.4.4 Settings Before Restoring."

7.4.4 Settings Before Restoring [4.3A40 or later] [RHEL 6]

Added the setting procedure after restoring.

7.4.5 Restoring (When System Can Be Booted) [EFI]
7.4.6 Restoring (When System Cannot Be Booted) [EFI]

Added "7.4.7 Settings After Restoring."

7.4.7 Settings After Restoring

Added "See."

7.13 Backing Up and Restoring Object Configurations

Added the procedure necessary for the cluster system.

7.13.2 Restoring

Added "Note."

7.15.1 I/O Error Due to Node Stop or Network Error

Added the note "Recreating shared class registered in a cluster application."

8.2 Changing Object Configuration

Changed the note "Changes in Special File Path Name by Changing Volume Name."

8.2.1.4 Volume Configuration

Changed the message of Restoration procedure 2).

9.5 OS Re-installation

Added the explanation necessary for restoring the object configuration of shared class registered in the cluster application.

A.5 Backing Up and Restoring Object Configuration (sdxconfig)

Changed the explanation of scope in SUB OPTIONS.

B.1.2 sdxdisk - sdxdisk - Disk Operations
B.1.7 sdxattr - sdxattr - Change Objects Attributes

Added the note about multiple and continuous execution to DESCRIPTION.

B.1.6 sdxinfo - Display Object Configuration and Status Information

Added the note about by-id file to DESCRIPTION.

B.1.6 sdxinfo - Display Object Configuration and Status Information

Added Note to DESCRIPTION.

B.1.11 sdxroot - Root File System Mirroring Definition and Cancellation [EFI]

Added the explanation for the cluster system to the descriptions of PRIMARY OPTIONS -S.

B.1.12 sdxparam - Configuration Parameter Operations

Added the conditions necessary to restart the node to the descriptions of PRIMARY OPTIONS Restore.

B.1.13 sdxconfig - Object Configuration Operations

Added the internal error message 26001.

C.2.3 Internal Error Messages (26000-26001)

Added g) to the Explanation and Resolution of the internal error message 48000.

C.3.5 Internal Error Messages (48000)

Added the descriptions to Resolution in the error message 60126.

C.4.1 Error Messages (60000-60499)

Added the note "Shutting down the node" to Resolution in "(2) Class cannot be started when booting the system."

D.1.4 Class Status Abnormality

Changed the procedure of Resolution in "(1) The netmirror slice that configures the netmirror volume is in INVALID state."

D.1.6 Errors in the Mirroring Among Servers

Changed the procedure of Resolution in "(4) The slice becomes INVALID or the operation stops after restarting the other node when the network error occurs."

D.1.6 Errors in the Mirroring Among Servers

Changed the starting method of cluster application of Resolution in "(6) The disk space of a file system on a shared disk is "Full" (100%)."

D.1.10 Cluster System Related Error

Changed Explanation in "(4) A devlabel message is output at node startup or during collection of investigation material."

D.1.14 OS Messages

Added "(12) Error messages of modprobe and systemd are output when the system is started and GDS is not started. [RHEL7]."

D.1.14 OS Messages

Changed the explanation of sdxsnap.sh.

D.2 Collecting Investigation Material
D.2.2 Collecting Investigation Material with sdxsnap.sh Script

Added the note about by-id file.

D.2 Collecting Investigation Material

Added "Appendix F Startup Scripts and Startup Daemons in GDS [RHEL6 or earlier]."

Appendix F Startup Scripts and Startup Daemons in GDS [RHEL6 or earlier]

Added "Appendix G Systemd Services and Startup Daemons in GDS [RHEL7]."

Appendix G Systemd Services and Startup Daemons in GDS [RHEL7]

Added the description for the case of a class where a netmirror volume is created to "Size of a private slice."

2.3.9 Disk Size

7.8

Described the necessity of the settings before registering the disk to the resource database.

4.11 Settings to Use Dell EMC PowerPath [4.3A10 or later]

Added the Note to "Update Physical Disk Information."

5.3.3 [Operation] Menu

Deleted the Note from "Update Now."

5.3.4 [View] Menu

Added the Note.

6.4 Updating Physical Disk Information

Added the procedure necessary for starting in single user mode to "4) Parting the proxy groups."

7.5.6 Backing Up

Added the procedure necessary after parting the proxy groups in single user mode to "6) Verifying the alternative boot environment."

7.5.6 Backing Up

Added the procedure necessary after parting the proxy groups in single user mode to "8) Switching to the alternative boot environment."

7.5.7 Restoring

Described the conditions that the entire slice becomes the copy target.

7.15.1 I/O Error Due to Node Stop or Network Error

Added ERROR to "Explanation" table.

C.2 Driver Messages

Added the error message 21000.

C.2.1 Error Messages (21000-21099)

Added "(13) The device name of GDS volume is displayed in the sfdskX format, therefore the message for which volume is unknown."

D.1.14 OS Messages

Added driver to the "Variable Names" list.

C.2 Driver Messages

7.9

Added the error message 21001.

C.2.1 Error Messages (21000-21099)

Reflected the procedure for the patch for PH15096 is applied in the "Resolution" in "(4) System cannot be booted. (Failure of all boot disks)."

D.1.5 System Disk Abnormality [EFI]

Changed the description of devices that cannot be used for the mirroring among servers.

3.9.2 Disk Area

7.10

Changed the procedure for swapping a disk connected to a netmirror group.

7.3.3 Swapping Disks of Netmirror Group

Added the error message 60451.

C.4.1 Error Messages (60000-60499)