Top
PRIMECLUSTER  Installation and Administration Guide 4.3

Preface

This manual serves as your starting point for using PRIMECLUSTER. It explains the workflow of the series of operations from installation to operation management of the PRIMECLUSTER system. Since the PRIMECLUSTER system comprises multiple features, there are several other manuals besides this one for each feature. However, by reading this manual first, you will be able to perform the series of operations because this manual refers readers to other manuals that contain feature-specific information that is necessary for the operations.

This manual also provides a functional overview of products that are supported by the PRIMECLUSTER system and describes operation procedures.

This manual only covers the basic operation of PRIMECLUSTER. For operations using different hardware and software configurations, see "Related Documentation."

The table below shows the operation flow from PRIMECLUSTER installation to the start of operation management and indicates the reference location in this manual for each operation.

Flow from PRIMECLUSTER system installation to operation management

PRIMECLUSTER system operation flow

Reference location in this manual

1. Understanding the flow of PRIMECLUSTER system building and designing the PRIMECLUSTER system

Part 1 Planning

2. Installing the PRIMECLUSTER system

Part 2 Installation

3. Monitoring the operation status of the PRIMECLUSTER system

Part 3 Operations

4. Changing the PRIMECLUSTER system configuration after system operation has been started

Part 4 System Configuration Modifications

5. Maintaining the PRIMECLUSTER system

Part 5 Maintenance

6. Installing products for the PRIMECLUSTER system

Part 6 PRIMECLUSTER Products

For detailed procedural explanations, refer to the reference manuals that are indicated in the target location of each part.

Target Readers

This manual is intended for all users who use PRIMECLUSTER 4.3 and perform cluster system installation and operation management. It is also intended for programmers who develop applications that operate on PRIMECLUSTER.

Configuration of This Documentation

This manual consists of six parts, appendixes, and a glossary. The contents of each part are described below.

Part 1 Planning

Audience: System administrators who build PRIMECLUSTER systems
Contents: This part describes the overall workflow from installation to operation of the PRIMECLUSTER system.

Part 2 Installation

Audience: System administrators who build PRIMECLUSTER systems
Contents: This part describes operations for software installation, cluster building, and application building.

Part 3 Operations

Audience: System administrators who manage system operations
Contents: This part describes operation methods for operations such as monitoring the PRIMECLUSTER system and investigating failure causes.

Part 4 System Configuration Modifications

Audience: System administrators who build PRIMECLUSTER systems
Contents: This part describes necessary work items for additions, modifications, and deletions to the PRIMECLUSTER system configuration.

Part 5 Maintenance

Audience: System administrators who build PRIMECLUSTER systems
Contents: This part describes the items related to the maintenance of the PRIMECLUSTER system.

Part 6 PRIMECLUSTER Products

Audience: Users who operate PRIMECLUSTER products on PRIMECLUSTER systems
Contents: This part describes the versions and levels of products that are supported by the PRIMECLUSTER system and provides a functional overview of those products.

Appendix A PRIMECLUSTER System Design Worksheets

Audience: Users who design PRIMECLUSTER systems
Contents: This appendix contains the PRIMECLUSTER System Design Worksheet.

Appendix B Manual Pages

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix describes the online manual pages that are used by the individual features of the PRIMECLUSTER system.

Appendix C Troubleshooting

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix describes corrective actions for problems that may occur in the PRIMECLUSTER system. It also explains how to collect data when requesting a problem investigation.

Appendix D Messages

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix describes the contents and actions for messages output by the userApplication Configuration Wizard GUI and messages output by procedure resources in PRIMECLUSTER systems. For other messages, this manual indicates the reference locations in other manuals.

Appendix E Registering, Changing, and Deleting State Transition Procedure Resources for PRIMECLUSTER Compatibility

Audience: All users who use PRIMECLUSTER-compatible resources
Contents: This appendix describes procedures for registering, changing, and deleting procedure resources when the cluster applications use procedure resources.

Appendix F Configuration Update Service for SA

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix descries Configuration Update Service for SA.

Appendix G Using PRIMECLUSTER in a KVM environment

Audience: All users who use PRIMECLUSTER systems in a KVM environment

Contents: This appendix describes the installation procedures for using the PRIMECLUSTER system in a KVM environment.

Appendix H Using PRIMECLUSTER in a VMware Environment

Audience: All users who use PRIMECLUSTER systems in a VMware environment
Contents: This appendix describes the installation procedures for using the PRIMECLUSTER system in a VMware environment.

Appendix I Startup Scripts and Startup Daemons, and Port Numbers in PRIMECLUSTER

Audience: System administrators who build PRIMECLUSTER systems
Contents: This appendix provides explanations on scripts and daemons that are started by PRIMECLUSTER, and the port numbers being used.

Appendix J Using Firewall

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix describes the procedure when using Firewall in the PRIMECLUSTER system.

Appendix K Changes in Each Version

Audience: All users who use PRIMECLUSTER 4.0A20, 4.1A20, 4.1A30, 4.2A00, 4.2A30, 4.3A00, or 4.3A10
Contents: This appendix describes the changes made to the specifications of PRIMECLUSTER 4.3A20.

Appendix L Release Information

Audience: All users who use PRIMECLUSTER systems
Contents: This appendix describes the main changes of this manual.

Glossary

Audience: All users who use PRIMECLUSTER systems
Contents: This section explains terms used to describe the PRIMECLUSTER system.

Related Documentation

Refer to the following manuals as necessary when setting up the cluster:

Note

The PRIMECLUSTER documentation includes the following documentation in addition to those listed above:

  • PRIMECLUSTER Software Release Guide and Installation Guide

    This Software Release Guide and Installation Guide are provided with each PRIMECLUSTER product package.

    The data is stored on "CD" of each package. For details on the file names, see the documentation.

Manual Series

Manual Printing

If you want to print a manual, use the PDF file found on the CD-ROM for the PRIMECLUSTER product. The correspondences between the PDF file names and manuals are described in the Software Release Guide for PRIMECLUSTER that comes with the product.

Adobe Reader is required to read and print this PDF file. To get Adobe Reader, see Adobe Systems Incorporated's website.

Online Manuals

To allow users to view the online manuals, use the Cluster management server to register each user name to one of the user groups (wvroot, clroot, cladmin, or clmon).

For information on user group registration procedures and user group definitions, see "4.3.1 Assigning Users to Manage the Cluster".

Conventions

Notation
Prompts

Command line examples that require system administrator (or root) rights to execute are preceded by the system administrator prompt, the hash sign (#). Entries that do not require system administrator rights are preceded by a dollar sign ($).

Manual page section numbers

References to the Linux(R) operating system commands are followed by their manual page section numbers in parentheses - for example, cp(1)

The keyboard

Keystrokes that represent nonprintable characters are displayed as key icons such as [Enter] or [F1]. For example, [Enter] means press the key labeled Enter; [Ctrl-b] means hold down the key labeled Ctrl or Control and then press the [B] key.

Typefaces

The following typefaces highlight specific elements in this manual.

Typeface

Usage

Constant Width

Computer output and program listings; commands, file names, manual page names and other literal programming elements in the main body of text.

Italic

Variables that you must replace with an actual value.

Bold

Items in a command line that you must type exactly as shown.

Example 1

Several entries from an /etc/passwd file are shown below:

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/bin/bash

daemon:x:2:2:daemon:/sbin:/bin/bash

lp:x:4:7:lp daemon:/var/spool/lpd:/bin/bash

Example 2

To use the cat(1) command to display the contents of a file, enter the following command line:

$ cat file

Notation symbols

Material of particular interest is preceded by the following symbols in this manual:

Point

Contains important information about the subject at hand.

Note

Describes an item to be noted.

Example

Describes operation using an example.

Information

Describes reference information.

See

Provides the names of manuals to be referenced.

Abbreviations

Export Controls

Exportation/release of this document may require necessary procedures in accordance with the regulations of your resident country and/or US export control laws.

Trademarks

Red Hat is a registered trademark of Red Hat, Inc. in the U.S. and other countries.

Linux is a registered trademark of Linus Torvalds.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Microsoft, Windows, and Internet Explorer are registered trademarks of Microsoft Corporation in the United States and other countries.

CORBA, OMG, and ORB are registered trademarks of Object Management Group (OMG).

NetWorker is a registered trademark or trademark of EMC Corporation in the United States and other countries.

VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions.

PRIMECLUSTER is a registered trademark of Fujitsu Limited.

Other product names are product names, trademarks, or registered trademarks of these companies.

Requests

Date of publication and edition

April 2008, First edition
October 2008, Second edition
June 2009, Third edition
August 2009, Fourth edition
January 2010, Fifth edition
June 2011, Sixth edition
July 2011, 6.1 edition
November 2011, 6.2 edition
April 2012, 6.3 edition
December 2012, Seventh edition
June 2013, 7.1 edition
August 2013, 7.2 edition
January 2014, 7.3 edition
June 2014, 7.4 edition
October 2015, 7.5 edition

Copyright notice

All Rights Reserved, Copyright (C) FUJITSU LIMITED 2008-2015.

Revision History

Revision

Location

Edition

Added the notes on the single-node cluster operation.

2.3.3 Single-Node Cluster Operation
4.3.3.1 Initial setup of the operation management server
Chapter 5 Building a Cluster
5.1.1 Setting Up CF and CIP
6.7.2.1 Creating Standby Cluster Applications
6.7.3.4 Setting Up Gds Resources

7.1 edition

Added the descriptions in the case of taking time for an I/O error to reach the ext3 file system.

3.1.3 Setting Up the Disk Units
3.2.1.1 Host OS setup (before installing the operating system on guest OS)
3.2.2.1 Host OS setup (before installing the operating system on guest OS)
3.2.3.1.3 Host OS setup (before installing the operating system on guest OS)

Added the note in the case of creating a redundant administrative LAN used in the shutdown facility by using GLS.

5.1.2 Setting Up the Shutdown Facility

Changed the setting of "weight" of "Node information" found on the Cluster Configuration Worksheet in System Design Examples.

A.11.1.1 Cluster Configuration Worksheet
A.11.2.1 Cluster Configuration Worksheet
A.12.1.1 Cluster Configuration Worksheet
A.12.2.1 Cluster Configuration Worksheet
A.13.1.1 Cluster Configuration Worksheet
A.13.2.1 Cluster Configuration Worksheet
G.7.1.1 Cluster Configuration Worksheet
G.7.2.1 Cluster Configuration Worksheet

Changed the note that try not to target a virtual machine that installs PRIMECLUSTER as cluster functions of VMware.

G.1 Cluster Systems in a VMware Environment

Changed the description on the installation of VMware vSphere.

G.3.1.1 Installation and Configuration of Related Software

Added the section "7.5 Notes on Operation."

Chapter 7 Operations

7.2 edition

Added the note when using Configuration Update Service for SA.

F.2 Operation Environment

Added the description about the setting of kdump.

3.3 PRIMECLUSTER Installation

7.3 edition

Added the note in the case of creating a redundant administrative LAN used in the shutdown facility by using GLS.

5.1.2 Setting Up the Shutdown Facility

Added RX500 series for the supported hardware device.

5.1.2 Setting Up the Shutdown Facility
5.1.2.3.4 kdump

Described the procedure to encrypt the password.

5.1.2.3.2 IPMI

Added the notes on characters allowed in passwords.

5.1.2.4.1 MMB

Added the notes when nodes were forcibly stopped by the SA_mmbr shutdown agent.

5.1.2.4.1 MMB

Changed the procedure to start the shutdown facility.

5.1.2.5.1 vmSP (Virtual Machine Service Provider)
5.1.2.5.2 libvirt
5.1.2.5.3 vmchkhost

Described the information about the maximum concurrent connections for SSH.

5.1.2.5.1 vmSP (Virtual Machine Service Provider)
5.1.2.5.2 libvirt
5.1.2.5.3 vmchkhost

Added the note on the hvw command.

6.7.2 Setting Up userApplication

Added the notes on the virtual machine function.

2.2.1 Virtual Machine Function

7.4 edition

Described the notes on adding virtual disks.

3.2.1.1 Host OS setup (before installing the operating system on guest OS)
3.2.1.2 Host OS setup (after installing the operating system on guest OS)
3.2.3.1.3 Host OS setup (before installing the operating system on guest OS)
3.2.3.1.4 Host OS setup (after installing the operating system on guest OS)

Added the descriptions that explain mirror volumes in the shared class created in the host OS can be used when mirroring the system disk in the guest OS.

3.2.1.1 Host OS setup (before installing the operating system on guest OS)
3.2.3.1.3 Host OS setup (before installing the operating system on guest OS)
6.3.1 Setting Up System Disk Mirroring

Modified the descriptions of the CF node name.

5.1.1 Setting Up CF and CIP

Added the note when the device name of the shared disk device varies depending on a node.

5.1.3.2 Registering Hardware Devices

Added the explanation of how to create a shared disk definition file to register a shared disk.

5.1.3.2 Registering Hardware Devices

Added the notes on using EC or REC function of ETERNUS disk array.

6.3 GDS Configuration Setup

Changed the notes on configuration on the virtual machine function.

6.3.2 Setting Up Shared Disks

Changed the operation procedure to set up takeover network resources.

6.7.3.6 Setting Up Takeover Network Resources

Modified the display example of the hvw command.

6.7.2.1 Creating Standby Cluster Applications
6.7.2.2 Creating Scalable Cluster Applications
6.7.3 Setting Up Resources
6.7.3.1 Setting Up Cmdline Resources
6.7.3.2 Setting Up Fsystem Resources
6.7.4 Generate and Activate
6.10.1 Setting Contents of a Cluster Application
8.1 Changing the Cluster Configuration
8.5.1 Changing Operation Attributes (CUI)
8.7.1 Deleting a Resource
8.7.3 Deleting a userApplication

Added the notes on Configuration-Activate.

6.7.4 Generate and Activate

Added the reference to set up Online/Offline scripts.

6.6 Setting Up Online/Offline Scripts

Deleted the reference.

6.7.2.1 Creating Standby Cluster Applications

Changed the explanation of maintenance mode.

7.2.2.6 Entering maintenance mode for Cluster Application

Changed the explanation of changing PRIMECLUSTER Operation Attributes.

8.6 Changing PRIMECLUSTER Operation Attributes

Changed the operation procedure of deleting a resource.

8.7.1 Deleting a Resource

Changed how to add the shared disk device.

8.12.1.1 Adding a shared disk device

Changed the descriptions of how to change the hardware.

8.12.3.1 Changing the shared disk device
8.12.3.2 Changing a network interface card used for the public LAN and the administrative LAN

Changed the notes on operation on the virtual machine function.

A.12.1 When building a cluster system between guest OSes on one host OS
A.12.2 When building a cluster system between guest OSes on multiple host OSes
A.13.1 When building a cluster system between guest OSes on one host OS
A.13.2 When building a cluster system between guest OSes on multiple host OSes

Changed the table of the search procedure.

D.1 Searching for a Message

Changed the description of the operation environment for Configuration Update Service.

F.2 Operation Environment

Added the appendix.

Appendix G Using PRIMECLUSTER in a KVM Environment

Changed the notes on the cluster system in a VMware environment.

H.1 Cluster Systems in a VMware Environment

Changed the descriptions of how to configure the /etc/hostid file to run I/O fencing properly.

H.2.1.1 Installation and Configuration of Related Software

Changed the descriptions of how to set the shutdown facility.

H.2.3.2 Setting Up the Shutdown Facility

Changed the description of the utilized port of S13SMAWsf.

I.2 Startup Script Lists

Changed the descriptions of information on incompatibility.

Appendix K Changes in Each Version

Added and changed the notes on Xen, KVM, and VMware environments.

2.2.1 Virtual Machine Function

7.5 edition

Added the description about the settings when performing immediate cluster failover if an I/O device where the system volume is allocated fails in the ext3 or the ext4 file system.

3.1.3 Setting Up the Disk Units
3.2.1.1 Host OS setup (before installing the operating system on guest OS)
3.2.2.1 Host OS setup (before installing the operating system on guest OS)
3.2.3.1.3 Host OS setup (before installing the operating
H.2.1.1 Installation and Configuration of Related Software
H.3.1.1 Installation and Configuration of Related Software

Added the remark to "Prerequisites for the IPMI shutdown agent settings."

3.1.6.1 PRIMERGY

Deleted the note on displaying the Web-Based Admin View GUI screen.

4.3.2 Preparing the Client Environment

Added the note on the initial setup of Web-Based Admin View.

4.3.3 Initial Setup of Web-Based Admin View

Added the note on the calculation for the survival priority.

5.1.2.2 Survival priority

Changed the description when displaying a message to a console and also how to start the console.

5.2 Setting Up Fault Resource Identification and Operator Intervention Request

Added the description of the environment variable that can be referred from the Check script.

6.6 Setting Up Online/Offline Scripts

Added the note on setting up Cmdline resources.

6.7.3.1 Setting Up Cmdline Resources

Changed the setting procedure of an Fsystem resource.

6.7.3.2 Setting Up Fsystem Resources

Changed the contents of setting file definition for the cluster.

6.7.3.6 Setting Up Takeover Network Resources

Added the notes on exclusive relationships between cluster applications.

6.7.7 Exclusive Relationships Between Cluster Applications

Changed the description of notes on operation.

7.5 Notes on Operation

Changed the procedure for adding shared disk devices.

8.12.1.1 Adding a shared disk device
8.12.2.1 Deleting a shared disk device
8.12.3.1 Changing a shared disk device

Changed the worksheets.

A.8 Cluster Application Configuration Worksheets
A.10 Cluster Application Worksheets
H.7.1.2 GLS Setup Worksheet
H.7.2.2 GLS Setup Worksheet

Added the note on changing the registration information of a procedure resource.

E.2.3 Changing registration information of a procedure resource

Added and changed the node on the cluster system in a VMware environment.

H.1 Cluster Systems in a VMware Environment

Changed the setting contents for installing VMware vSphere.

H.2.1.1 Installation and Configuration of Related Software
H.3.1.1 Installation and Configuration of Related Software

Added S06clonltrc to /etc/rc3.d.

I.2 Startup Script Lists

Added the service names, to be supported to port numbers, to Remarks.

I.2 Startup Script Lists

Changed the description when changing the configuration of iptables described in Note.

Appendix J Using Firewall