This section explains how to register resources to the userApplication that was set up in the previous section.
You can register the following resources:
Cmdline resources
You can use Cmdline resources to set up script files or commands as resources. The Cmdline resources are required to generate the state transition of userApplication along with the stop of user applications, and conversely, to start or stop ISV applications or user applications along with the state transition of the userApplication.
Fsystem resources
Used when you mount a file system along with userApplication startup.
Note
To use a file system in a class created by GDS as an Fsystem resource, you must register the Gds resource to the same userApplication.
Gds resources
Used when you start and stop a disk class to be defined by GDS by linking it with the userApplication.
Gls resources
Used when you set up a takeover IP address that is to be defined in a userApplication with the redundant line control function of GLS, or when you set a takeover IP address in a userApplication with the single line control function.
Takeover network resource
Used when you set up a takeover IP address on a single network interface without using GLS.
Procedure resources
Used when you register a state transition procedure in the userApplication.
Resource setup flow
This section describes the procedure for creating Cmdline resources.
For notes when setting Cmdline resources, see "6.11 Notes When Setting Cmdline Resources."
Operation Procedure:
Select "CommandLines" from "turnkey wizard "STANDBY"".
"CommandLines" will appear. Select "AdditionalStartCommand."
Select "FREECHOICE" and then enter the full path of the StartCommand. If you need to specify arguments, delimit them with blanks.
StartCommand is executed during Online processing to start user applications.
[StartCommand exit codes]
StartCommand has the following exit codes:
0: Normal exit. The Online processing is successfully done.
Other than 0: Abnormal exit. The Online processing fails. When the script exits with the cord other than 0, the resource will enter Faulted.
Note
The following characters cannot be used in the script path and the arguments that set for StartCommand, and StopCommand and CheckCommand to be described later.
= \ ~ % @ &
If you need to use those characters, describe them within the script that sets to Cmdline resources.
"CommandLines" will appear. If you need to stop the user programs, select "StopCommands."
StopCommand is executed during Offline processing to stop user applications.
You do not always have to set up the StopCommand.
[StopCommand exit codes]
StopCommand has the following exit codes:
0: Normal exit. The Offline processing is successfully done.
Other than 0: Abnormal exit. The Offline processing fails. When the script exits with the cord other than 0, the resource will enter Faulted.
If you do not use StopCommand, start from step 6.
Note
If "none" is set to StopCommands, regardless of the settings of Flags, LIEOFFLINE attribute is enabled and CLUSTEREXCLUSIVE is disabled. In this status, the Cmdline resource is started and monitored.
Select "FREECHOICE" and then enter the full path of StopCommand. If you need to specify arguments, delimit them with blanks.
"CommandLines" will appear. Select "CheckCommands."
CheckCommand is executed periodically to notify RMS of the state of the user applications.
0: Online.
Other than 0: Offline.
If you enable the ALLEXITCODES attribute, the script that is defined in CheckCommand provides more detailed state of the user applications. Change the attribute in step 8.
Select "FREECHOICE" and then enter the full path of the CheckCommand. If you need to specify arguments, delimit them with blanks.
Change the attribute.
Change the attribute to suit the purpose. To change the attribute, select "Flags[0]." For details on the attribute, see "Table 6.2 Attributes of the Cmdline resource" in "6.11 Notes When Setting Cmdline Resources."
Note
If you enable the "NULLDETECTOR" attribute, CheckCommand is not started from RMS. For hot-standby operation, enable the following two attributes;
STANDBYCAPABLE
RMS executes Standby processing of the resources on all the nodes where the userApplication is Offline.
ALLEXITCODES
Check script provides the detailed state of the resource with the exit code.
For further details about the hot-standby operation settings, see "6.6 Setting Up Online/Offline Scripts."
Finally, select "SAVE+EXIT."
An Fsystem resource must be set up if you want to mount a file system when userApplication is started.
For notes when setting Fsystem resources, see "6.12 Notes When Setting Fsystem Resource."
You need to set up the Fsystem resource according to the flow below:
Note
The file system on the volume of LVM (Logical Volume Manager) cannot be controlled in Fsystem resource.
Defining mount points
Define mount points for file systems that are used as resources in the /etc/fstab.pcl file on all the nodes where userApplication is configured. Each line must begin with "#RMS#."
The description format of the first field to the sixth field in a line beginning with "#RMS#" is the same as that of the /etc/fstab file of RHEL. For details, refer to the official documentation of RHEL.
Example: /etc/fstab.pcl file
#RMS#/dev/sdd1 /mnt/swdsk1 ext3 defaults 0 0 #RMS#/dev/sdd2 /mnt/swdsk2 ext4 nodelalloc 0 0 #RMS#/dev/sdd3 /mnt/swdsk3 xfs defaults 0 0
If you plan to use GDS volumes, you need to define the /etc/fstab.pcl file as follows.
Example: /etc/fstab.pcl file
#RMS#/dev/sfdsk/class0001/dsk/volume0001 /mnt/swdsk1 ext3 defaults 0 0 #RMS#/dev/sfdsk/class0001/dsk/volume0002 /mnt/swdsk2 ext4 nodelalloc 0 0 #RMS#/dev/sfdsk/class0001/dsk/volume0003 /mnt/swdsk3 xfs defaults 0 0
Note
If you have defined the same device or mount point in the /etc/fstab file, those definitions can be removed by making them into comment lines. If those definitions are remained, userApplications may fail to be started normally.
Ext4 and xfs are used to make the allocation of the disk area more efficiently, and to improve the writing performance, by using their "Delayed Allocation" feature. As a result of the implementation of "Delayed Allocation", there is a possibility that a part of data is lost by OS panic or power supply interruption of servers, because the sojourn time on the memory of data that should be stored on the disk becomes longer.
When a program has to guarantee writing immediately after writing in file system, the application which writes the file should issue the fsync() call. Refer to Storage Administration Guide of the Red Hat, Inc. for "Delayed allocation."
For the directory paths that are specified as the mount points, specify any paths that do not include symbolic links.
Starting the GDS volume (only if necessary)
If a file system or a file to be maintained exists in a volume managed by GDS, start the GDS volume in any one for nodes configuring a cluster.
Example: When starting the volume volume0001 of the disk class class with a command
# /usr/sbin/sdxvolume -N -c class -v volume0001Confirming file systems are created and can be mounted
It is necessary to create file systems on disk partitions that are used as resources. Refer to Storage Administration Guide of the Red Hat, Inc. for "Create file systems".
Example: Creating the ext3 file system
Create the file system.
# /sbin/mkfs -t ext3 /dev/sdd1Check if the file systems can be mounted.
# /bin/mount -t ext3 /dev/sdd1 /mnt/swdsk1 # /bin/umount /mnt/swdsk1
Example: Creating the ext4 file system
Create the file system.
# /sbin/mkfs.ext4 /dev/sdd2Check if the file systems can be mounted.
# /bin/mount -t ext4 /dev/sdd2 /mnt/swdsk2 # /bin/umount /mnt/swdsk2
Example: Creating the xfs file system
Create the file system.
# /sbin/mkfs.xfs /dev/sdd3Check if the file systems can be mounted.
# /bin/mount -t xfs /dev/sdd3 /mnt/swdsk3 # /bin/umount /mnt/swdsk3
Tuning file systems
Set up the cluster environment in according to their file system types.
Forcible file system check prevention (recommended for ext3 and ext4)
If ext3 or ext4 is used for a file system, the file system might forcibly be checked during Online processing of a switching file system. It is part of the ext3 and ext4 specification that file systems are checked when a certain number of mounting has been executed since the last file system check, or a certain period of time has passed.
If the file systems are forcibly checked along with startup or failover of the cluster application, timeout occurs due to file system Online processing, and PRIMECLUSTER startup or failover might fail.
It is necessary to prevent the file systems from being checked by executing the following command for all the ext3 and ext4 switching files.
Example: Configuring and confirming the prevention of file systems from being checked
# /sbin/tune2fs -c0 -i0 /dev/sdd1After executing the above command, check if "Maximum mount count :-1", "Check interval:0"is displayed using the following command:
# /sbin/tune2fs -l /dev/sdd1 | /bin/grep "Maximum mount count" Maximum mount count: -1 # /sbin/tune2fs -l /dev/sdd1 | /bin/grep "Check interval" Check interval: 0 (<none>)
If the forcible file system check is prevented, file systems might corrupt due to failures such as disk errors and kernel bug. These failures cannot be detected through file system logging and journaling. The file system corruption might cause data corruption.
To prevent this, execute the "fsck - f" command to enable the file system forcible check during periodic maintenance.
Set Delayed Allocation disabled. (Only for ext4)
For ext4 file systems, Delayed Allocation feature can be disabled by specifying nodelalloc for the mount attribute.
You need to specify mount attribute field in the /etc/fstab.pcl file as follows.
#RMS#/dev/sdd2 /mnt/swdsk2 ext4 nodelalloc 0 0
Stopping the GDS volume (Only when Step 2 has already been implemented)
Stop the GDS volume started in Step 2.
Example: Stopping the volume volume0001 of the disk class class with a command
# /usr/sbin/sdxvolume -F -c class -v volume0001Registering Fsystem resource to Cluster Applications
Select "LocalFileSystems" from "turnkey wizard "STANDBY"".
Settings of turnkey wizard "STANDBY" (APP1:not yet consistent) 1) HELP 10) Enterprise-Postgres(-) 2) - 11) Symfoware(-) 3) SAVE+EXIT 12) Procedure:SystemState3(-) 4) - 13) Procedure:SystemState2(-) 5) ApplicationName=APP1 14) Gls:Global-Link-Services(-) 6) Machines+Basics(app1) 15) IpAddresses(-) 7) CommandLines(-) 16) LocalFileSystems(-) 8) Procedure:Application(-) 17) Gds:Global-Disk-Services(-) 9) Procedure:BasicApplication(-) Choose the setting to process: 16
Select "AdditionalMountPoint."
File systems (Lfs_APP1:not yet consistent) 1) HELP 4) REMOVE+EXIT 7) (Timeout=180) 2) - 5) AdditionalMountPoint 3) SAVE+EXIT 6) (Filter=) Choose the setting to process: 5
The mount points, which are defined in the above-mentioned /etc/fstab.pcl file for preliminary setup, will appear. Select the mount point to be set in userApplication.
1) HELP 6) /mnt/swdsk2 2) RETURN 7) /mnt/swdsk3 3) FREECHOICE 4) ALL 5) /mnt/swdsk1 Choose a mount point: 5
Select "SAVE+RETURN."
Set flags for mount point: /mnt/swdsk1 Currently set: LOCAL,AUTORECOVER (LA) 1) HELP 4) DEFAULT 7) SHARE(S) 2) - 5) SYNC(Y) 8) MONITORONLY(M) 3) SAVE+RETURN 6) NOT:AUTORECOVER(A) Choose one of the flags: 3
If you register multiple mount points, repeat steps 2 to 4 for each mount point. After you have registered all necessary mount points, Select "SAVE+EXIT."
File systems (Lfs_APP1:consistent) 1) HELP 6) MountPoints[0]=LA:/mnt/swdsk1 2) - 7) MountPoints[1]=LA:/mnt/swdsk2 3) SAVE+EXIT 8) MountPoints[2]=LA:/mnt/swdsk3 4) REMOVE+EXIT 9) (Filter=) 5) AdditionalMountPoint 10) (Timeout=180) Choose the setting to process: 3
[Prerequisites]
If you need to set up a Gds resource, you must first set up a shared volume.
Also, before you make the settings required for the Gds resources, execute the following command on either node.
This command operation is required for linking the specified Gds resources with the status of the userApplication and to start and stop the GDS volume. In addition, this command sets all the nodes sharing the volume to Offline status.
# /opt/SMAW/SMAWRrms/bin/hvgdsetup -a [class-name]
... Do you want to continue with these processes ? [yes/no] y
Information
To check the setup status of a shared volume, execute the following command:
# /opt/SMAW/SMAWRrms/bin/hvgdsetup -lNote
If the preliminary setup is not performed, the cluster application is set to Inconsistent status. For details, see "Cluster applications become "Inconsistent"." in "Cluster System Related Error" of "PRIMECLUSTER Global Disk Services Configuration and Administration Guide."
This operation must not be performed when a GFS shared file system is used.
After completing the preliminary setup for the Gds resources, register the Gds resources to the userApplication. The procedure for setting Gds resources to a userApplication is described below.
Operation Procedure:
Select "Gds:Global-Disk-Services" from "turnkey wizard "STANDBY"".
Select "AdditionalDiskClass" from "Volume management."
If you want to register multiple resources, repeat steps 2 to 4 for each resource.
Example) When you register the class [CL] of the shared volume in userApplication:
A list of the cluster resources that can be registered in userApplication appears.
In the case of the multiple-node cluster operation
Select the cluster resource.
In the case of the single-node cluster operation
Enter the created class name of GDS by selecting "FREECHOICE", since the list of the cluster resources is not displayed.
Set the operation mode of the Gds resources.
Set a flag for the disk class: CL Currently set: 1) HELP 5) MONITORONLY(M) 2) - 3) SAVE+RETURN 4) DEFAULT Choose additionally one of the flags:
DEFAULT
Selecting this attribute restores all settings to their default values.
MONITORONLY, NOT:MONITORONLY
Setting these attributes specifies the error notification from a class to the cluster. The default value is "NOT:MONITORONLY."
MONITORONLY
If this attribute is set, the error notification is not sent to the cluster even when an error is detected in a class of the shared volume.
"MONITORONLY" is displayed for "Currently set", and "5)NOT:MONITORONLY" is displayed for the menu. If "MONITORONLY" is set, a switch due to an error in a class of the shared volume does not occur.
NOT:MONITORONLY
Setting this attribute cancels the setting of "MONITORONLY."
"NOT:MONITORONLY" is displayed for "Currently set", and "5) MONITORONLY" is displayed for the menu.
Note
Set "NOT:MONITORONLY" for at least one class among the shared volumes registered in the Gds resources.
Select "SAVE+EXIT."
[Prerequisites]
Before setting up the Gls resources, you must first set up the virtual interfaces and takeover virtual interfaces. For details, see "PRIMECLUSTER Global Link Services Configuration and Administration Guide: Redundant Line Control Function."
Operation Procedure:
Select "Gls:Global-Link-Services" from "turnkey wizard "STANDBY"".
Select "AdditionalTakeoverIpaddress" and then set the takeover IP address.
If you need to register multiple resources, repeat steps 2 to 4 for each resource.
The takeover IP addresses are displayed as options.
Select one.
Set up the operation mode of the resource.
The operation modes of the resources are "AUTORECOVER(A)" (selective value) and "NOT:AUTORECOVER(N)" (default value). If you select "SAVE+RETURN," the default value "NOT:AUTORECOVER(N)" will be set. Select "AUTORECOVER(A)" if an attempt to recover the resource is to be made for a given duration (default: 60 seconds) when a resource error occurs, or select "NOT:AUTORECOVER(N)" if switchover to another node is to be performed immediately.
To save the Gls resource settings and then exit, select "SAVE+EXIT."
You can change the timeout value of the Gls resource by selecting "(Timeout=60)" and setting any value (seconds).
See
By setting up the value in the StandbyTransitions attribute when the cluster application is created, Gls resources on the standby node can be switched to the "Standby" state and the state of the Gls resources on the standby node can be monitored. For information on how to make this setting, see "6.7.2.1 Creating Standby Cluster Applications."
[Prerequisites]
You need to configure the following information before using takeover network resources.
Define an IP address and a host name to be taken over between nodes
First, define a host address to be taken over between nodes. Next, allocate a host name to this IP address. The allocated host name should be defined in the /etc/hosts file for the node which uses the takeover IP address.
Define a target host
If you want to monitor the reachability of your network using ICMP while a takeover IP address is active, define a host name and an IP address of the device (e.g. router or layer 3 hub) which enables ICMP communication between more than one node in the /etc/hosts file. This process can be skipped if you do not need to monitor the reachability of the network.
Confirming a network interface
Confirm the network interface (e.g. eth2) on which the takeover IP address will be active is being recognized by the operating system and available.
Define a setting file for the cluster
Add the takeover IP address to the following file:
/usr/opt/reliant/etc/hvipalias
Enter entries according to the following rules.
<node name> <takeover> <interface> <netmask/prefix>
<node name> : CF node name of the node which uses the takeover IP address <takeover> : Host name of the takeover IP address <interface> : Network interface name on which the takeover IP address will be activated <netmask/prefix> : Netmask for the takeover IP address (for IPv4), or network prefix length (for IPv6)
Example
When an IPv4 address for the host "takeover" (netmask 255.255.255.0) is taken over between two nodes (node0 and node1) on the network interface eth2, define as follows (specify the 8-digit netmask in hexadecimal).
node0 takeover eth2 0xffffff00 node1 takeover eth2 0xffffff00
When an IPv6 address for the host "takeover6" (network prefix length: 64) is taken over on the network interface eth3, define as follows.
node0 takeover6 eth3 64 node1 takeover6 eth3 64
Note
An IPv6 link local address cannot be used as a takeover network resource. Moreover, it cannot be used as a communication destination of reachability monitoring.
When defining a host name in the /etc/hosts file, do not assign the same host name to the IPv4 address and the IPv6 address.
Operation Procedure:
Select "IpAddresses" of "turnkey wizard "STANDBY"".
Settings of turnkey wizard "STANDBY" (APP1:not yet consistent) 1) HELP 10) Enterprise-Postgres(-) 2) - 11) Symfoware(-) 3) SAVE+EXIT 12) Procedure:SystemState3(-) 4) - 13) Procedure:SystemState2(-) 5) ApplicationName=APP1 14) Gls:Global-Link-Services(-) 6) Machines+Basics(app1) 15) IpAddresses(-) 7) CommandLines(-) 16) LocalFileSystems(-) 8) Procedure:Application(-) 17) Gds:Global-Disk-Services(-) 9) Procedure:BasicApplication(-) Choose the setting to process:15
When you have previously specified the target host to monitor its network reachability using ICMP, select "AdditionalPingHost" and specify that target host.
Ipaddresses and ipaliases (Adr_APP1:not yet consistent) 1) HELP 4) REMOVE+EXIT 7) (NeedAll=yes) 2) - 5) AdditionalInterface 8) (Timeout=60) 3) SAVE+EXIT 6) AdditionalPingHost 9) (InterfaceFilter=) Choose the setting to process:6
The target host name registered in the process of prerequisites will be shown as an option. Select the host name you have previously specified.
1) HELP 2) RETURN 3) FREECHOICE 4) router 5) l3hub 6) takeover Choose another trusted host to ping:4
When you finish specifying the target host, you will be brought back to the previous screen. Since you are required to specify more than one target host, you need to select "AdditionalPingHost" again to add another target host on the previous screen.
Select "AdditionalInterface" to set up the takeover IP address.
When you have more than one IP address, you need to repeat this process for each IP address.
Ipaddresses and ipaliases (Adr_APP1:not yet consistent) 1) HELP 7) PingHostPool[0]=router 2) - 8) PingHostPool[1]=l3hub 3) SAVE+EXIT 9) (NeedAll=yes) 4) REMOVE+EXIT 10) (Timeout=60) 5) AdditionalInterface 11) (InterfaceFilter=) 6) AdditionalPingHost Choose the setting to process:
Takeover IP address registered in the process of prerequisites will be shown as an option.
Select the host name for the takeover IP address you have previously specified.
1) HELP 2) RETURN 3) FREECHOICE 4) router 5) l3hub 6) takeover Choose an interface name:6
Specify the operation mode for the takeover IP address.
Currently set: VIRTUAL,AUTORECOVER (VA) 1) HELP 4) DEFAULT 7) MONITORONLY(M) 2) - 5) BASE(B) 8) PING(P) 3) SAVE+RETURN 6) NOT:AUTORECOVER(A) Choose one of the flags:
DEFAULT
If you choose "DEFAULT", all values will revert back to their default values.
BASE, VIRTUAL
This attribute is effective only when using an IPv4 address. When using an IPv6 address, do not change this attribute. The default value is "VIRTUAL".
- BASE
If you specify "BASE", activation/deactivation of the takeover IPv4 address and activation/deactivation of the physical interface (for example, eth2) are performed at the same time. "BASE" will be shown on "Currently set" and "5) VIRTUAL" is shown on the menu page.
- VIRTUAL
If you specify "VIRTUAL", activation/deactivation of the takeover IPv4 address and activation/deactivation of the logical interface (for example, eth2:1) are performed at the same time. "VIRTUAL" will be shown on "Currently set" and "5) BASE" is shown on the menu page.
You must activate the IPv4 address on the physical interface (for example, eth2) where the logical interface will be created beforehand because the takeover IPv4 address with this attribute specifies the IPv4 address for the logical interface. To activate the IPv4 address on the physical interface beforehand, make settings so that the IPv4 address is activated on the physical interface at startup of the operating system, or register the takeover IPv4 address with the above-mentioned "BASE" attribute to the same takeover network resource.
AUTORECOVER, NOT:AUTORECOVER
If you reactivate the takeover IP address, specify this attribute. The default value is "AUTORECOVER".
- AUTORECOVER
If you specify "AUTORECOVER" and the network interface goes down or becomes unreachable due to an error, it will try to activate the takeover IP address only once. "AUTORECOVER" will be shown on "Currently set"and "6) NOT:AUTORECOVER" is shown on the menu page. When the activation of the takeover IP address fails, it will be notified to the cluster.
- NOT:AUTORECOVER
If you specify "NOT:AUTORECOVER", the "AUTORECOVER" setting will be disabled. "NOT:AUTORECOVER" will be shown on "Currently set" and "AUTORECOVER" is shown on the menu page."
BASIC-IF
You cannot use this attribute. Do not change.
MONITORONLY, NOT:MONITORONLY
- MONITORONLY
If you specify "MONITORONLY" and the network interface goes down or becomes unreachable due to an error, the error will not be notified to the cluster. "MONITORONLY" will be shown on "Currently set" and "7) NOT:MONITORONLY" is shown on the menu page. If you specify this attribute, a switchover due to a takeover IP address failure will not occur.
- NOT:MONITORONLY
If you specify "NOT:MONITORONLY", the "MONITORONLY" setting will be disabled. "NOT:MONITORONLY" will be shown on "Currently set" and "7) MONITORONLY" is shown on the menu page.
Note
At least one out of all the takeover IP addresses you have registered to the takeover network resources should be set to "NOT:MONITORONLY".
PING
By setting this attribute, you can specify the previously configured target host for the takeover IP address. Select the target host name to be monitored which you have set in the process of prerequisites.
1) HELP 2) RETURN 3) router(000) 4) l3hub Choese a ping host of the pool ():3
Change "Timeout" if needed.
Use "Timeout" to specify the timeout value (in seconds) for the system to wait until all registered takeover IP addresses become active or inactive. When any takeover IP address does not become active or inactive after the timeout value elapses, the resource will notify the error to the cluster. The default value is 60 seconds. Specify the value bigger than 45 seconds.
Note
NeedAll, InterfaceFilter
You cannot use these attributes. Do not change.
The procedure resource setting is used to register the state transition procedure resource of the products supported by PRIMECLUSTER in userApplication.
[Prerequisites]
To create a procedure resource, you must first create a state transition procedure and register the procedure to the resource database.
For details, see "D.1 Registering a Procedure Resource."
Operation Procedure:
Select "Procedure:XXXXXXXXXX" from "turnkey wizard "STANDBY"".
Example of registering cluster resources of the BasicApplication class to a userApplication:
Select "AdditionalProcedureResource" from "Application detail Resource wizard."
To register multiple resources, execute steps 2 to 4 for each resource.
The list of cluster resources will appear. Select one.
Note
If a cluster resource does not appear on this screen, it indicates that the cluster resource has not been registered in the resource database. Confirm whether the cluster resource has been registered on each node of the userApplication, which is designed with "6.7.2 Setting Up userApplication." Register cluster resources if they are not registered. For details on the "clgettree" command, see the manual pages of this command. For details on registering the cluster resource in the resource database, see "D.1 Registering a Procedure Resource."
You can change the following on this screen. If necessary, select "SAVE+RETURN" from "Application detail Resource wizard" after that.
Timeout value of the state transition procedure
The default value is 1,800 seconds. If you use a machine that requires more than 1800 seconds for timeout, you need to change the value by selecting "TIMEOUT."
Priority within the resource class
The default value is specified by -p option when the state transition procedure resource is registered with the "claddprocrsc" command. If the -p option is not specified, 65535 is used as the default value. If you register multiple resources of the same class in the cluster application and specify the order of Online and Offline, change this value by selecting "PRIORITY" from the following screen. The resources will be Online in the ascending order, and will be Offline in the descending order.
