PRIMECLUSTER Installation and Administration Guide 4.1 (for Linux)

5.2 Setting Up Fault Resource Identification and Operator Intervention Request

The fault resource identification is a function that outputs a message to syslogd(8) and Cluster Admin and a history of failed resources to Resource Fault History if a failure occurs in a resource or node that is registered to a cluster application.

After setting the initial configuration of the resource database, specify the settings for enabling fault resource identification and operator intervention request. An example of a message displayed by fault resource identification is shown below.

6750 A resource failure occurred.  SysNode:node1RMS userApplication:app0 Resource:apl1

The operator intervention request function displays a query-format message to the operator if a failed resource or a node in which RMS has not been started is found when a cluster application is started. The messages for operator intervention requests are displayed to syslogd(8) and Cluster Admin.

1421 userApplication "app0" was not started automatically because all SysNodes that make up userApplication were not started within the prescribed time.
Forcibly start userApplication in SysNode "node1RMS"? (no/yes)

Message number: 1001
Warning: When userApplication is forcibly started, the safety check becomes disabled.
If the operation is used incorrectly, data may be damaged and the consistency may be lost.
Check that userApplication to be forcibly started is not online in the cluster before executing the forced startup.

For details on the messages displayed by the fault resource identification function and the messages displayed by the operator intervention request function, see "Failed Resource and Operator Intervention Messages (GUI)" and "Operator Intervention Messages."

This section describes procedures for operating fault resource identification and operator intervention request.

• After PRIMECLUSTER is installed, fault resource identification and operator intervention request are initially disabled. The following Cluster Admin functions are also disabled:
  • Messages for fault resource identification and operator intervention request are not displayed to Cluster Admin.
  • The list of resources that are currently affected by faults is not displayed in the Resource Fault History screen of Cluster Admin.
  • The fault history of the resources is not displayed in the Resource Fault History screen of Cluster Admin.
• To view the manual pages of each command, add "/etc/opt/FJSVcluster/man" to the MANPATH variable.

Preparation prior to displaying fault resource identification and operator intervention request messages

The fault resource identification and operator intervention request messages are displayed by using syslogd(8). daemon.err is specified to determine the priority (facility.level) of the fault resource identification and operator intervention request messages. For details on the priority, see the manual page describing syslog.conf(5). The procedure for displaying the fault resource identification and operator intervention request messages on the console is explained below. Execute the following procedure on all the nodes.

Procedure:

1. Log in the node using system administrator access privileges.
2. Check the setting of syslogd in /etc/syslog.conf to see that daemon.err is set to be displayed on the console. For further details on /etc/syslog.conf, see the manual pages of syslog.conf(5).

   (Example) Daemon.err is set to be displayed on the console.

          daemon.err        /dev/console

   If daemon.err is not set to be displayed on the console, change the setting of syslogd in /etc/syslog.conf. To enable this change, send the SIGHUP signal to syslogd. Execute the following command to let syslogd reread the definition file.

   # kill -SIGHUP `cat /var/run/syslogd.pid`

3. Starting the console.

   Start the console.

   (Example)

     # kterm -C

If a Japanese-language system is being used, use a console that supports Japanese.

By setting the operator intervention request messages, messages other than the operator intervention request messages will be displayed on the console. If a console that does not support Japanese is used, Japanese messages will not be displayed.

Enabling the operation of fault resource identification and operator intervention request

1. Execute the "clsetparam" command and specify the settings for enabling the fault resource identification and operator intervention request. Execute this procedure in any node that is part of the cluster system.

   # /etc/opt/FJSVcluster/bin/clsetparam -p AppWatch ON

2. Execute the "clsetparam" command, and check that the parameters are set so that the operation of fault resource identification and operator intervention request is enabled. Execute this procedure on any node that is part of the cluster system.

   # /etc/opt/FJSVcluster/bin/clsetparam -p AppWatch

   ON

3. Restart all the operating nodes.

   If a node is stopped, the fault resource identification and operator intervention request begin operating in that node from the next node reboot.

Disabling the operation of fault resource identification and operator intervention request

To cancel the setting previously made for "Enabling the operation of fault resource identification and operator intervention request," perform the following procedure:

1. Execute the "clsetparam" command and specify the settings for disenabling the fault resource identification and operator intervention request. Execute this procedure in any node that is part of the cluster system.

   # /etc/opt/FJSVcluster/bin/clsetparam -p AppWatch OFF

2. Execute the "clsetparam" command, and check that the parameters are set so that the operation of fault resource identification and operator intervention request is disenabled. Execute this procedure on any node that is part of the cluster system.

   # /etc/opt/FJSVcluster/bin/clsetparam -p AppWatch

   OFF

3. Restart all the operating nodes.

   If a node is stopped, the fault resource identification and operator intervention request stop operating in that node from the next node reboot.