5.2 Setting Up Fault Resource Identification and Operator Intervention Request

The fault resource identification is a function that outputs a message to syslogd(8) and Cluster Admin and a history of failed resources to Resource Fault History if a failure occurs in a resource or node that is registered to a cluster application.

After setting the initial configuration of the resource database, specify the settings for enabling fault resource identification and operator intervention request. An example of a message displayed by fault resource identification is shown below.

6750: A resource has faulted. SysNode:node1RMS userApplication:app0 Resource:apl1

The operator intervention request function displays a query-format message to the operator if a failed resource or a node in which RMS has not been started is found when a cluster application is started. The messages for operator intervention requests are displayed to syslogd(8) and Cluster Admin.

1421: The userApplication "userApplication" did not start automatically because not all of the nodes 
where it can run are online.
Forcing the userApplication online on the SysNode "SysNode" is possible.
Warning: When performing a forced online, confirm that RMS is started on all nodes in the cluster, 
manually shutdown any nodes where it is not started and then perform it.For a forced online, 
there is a risk of data corruption due to simultaneous access from several nodes.
In order to reduce the risk, nodes where RMS is not started maybe forcibly stopped.
Are you sure wish to force online? (no/yes) Message No.: number

See

For details on the messages displayed by the fault resource identification function and the messages displayed by the operator intervention request function, see "3.2 CRM View Messages" and "4.2 Operator Intervention Messages" in "PRIMECLUSTER Messages."

Note

To view the manual pages of each command, add "/etc/opt/FJSVcluster/man" to the MANPATH variable.

Preparation prior to displaying fault resource identification and operator intervention request messages

The fault resource identification and operator intervention request messages are displayed by using syslogd(8)/rsyslogd(8). daemon.err is specified to determine the priority (facility.level) of the fault resource identification and operator intervention request messages.

For details on the priority, see the manual page describing syslog.conf(5)/rsyslogd.conf(5).

If the fault resource identification and operator intervention request messages need to be output to the console, execute the following procedure on all the nodes.

Procedure:

Log in the node using system administrator access privileges.
Check the setting of syslogd/rsyslogd and change it so that daemon.err will be displayed on the console.
- RHEL6
  1. Check the setting of rsyslogd in /etc/rsyslog.conf to see that daemon.err is set to be displayed on the console.
    (Example) Daemon.err is set to be displayed on the console.
```
daemon.err        /dev/console
```
    For further details on /etc/rsyslog.conf, see the manual pages of rsyslog.conf(5).
  2. If daemon.err is not set to be displayed on the console, change the setting of rsyslogd in /etc/rsyslog.conf.
    To enable this change, restart the system log daemon by executing the following command.
```
# /etc/init.d/rsyslog restart
```
- RHEL7
  1. Check the setting of rsyslogd in /etc/rsyslog.conf to see that daemon.err is set to be displayed on the console.
    (Example) Daemon.err is set to be displayed on the console.
```
daemon.err        /dev/console
```
    For further details on /etc/rsyslog.conf, see the manual pages of rsyslog.conf(5).
  2. If daemon.err is not set to be displayed on the console, change the setting of rsyslogd in /etc/rsyslog.conf.
    To enable this change, restart the system log daemon by executing the following command.
```
# systemctl restart rsyslog.service
```
Starting the console.
If you are using the graphical environment, execute the following command to start the console. For the textual environment or the remote environment using SSH or Telnet, this step is not required.
```
# xterm -C
```

Identifying the fault resource and changing the operation setting of operator intervention request

Use the clsetparam command to change the setting. For details, see the manual page of clsetparam.