The SF provides the interface for managing the shutdown of cluster nodes when error conditions occur. The SF also advises other PRIMECLUSTER products of the successful completion of node shutdown so that recovery operations can begin.
The SF is made up of the following major components:
The Shutdown Daemon (SD)
One or more Shutdown Agents (SA)
sdtool(1M) command
Shutdown Daemon (SD)
The SD is started at system boot time and is responsible for the following:
Monitoring the state of all cluster nodes
Monitoring the state of all registered SAs
Reacting to indications of cluster node failure and eliminating the nodes forcibly
Resolving split-brain conditions
Notifying other PRIMECLUSTER products that nodes were forcibly eliminated
Checking the connection to the option hardware or the virtual machine function that forcibly eliminates cluster nodes periodically (in 10-minute intervals)
The SD uses SAs to perform most of its work with regard to cluster node monitoring and forced node elimination. In addition to SA's, the SD interfaces with the Cluster Foundation layer's ENS system to receive node failure indications and to advertise node elimination completion.
The SD starts the SA periodically (in 10-minute intervals) to check the route that forcibly eliminates cluster nodes. The SD reflects the checked route status to the test status of each SA (Test State) displayed by the sdtool(1M) command.
Shutdown Agents (SA)
The SA's role is to attempt to shut down a remote cluster node in a manner in which the shutdown can be guaranteed. Some of the SAs are shipped with the SF product, but may differ based on the architecture of the cluster node on which SF is installed. SF allows any PRIMECLUSTER service layer product to shut down a node whether RMS is running or not.
An SA is responsible for shutting down, and verifying the shutdown of a cluster node. Each SA uses a specific method for performing the node shutdown such as:
SA_blade offers the shutdown agent for the Fujitsu Technology Solutions Blade servers.
SA_ipmi offers the shutdown agent for IPMI-based systems.
SA_lkcd offers the shutdown agent that uses the kernel panic status of other nodes.
SA_mmb offers the shutdown agent that uses the management board (MMB) on the node PRIMEQUEST 2000 series.
SA_irmc offers the shutdown agent that uses iRMC/MMB on the node PRIMEQUEST 3000 series.
SA_icmp offers the shutdown agent that checks whether a node to be stopped is in the active or inactive state by using a network route.
SA_vmchkhost offers the shutdown agent of the system which uses the KVM virtual machine function.
SA_libvirtgp and SA_libvirtgr offer the shutdown agent of the system which uses the KVM virtual machine function.
SA_vwvmr offers the shutdown agent that is cooperated with VMware vCenter Server.
SA_vmk5r offers the shutdown agent using the FUJITSU Hybrid IT Service FJcloud-O/FJcloud-Baremetal API.
SA_vmosr offers the shutdown agent using the OpenStack API.
SA_vmawsAsyncReset offers the shutdown agent using the AWS CLI.
SA_vmazureReset offers the shutdown agent using the Azure CLI.
SA_vmnifclAsyncReset offers the shutdown agent using the NIFCLOUD API.
See "7.2 Available SAs" for more information on SA.
If more than one SA is used, the first SA in the configuration file is used as the primary SA. SD always uses the primary SA. The other secondary SAs are used as fall back SAs only if the primary SA fails for some reason.
sdtool command
The sdtool(1M) command is the command line interface for interacting with the SD. With it the administrator can:
Start and stop the SD (although this is typically done with an RC script run at boot time)
View the current state of the SA's
Force the SD to reconfigure itself based on new contents of its configuration file
Dump the contents of the current SF configuration
Enable/disable SD debugging output
Eliminate a cluster node
Note
Although the sdtool(1M) command provides a cluster node elimination capability, the preferred method for controlled shutdown of a cluster node is the /sbin/shutdown command.
