4.2.12 Switchover Using the Batch Command [Switchover Destination Site]

Perform switchover using a batch command.

When the automatic collection of switchover information is enabled, performing switchover using the batch command disables the setting.

If the maintenance mode of image is set, release the maintenance mode of image before executing the batch command.

[Windows Manager]

>Installation_folder\SVROR\Manager\bin\rcxadm imagemgr set -attr vm.clone.maintenance=false <RETURN>

[Linux Manager]

# /opt/FJSVrcvmr/bin/rcxadm imagemgr set -attr vm.clone.maintenance=false <RETURN>

For Site Switchover
Execute the batch command on the switchover destination site to switch the site.
[Windows Manager]
>Installation_folder\SVROR\Manager\bin\rcxrecovery <RETURN>
[Linux Manager]
# /opt/FJSVrcvmr/bin/rcxrecovery <RETURN>

For Tenant Switchover

Execute the batch command on the switchover destination site to switch over tenants.

Note

When performing the switchover of the last tenant that will remain on the switchover source site, specify the -global option in addition to the -tenant option. For the Active-Active operation, do not use the -global option.

For Tenant Switchover (tenant1) in the Active-Active Operation
[Windows Manager]
>Installation_folder\SVROR\Manager\bin\rcxrecovery -tenant tenant1 <RETURN>
[Linux Manager]
# /opt/FJSVrcvmr/bin/rcxrecovery -tenant tenant1 <RETURN>
When switching over tenants (tenant2, tenant3) in the Active-Active operation (No tenants will remain on the switchover source site after the switchover operation)
[Windows Manager]
>Installation_folder\SVROR\Manager\bin\rcxrecovery -tenant tenant2,tenant3 -global <RETURN>
[Linux Manager]
# /opt/FJSVrcvmr/bin/rcxrecovery -tenant tenant2,tenant3 -global <RETURN>

For details on the rcxrecovery command, refer to "B.2 rcxrecovery".
The switchover status of the rcxrecovery command can be confirmed from output messages or the DR execution log.
For details on the DR execution log, refer to "D.2 Logs Regarding the rcxrecovery Command".

In addition, the following operations require manual operations or decisions by an operator. Respond to inquiries from the system to continue execution of the command.

For virtual L-Server switchover (VM guest), the following messages are displayed.

It is necessary to manually delete the network settings on which cleanup is performed in the network settings "corresponding to network resources" configured on the switchover destination site beforehand. It is necessary to manually create the network (virtual switches, port groups) which has been configured on the switchover source site beforehand.
After the operation is complete, enter "y", and continue the command.

For network resources that have been cleaned up, clear their network configurations manually using VM management software or the network devices. Also, if you have manually configured the network on the primary site, configure the network for the backup site again.

Press [y] when the reconfiguration is complete. : y

When neither the -tenant option nor the -nocleanup option are specified for cleanup of the backup site, an inquiry will be displayed to ask if you really want it to be cleaned up. Press "y" to approve the cleanup and continue the rcxrecovery command. If cleanup of the backup site cannot be performed, press "n" to quit the command.
When VM management software is not configured on a physical L-Server using the physical/virtual L-Server switchover (VM host/VM guest) method, it is necessary to manually restore the VM management software during switchover. The rcxrecovery command will show a prompt while waiting for the restoration of VM management software. In that case, continue the command after restoration of VM management software is completed.
If there is a physical L-Server with VMware ESXi installed on the switchover target server, it is necessary to register the VM host with SVOM and register the Resource Orchestrator agent with the Resource Orchestrator manager during execution of the rcxrecovery command.
The command will be suspended until registration of the VM host and agent is completed. When the registration is completed, resume the command.
If the Resource Orchestrator agent has already been registered with the Resource Orchestrator manager, it is not necessary to register the agent again.

If the maintenance mode of images is released before execution of the batch command, set the maintenance mode of images as necessary.

[Windows Manager]

>Installation_folder\SVROR\Manager\bin\rcxadm imagemgr set -attr vm.clone.maintenance=true <RETURN>

[Linux Manager]

# /opt/FJSVrcvmr/bin/rcxadm imagemgr set -attr vm.clone.maintenance=true <RETURN>

Information

The rcxrecovery command performs the following:

Mapping of storage
Storage resource mapping is performed, based on the files defining the relations between the storage on the primary and backup sites.
Division of import resources
Imported resources are divided using the following units:
- Resources in the global pool and resources belonging to individual tenants
- Physical L-Server resources and virtual L-Server resources
Deletion of the backup site environment
Deletes the following environments used for backup site operations.
- L-Platform Templates
- L-Platform Configuration Information
- Resource Information
- Accounting Information
- Metering Logs
- Usage Charges
Importing of information
Import the following information:
- L-Platform Templates
- L-Platform Configuration Information
- Resource Information
- Accounting Information
- Metering Logs
- Usage Charge Files

Note

Configured physical servers and linked L-Servers for which only configuration definition has been created are not covered by disaster recovery switchover.
To facilitate the operation for linking with L-Servers again after switchover, execute the following rcxadm config command in advance on the primary site, and then back up the configuration information.

[Windows Manager]

>Installation_folder\SVROR\Manager\bin\rcxadm config filter -convert -indir indir -outdir outdir <RETURN>

[Linux Manager]

# /opt/FJSVrcvmr/bin/rcxadm config filter -convert -indir indir -outdir outdir <RETURN>

The following information will be output to outdir, using the file name convert.txt.
After completing the restoration procedure, perform creation of a link with an L-Server and setting the scope again as shown below:

[command]
rcxadm lserver convert -with BX900_1 -name physical_lserver_1
rcxadm lserver convert -with BX900_3 -name physical_lserver_2 -to /tenant_1
rcxadm lserver convert -with BX900_6 -name physical_lserver_3 -label test_label3 -comment test_comment3 -to /folder_1
[user]
user_1, physical_lserver_1
user_1, /tenant_1/physical_lserver_2
user_2, /folder_1/physical_lserver_2
[usergroup]
usergroup_1, physical_lserver_1
usergroup_2, /folder_1/physical_lserver_3

Storage using the ETERNUS dynamic LUN mirroring is switched to the LUN that was prepared on the backup site when the switchover operation is performed after a disaster. Operation may be different if the storage is not released even after deleting an L-Platform.
Perform operation so that the import of the resource configuration information and user definition file (XML file) using the rcxrecovery command is not executed at the same time as modification of the resource configuration information or user definition file (XML file) due to L-Server creation, or export operations using the rcxmgrexport command.
If the -allon option or the -alloff option is not specified for the rcxrecovery command, the power status of L-Servers recovered using the rcxrecovery command becomes the same as that when the switchover information was collected from the switchover source site. L-Servers start up in order of the priority set for L-Servers.
The RC console may not display the correct power status of L-Servers immediately after a disaster recovery switchover. Wait for about three minutes after the switchover to see the correct power status of L-Servers.
As the performance information (the information displayed in the dashboard and in operational status displays) depends on the configuration of the primary site, this is not carried over to the backup site when switched over. The information on the backup site is displayed.
Importing may fail when switchover is performed to the backup site, due to a VM guest switchover failure. In this case, perform the import again using the following procedure, but excluding the managed servers for which import failed. Disaster recovery switchover fails when configuration import fails. In this case, execute the failover command again after taking the corrective action for the cause of the import failure. When you execute the command again, the -nocleanup option needs to be specified if the previous command was executed without the -nocleanup option and configuration cleanup was performed and the "cleanup of resources" process ended in "completed".
The following shows an example of where the VM guest could not be restored during L-Server import.
Example
Example 1
1. As an error occurred during import, the following error is displayed:
  >rcxrecovery <RETURN> FJSVrcx:ERROR:62569:/tenant1/l-platform1/l-server1:lserver:import was interrupted. Message=:67154: VM Guest not found
2. The status of the relevant resources is displayed to help determine the cause of the error.
  >rcxadm config show -type lserver -name /tenant1/l-platform1/l-server1 -dir dir1 -format xml<RETURN>
  For -dir, specify the folder that was most recently exported in the folder below.
  - The folder specified in the rcxrecovery -dir option
  [Windows Manager]
  - The import folder specified in #rcxrecovery_define of fa_dr.rcxprop\ManagerExport\ROR export_date
  [Linux Manager]
  - The import folder specified in #rcxrecovery_define of fa_dr.rcxprop/ManagerExport/ROR export_date
3. Delete the resource where the problem occurred from the export file. Make a note of the name of the deleted resource.
  >rcxadm config filter -exclude lserver -name /tenant1/l-platform1/l-server1 -indir dir1 -outdir dir2 <RETURN>
4. Use the export file output by the command in step 3 to overwrite the file in the folder specified for -indir in step 3.
5. Import again.
  >rcxrecovery -nocleanup<RETURN>
Example 2
1. As an error occurred during import, the following error is displayed:
  >rcxrecovery <RETURN> FJSVrcx:ERROR:67335:registering of VM guest failed. detail=(registervm L-ServerB-2:task result error, vmerrno=104,ip=192.168.201.70,host=192.168.201.115)
2. Identify the L-Server on which the error occurred.
  In the switchover information folder, check the l_servers.xml file that was most recently exported.
  [Windows Manager]
  The import folder specified in #rcxrecovery_define of fa_dr.rcxprop/ManagerExport/ROR_export_date\l_servers.xml
  [Linux Manager]
  The import folder specified in #rcxrecovery_define of fa_dr.rcxprop/ManagerExport/ROR_export_date/l_servers.xml
  In the xml file, find the VM guest name (L-ServerB-2) displayed in the error message, and check the name and id in the L-Server element in which the entry is included.
  <LServer name="L-ServerB" id="******">
  ...
  <VmGuest name="L-ServerB-2">
3. Check resource_folders.xml in the switchover information folder.
  When there are L-Platforms and tenants in which the L-Server with id checked in step 2 is included in the xml file, check the names.
  <Folder name="TenantB" tenant="TENANT">
  ...
  <Folder name="L-platformB" tenant="LPlatform">
  ...
  <LServer name="L-ServerB" id="******">
4. Delete the resource where the problem occurred from the export file. Make a note of the name of the deleted resource.
  >rcxadm config filter -exclude lserver -name /tenantB/l-platformB/L-ServerB -indir dir1 -outdir dir2 <RETURN>
  Add the tenant name, the L-Platform name, and the L-Server name checked in steps 2 and 3 to -name, connecting them with slashes ("/").
  - When a tenant is included
    /tenantB/l-platformB/L-Server-B
  - When a tenant is not included
    /L-Server-B
  Specify the folder checked in step 2 for -dir.
  [Windows Manager]
  The import folder specified in #rcxrecovery_define of fa_dr.rcxprop/ManagerExport/ROR_export_date
  [Linux Manager]
  The import folder specified in #rcxrecovery_define of fa_dr.rcxprop/ManagerExport/ROR_export_date
5. Use the export file output by the command in step 4 to overwrite the file in the folder specified for -indir in step 4.
6. Import again.
  >rcxrecovery -nocleanup<RETURN>

Point

When the automatic collection of switchover information is enabled, performing cleanup disables the setting.
Refer to "4.2.18 Exporting Configuration Information and Resuming Automatic Collection of Switchover Information [Switchover Destination Site]" to enable the setting when necessary.