The configuration for distributed execution using Auto Scaling linking with Amazon Web Service is shown below.
Definition procedure
The procedure to build the above configuration are as follows.
In additions, for the detail of the cloud service, refer to the documentation provided from Amazon Web Service.
In the Amazon Elastic Compute Cloud (hereinafter referred to as Amazon EC2), set the following to the machine installed the Systemwalker Operation Manager for the execution server.
Settings required for distributed execution
When registering execution users for distributed execution and using distributed execution, define the necessary job execution controls.
For the detail, refer to the "2.8 Definition of Job Execution Control".
Creating multiple subsystem environment (Only using multiple subsystem)
When using multiple subsystem, create subsystem environment before the following procedure. For setting up multiple subsystem operations, refer to the "2.5 Definition of Multi-Subsystem Operations".
In the Amazon EC2, create the Amazon machine image (hereinafter referred to as AMI) for the execution server from the machine of the execution server that set procedure 1.
Create "Launch configurations" in the Auto Scaling of the Amazon EC2.
For AMI, select the AMI that created in the procedure 2.
Set security groups to allow communication between the schedule server and the execution server. For the port number used for two-way communication between the schedule server and the execution server, refer to the "C.1 Listing of Port Numbers".
Create "Auto Scaling Group" in the Auto Scaling of the Amazon EC2.
For the launch configurations , select the "Launch configurations" that created in the procedure 3.
Set the schedule server and the execution servers of the Auto Scaling group to have private connections.
At least one instance must always be running in the Auto Scaling group. Therefore, in the scaling policy, set the minimum size of the instance to be started to 1 or greater.
Add the End Instance lifecycle hook to the Auto Scaling group so that the instance is not deleted as soon as it is scaled in. Set the pause time for scale-in in the heartbeat timeout for the life cycle hook. If the job running on the execution server does not finish within the set time, it will abend.
In the schedule server, create a queue to use the distributed execution. In the configure of the host group to be assigned to the distributed execution queue, register the instance of the Auto Scaling group that you started in step 4 as the configuration host. For an example design of the distributed execution, refer to the "I Want to Automatically Distribute Jobs for Execution on Servers with Low Load" in the Systemwalker Operation Manager How-To Guide.
In AWS Lambda, create a function that asks the Systmwalker Operation Manager to add/remove a configuration host. The function must be created in the same AWS account and region as the Auto Scaling group. Create a function that execute the following.
Determines whether to scale out or scale in from event data sent to AWS Lambda during autoscaling, and extracts instance ID.
Determine the scale action type from the "detail-type" value of the event data
EC2 Instance Launch Successful: scale out
EC2 Instance-terminate Lifecycle Action: scale in
Extracts the instance ID from the value of "EC2InstanceId" in the event data
Extracts the host name requested to be added/removed as a distributed execution destination from the target instance ID, and requests to add/remove a configuration host
Scale out: Request to Systemwalker Operation Manager to add a configuration host
Scale in: Request to Systemwalker Operation Manager to delete a configuration host
Use the Web API to request the addition or removal of a configuration host. For the how to use Web API, refer to the "Web API [Windows] [Linux]" in the Systemwalker Operation Manager Reference Guide.
Point
You can specify the multiplicity when you request to add a configuration host. If omitted, multiplicity 10 is set. Set the multiplicity as needed.
In the Amazon CloudWatch Events, set the Lambda function to be executed in response to an auto Scaling event. Create the following rules in the Amazon CloudWatch Events. Rules should be created in the same AWS account and region as the Auto Scaling group.
Event source
Service name: Auto Scaling
Event type: Instance Launch and Terminate
Specific instance event: select the following
EC2 Instance Launch Successful
EC2 Instance-terminate Lifecycle Action
Target: Lambda function created in procedure 6
Information
If a running job does not finish within the heartbeat timeout period set in procedure 4 and ends abnormally, review the Auto Scaling group settings and execute the job again.
