When registering a large number of jobs, design the system so that multiple job processes are executed in a single job reducing the number of jobs.
Since queue control, exclusive control and job execution environment configuration are carried out for each job, more jobs can result in larger overheads. If the schedule server and the execution server are different, communication and transfer of job output files between the servers will also result in overheads.
Guides when registering a large number of jobs
Estimate registered number of jobs and process time of each job and make sure that they are processed within the operation time. For network jobs, carry out confirmation in the network job environment.
Confirm in advance that the jobs are processed within operation time: for example, use a business model in which provisional jobs are set to "sleep" status, during the time estimation process.
[Example] Business model
Number of registered jobs | 3,000 |
Operation time | 1 hour |
Job process time | 1 sec each |
If all the jobs have not been completed within the operation time as a result of executing the above business model, change the business model by putting multiple jobs into a single job to reduce the registered number of jobs and test again.
Reducing the number of jobs
Create jobs as follows and make settings so that the number of jobs is as less as possible.
Example 1: Register multiple commands as one shell script.
(Before reduction) One command is described as one job
Job 1: date>>/var/tmp/gyoumu.log Job 2: gyoumushori.exe Job 3: date>>/var/tmp/gyoumu.log
(After reduction) Multiple processes are described in a shell script as one job.
Job 1: date>>/var/tmp/gyoumu.log gyoumushori.exe date>>/var/tmp/gyoumu.log
Before improvement, each of three commands (log date, process and log date again) is registered as one job. Overheads by queue control processes such as the limit set for Max Execution Jobs, authentication processes, internal processes and job execution environment generation processes such as preparation of standard output/error output files increase the CPU usage, and as a result, it may be possible that the CPU cannot be used for the original job processes. Putting three jobs into one reduces the processes executed for each job.
Example 2: Use the job step of JCL (Job Control Language).
In Systemwalker Operation Manager, it is possible to create jobs in the job step configuration, using the JCL (Job Control Language).
Multiple job steps can be described in one job and shell scripts can be executed corresponding to the job steps. Therefore multiple shell scripts can be configured as one job.
For example, if there are 100 processes and each process is made up of 10 shell scripts as one job net, 1000 jobs will be needed without using job steps. However, by using job steps, 10 shell scripts can be described as one job, reducing the number of jobs to 100.
However, when using JCL to describe network jobs, it is possible to specify the execution destination server for each job step, which may result in additional overheads due to processes such as communication with the destination server per job step and transferring output files. Therefore, putting multiple shell scripts into one job may not improve performance.
For details on the JCL, see the Systemwalker Operation Manager Reference Guide.
