Top
Interstage Big Data Complex Event Processing Server V1.0.0 User's Guide
Interstage

6.3.4 Tuning

This section explains tuning for this product, as follows:

6.3.4.1 Tuning JVM Options

Two Java VMs (hereafter, referred to as "JVMs") operate within one CEP engine - one performs input adapter and high-speed filter processing, and the other performs complex event processing and output adapter processing.

If analysis of the resource log has revealed an excess or insufficiency of JVM heap memory, tune the JVM options of the CEP engine. This will avoid problems such as decline in performance caused by garbage collections in the CEP engine.

The table below lists the JVM options that can be tuned:

JVM option

Initial value at CEP engine creation

Maximum value of the memory allocation pool

2048MB

Initial value of the memory allocation pool

512MB

Maximum value of the permanent generation area

192MB

Follow the steps below to tune the JVM options:

  1. Check the current JVM option settings.

  2. Using the resource log, check the heap memory usage.

  3. Calculate the heap memory size required - if there is enough memory, there is no need to tune.

  4. Change the JVM option settings.

  5. Start or restart the CEP engine (to reflect the changed settings).

  6. Follow these steps again to check the changed settings.

Check the current JVM option settings

Execute cepgetjvmopt to check the JVM option settings of the CEP engine.

A general user permission can be used to execute this command.

Example

Executing cepgetjvmopt

When checking the JVM options of complex event processing for a CEP engine (CepEngine).

$ cepgetjvmopt cep -e CepEngine<ENTER>
xmxSize :5120m

xmsSize :256m

maxPermSize :96m

Command cepgetjvmopt executed successfully.

Note that "m" stands for "MB".

Using the resource log, check the heap memory usage

Analyze the resource log to check the maximum value of the Java VM heap memory usage for the CEP engine. Specifically, check the new generation area, old generation area, and permanent generation area.

The items below relate to JVM heap memory:

This applies to the resource log of the high-speed filter and the resource log of complex event processing.

Refer to "6.1.6.4 Checking the Resource Usage of the CEP Engine" for information on the format of the resource log.

Calculate the heap memory size required

If the resource log has been used to analyze the respective maximum values of the new generation area, old generation area, and permanent generation area, calculate the heap memory size required, as shown below.

If the heap memory size required is less than the setting value, then there is no need to tune.

Calculate the maximum value of the memory allocation pool
(maxValOfNewGenerationArea + maxValOfOldGenerationArea + maxValOfPermanentGenerationArea) x 1.2

1.2 is the safety factor - if heap memory usage can vary widely with the time of the year or time period, set a higher safety factor.

Calculate the maximum value of the permanent generation area
maxValOfPermanentGenerationArea x 1.2

1.2 is the safety factor - if heap memory usage can vary widely with the time of the year or time period, set a higher safety factor.

Change the JVM option settings

Execute cepsetjvmopt as a superuser to change the JVM option settings of the CEP engine.

Example

Executing the cepsetjvmopt command

When changing the maximum value of the memory allocation pool in the JVM options for complex event processing of a CEP engine (CepEngine) to 512 MB. The respective defaults are used for the initial value of the memory allocation pool and the maximum value of the permanent generation area.

# cepsetjvmopt cep -xmx 512m -e CepEngine<ENTER>
Command cepsetjvmopt executed successfully.

Note that "m" stands for "MB".

Note

If an option of cepsetjvmopt (-xmx, -xms, or -xxmp) is omitted, the default value for that option will be used.

Start or restart the CEP engine.

Start or restart the CEP engine in order to reflect the changed JVM option settings in the CEP engine.

If the CEP engine is running, it must be stopped (refer to "6.1.7 Stopping the CEP Engine" for details).

Refer to "6.1.5 Starting the CEP Engine" for details.

A general user permission can be used to perform this task.

Example

Restarting the CEP engine

When restarting a running CEP engine (CepEngine).

$ cepstopeng -e CepEngine<ENTER>
Command cepstopeng executed successfully.

$ cepstarteng -e CepEngine<ENTER>
Command cepstarteng executed successfully.

6.3.4.2 Tuning File Descriptors

The number of file descriptors required for operating one CEP engine depends on the number of simultaneous socket connections or high-speed filter statements described in the rule definition filter rules.

If an error occurs due to insufficient filter descriptors, or file descriptors are found to be excessive or inadequate according to resource log analysis, it is necessary to specify the file descriptor upper limit.

The required number of file descriptors for each CEP engine is calculated as follows:

numOfFileDescriptorsRequired = A + (8 x B) + 371
Table 6.1 Placeholders in the file descriptor number tuning formula

Item

Description

Required Number

A

Number of simultaneous socket connections

Use the largest number of assumed simultaneous connections.

Refer to the CEP engine operation status or the resource log output of the number of simultaneous connections to check the change in the current or previous number of simultaneous connections.

Refer to "8.4 cepdispeng" or "6.1.6.4 Checking the Resource Usage of the CEP Engine" for information on how to reference it.

B

Total number of IF-THEN statements described in the filter rules

Refer to the CEP engine operation status or the resource log output of the number of high-speed filter statements to check the current number of IF-THEN statements.

Refer to "8.4 cepdispeng" or "6.1.6.4 Checking the Resource Usage of the CEP Engine" for information on how to reference it.

Note

When the file descriptors are exhausted, operations might no longer be able to continue. Therefore, specify a value with some allowance for the number of simultaneous connections in A.

Compare the value calculated above with the actual number of file descriptors that can be used by the engine execution user. If there are multiple CEP engines, compare the largest value of the calculation result from each CEP engine.

Execute the command below as a superuser to display the maximum number of file descriptors that the engine execution user can use:

# /bin/su -c 'ulimit -n' engineExecutionUserName <ENTER>

If the calculated value is larger, then change /etc/security/limits.conf with it and restart the OS (refer to OS documentation for information on how to change the value).

Example

Setting /etc/security/limits.conf

This example changes the maximum number of file descriptors that the engine execution user "isbdcep" can use from the default value of 1024 to 2048.

isbdcep    soft    nofile    2048
isbdcep hard nofile 2048

6.3.4.3 Tuning Trace Logs

In this product, the input-output information for event sender applications and receipt processing states for event sender application requests are output to trace logs.

By stopping the output of trace logs, it is possible to improve the efficiency of event processing when the HTTP adapter and SOAP adapter are used.

Note

Stopping the output of trace logs will mean that more time will have to be spent to determine the cause of eventual problems. If efficiency is not a problem, it is recommended to output the trace logs.

The flow to stop (or resume) trace log output is as follows (perform these operations as superuser):

(1) Stop (or Resume) Logging Event Sender Application I/O Information

(2) Stop (or Resume) Receipt Processing State of Event Sender Application Requests Log Output

(3) Restarting the CEP Service

(4) Starting the CEP Engine

(1) Stop (or Resume) Logging Event Sender Application I/O Information

Edit /var/opt/FJSVihs/servers/FJapache/conf/httpd.conf and set the IHSTraceLog setting value to "off".

Example

Setting /var/opt/FJSVihs/servers/FJapache/conf/httpd.conf

Before changing

IHSTraceLog "|/opt/FJSVihs/bin/ihsrlog -s logs/tracelog 2 5"

After changing

#IHSTraceLog "|/opt/FJSVihs/bin/ihsrlog -s logs/tracelog 2 5"
IHSTraceLog off

When resuming log output, also edit the httpd.conf file and revert the contents to its previous state.

(2) Stop (or Resume) Receipt Processing State of Event Sender Application Requests Log Output

Execute the commands below to stop the receipt processing state log output contents of the event sender application request (please note that we added a backslash ("\") and newline characters in the command line example below for readability - the actual command line does not have a backslash and newline):

# /opt/FJSVisjee/bin/asadmin set \
CEPengineName_flt-config.http-service.property.ISJEELogHttpTraceEnable=false <ENTER>

To resume log output, perform the following:

# /opt/FJSVisjee/bin/asadmin set \
CEPengineName_flt-config.http-service.property.ISJEELogHttpTraceEnable=true <ENTER>

Example

Executing the command

If the CEP engine name is "CepEngine", perform the following to deter log output (please note that we added a backslash ("\") and newline characters in the example below for readability - the actual command line does not have a backslash and newline).

# /opt/FJSVisjee/bin/asadmin set \
CepEngine
_flt-config.http-service.property.ISJEELogHttpTraceEnable=false <ENTER>
CepEngine_flt-config.http-service.property.ISJEELogHttpTraceEnable=false

(3) Restarting the CEP Service

To reflect your changes, execute cepstopserv and then cepstartserv to restart the CEP service for the log output.

Example

Restarting the CEP service

# cepstopserv <ENTER>
(...)
# cepstartserv <ENTER>
(...)

(4) Starting the CEP Engine

Execute cepstarteng to start the CEP engine.

Example

Executing cepstarteng

# cepstarteng -e CepEngine <ENTER>
Command cepstarteng executed successfully.