Top
Interstage Big Data Complex Event Processing Server V1.0.0 User's Guide
Interstage

6.1.6 Typical Operation Tasks

This section explains the following operation tasks performed after the CEP engine is started:

6.1.6.1 Displaying the Operation Status of the CEP Service

Execute cepdispserv as a superuser to display the status of the CEP service.

This allows you to check that the CEP service is running normally at any time, such as immediately after operation starts or during peak business hours. Execute cepstartserv as a superuser to start any service that is not running.

Example

Using cepdispserv to check the operation status of the CEP service

Check that the content below is output (nnnn indicates the process number, CepEngine indicates the created CEP engine name):

# cepdispserv<ENTER>
(...)
Interstage Java EE DAS started

(...)
Interstage Java EE Node Agent started

(...)
CEPAgentIJServerCluster running

CepEngine_flt not running
CepEngine_cep not running
(...)
Status : Running

(...)
jsvc (pid nnnn nnnn) is running...
(...)
pg_ctl: server is running (PID: nnnn)
(...)
Command cepdispserv executed successfully.

Information

The CEP service entity

A CEP service is made up of multiple processes. Refer to "8.5 cepdispserv" for details.

6.1.6.2 Displaying the Operation Status of the CEP Engine

Execute cepdispeng to display the status of the CEP engine.

This allows you to check that the CEP engine is running normally at any time, such as immediately after operation starts or during peak business hours (refer to Chapter 5, "Errors during Operation" in Troubleshooting if the CEP engine has ABNORMAL or STOP status when it should be running).

A general user permission can be used to execute this command.

Example

Checking the operation status of the CEP engine

When checking the operation status of a CEP engine (CepEngine) - information such as the number of input and output events is also displayed.

$ cepdispeng -e CepEngine<ENTER>
engineId :CepEngine

(...)
status_filter :
RUN
status_cep :
RUN
inEvent_filter :100

inEvent_cep :100

outEvent_filter :100

outEvent_cep :100

(...)
Command cepdispeng executed successfully.

6.1.6.3 Monitoring Abnormalities Using Logs

When an abnormality occurs in the CEP service and the CEP engine, a message will be output to the system log and to the engine logs.

The system log and the engine logs are used differently. For example, in addition to the messages reporting abnormalities, the engine logs also contained more detailed information on CEP engine operation, such as records sent using the SOAP listener and debug information. Therefore, the usual abnormality monitoring monitors only the system log and, when an abnormality occurs, references the engine logs to analyze its cause.

When an abnormality occurs, respond on the basis of the messages output to each log. Refer to Messages for information on specific responses.

The details of log output are explained as follows:

Output destinations of the system log and the engine logs

The output destinations of the system log and the engine logs are shown below.

One CEP engine outputs two engine logs - one contains output relating to input adapter and high-speed filter processing, and the other contains output relating to complex event processing and output adapter processing. The engine logs have no predetermined format.

System log
/var/log/messages
Engine log for the high-speed filter
/var/opt/FJSVcep/cep/flt/logs/EngineLog/CEPengineName/engine.log
Engine log for complex event processing
/var/opt/FJSVcep/cep/cep/logs/EngineLog/CEPengineName/engine.log

The engine logs of the high-speed filter and complex event processing undergo rotation as follows:

Note

Engine logs older than 9 generations are automatically deleted. If they are required, periodically store them in a separate location.

Message format

The format of the messages output by this product is shown below.

Refer to Section 1.1, "Message Format" in Messages for details.

CSPF_CEP: errorType: messageNumber: messageText

Example

Displaying a message

An example message is shown below.

CSPF_CEP: ERROR: cep10108e: Event type is not found. EngineId=CepEngine, eventType=EVENT01

6.1.6.4 Checking the Resource Usage of the CEP Engine

The resource usage of the CEP engine (such as the heap memory usage of Java VM) is recorded in the resource logs. The logs are output in CSV format and therefore can be analyzed using a tool such as Excel, in order to detect resource excesses or deficiencies in the CEP engine.

The details of the resource logs are explained as follows:

Output destinations of the resource logs

Two resource logs are output for each CEP engine - one contains output relating to input adapter and high-speed filter processing, and the other contains output relating to complex event processing and output adapter processing.

The output destination of each resource log is shown below.

Resource log for the high-speed filter
/var/opt/FJSVcep/cep/flt/logs/ResourceLog/CEPengineName/resource.log
Resource log for complex event processing
/var/opt/FJSVcep/cep/cep/logs/ResourceLog/CEPengineName/resource.log

The resource logs of the high-speed filter and complex event processing undergo rotation as follows:

Note

Resource logs older than 13 generations are automatically deleted. If they are required, periodically store them in a separate location.

Format of the resource log for the high-speed filter

This section explains the format of the resource log for the high-speed filter.

The format is shown below:

time,resourceID,jheapNewUsed,jheapNewFree,jheapNewTotal,jheapOldUsed,jheapOldFree,
jheapOldTotal,jheapNewPlusOldTotal,jheapPermUsed,jheapPermFree,jheapPermTotal,VSZ,
numOfInEvents,numOfOutEvents,numOfLogs,numOfRulesDeployed,numOfSocketConnections

Please note that we added newlines in the format above for readability only - the actual log does not have a newline.

The table below explains each item:

Item name

Content

time

Output date and time of the resource log. The format is as follows:

yyyy-MM-dd HH:mm:ss
  • yyyy: Year

  • MM: Month

  • dd: Day

  • HH: Hour

  • mm: Minute

  • ss: Second

resourceID

Resource ID. The format is as follows:

CSPF_CEP_hostName_engineID_flt

jheapNewUsed

Java VM heap memory usage (new generation area) for the high-speed filter (unit: bytes) (*1)

jheapNewFree

Java VM heap free memory (new generation area) for the high-speed filter. (unit: bytes) (*1)

jheapNewTotal

Java VM heap memory (new generation area) for the high-speed filter. (unit: bytes) (*1)

jheapOldUsed

Java VM heap memory usage (old generation area) for the high-speed filter. (unit: bytes) (*1)

jheapOldFree

Java VM heap free memory (old generation area) for the high-speed filter. (unit: bytes) (*1)

jheapOldTotal

Java VM heap memory (old generation area) for the high-speed filter. (unit: bytes) (*1)

jheapNewPlusOldTotal

Java VM heap memory (new generation area + old generation area) for the high-speed filter. (unit: bytes) (*1)

jheapPermUsed

Java VM heap memory usage (permanent generation area) for the high-speed filter. (unit: bytes) (*1)

jheapPermFree

Java VM heap free memory (permanent generation area) for the high-speed filter. (unit: bytes) (*1)

jheapPermTotal

Java VM heap memory (permanent generation area) for the high-speed filter. (unit: bytes) (*1)

VSZ

Process memory usage for the high-speed filter. (unit: KB) (*1)

numOfInEvents

Number of events input from devices on which events are occurring. (*1)

numOfOutEvents

Number of events sent to the engine on the complex event processing side. (*1)

numOfLogs

Number of logs in the input adapter. (*1)

numOfRulesDeployed

Number of rules deployed to the high-speed filter (number of high-speed filter statements). (*1)

numOfSocketConnections

Number of simultaneous socket connections. (*1)

*1: If the CEP engine status is not RUN (running normally), an empty string ("") will be output.

Example

Content of the resource log (high-speed filter)

2012-08-01 14:00:04,CSPF_CEP_cepsv_CepEngine_flt,30130928,9125136,39256064,39224768,
475888192,515112960,554369024,68207736,17292168,85499904,3670240,0,0,0,2,0

Please note that we added a newline in the example above for readability only - the actual log does not have a newline.

Format of the resource log for complex event processing

This section explains the format of the resource log for complex event processing.

The format is shown below:

time,resourceID,jheapNewUsed,jheapNewFree,jheapNewTotal,jheapOldUsed,jheapOldFree,
jheapOldTotal,jheapNewPlusOldTotal,jheapPermUsed,jheapPermFree,jheapPermTotal,VSZ,
reserve1,reserve2,reserve3,reserve4,reserve5,reserve6,numOfInEvents,numOfOutEvents,
numOfLogs,numOfRulesDeployed,numOfListenersDeployed

Please note that we added newlines in the format above for readability only - the actual log does not have a newline.

The table below explains each item:

Item name

Content

time

Output date and time of the resource log. The format is as follows:

yyyy-MM-dd HH:mm:ss
  • yyyy: Year

  • MM: Month

  • dd: Day

  • HH: Hour

  • mm: Minute

  • sis: Second

resourceID

Resource ID. The format is as follows:

CSPF_CEP_hostName_engineID_cep

jheapNewUsed

Java VM heap memory usage (new generation area) for complex event processing. (unit: bytes) (*1)

jheapNewFree

Java VM heap free memory (new generation area) for complex event processing. (unit: bytes) (*1)

jheapNewTotal

Java VM heap memory (new generation area) for complex event processing. (unit: bytes) (*1)

jheapOldUsed

Java VM heap memory usage (old generation area) for complex event processing. (unit: bytes) (*1)

jheapOldFree

Java VM heap free memory (old generation area) for complex event processing. (unit: bytes) (*1)

jheapOldTotal

Java VM heap memory (old generation area) for complex event processing. (unit: bytes) (*1)

jheapNewPlusOldTotal

Java VM heap memory (new generation area + old generation area) for complex event processing. (unit: bytes) (*1)

jheapPermUsed

Java VM heap memory usage (permanent generation area) for complex event processing. (unit: bytes) (*1)

jheapPermFree

Java VM heap free memory (permanent generation area) for complex event processing. (unit: bytes) (*1)

jheapPermTotal

Java VM heap memory (permanent generation area) for complex event processing. (unit: bytes) (*1)

VSZ

Process memory usage for complex event processing. (Unit: KB) (*1)

reserve1

Reserved areas. Empty strings ("") will be output for all of these.

reserve2

reserve3

reserve4

reserve5

reserve6

numOfInEvents

Number of events input from high-speed filter processing. (*1)

numOfOutEvents

Number of events sent to user-developed Web services. (*1)

numOfLogs

Number of logs in complex event processing. (*1)

numOfRulesDeployed

Number of rules deployed to complex event processing (number of complex event processing statements). (*1)

numOfListenersDeployed

Number of deployed SOAP listener definitions. (*1)

*1: If the CEP engine status is not RUN (running normally), an empty string ("") will be output.

Example

Content of the resource log (complex event processing)

2012-08-01 14:00:04,CSPF_CEP_cepsv_CepEngine_cep,14042608,25213456,39256064,41802024,
473310936,515112960,554369024,76981880,8481160,85463040,3385020,,,,,,,0,0,0,6,0

Please note that we added a newline in the example above for readability only - the actual log does not have a newline.

Collection interval for resource usage

The resource usage is collected in 10-minutes intervals and is output to the resource log.

This process is performed by using cron to periodically call an obtain process.

The cron setting is set in the following file. The engine execution user is the user name specified at installation:

/var/spool/cron/engineExecutionUser