Jump to: navigation, search

HTTP Monitoring Interface

Starting with version 8.1.102.13, SIP Server provides the ability to monitor various operational statistics for its internal modules and statistics relating to trunks.

Starting with version 8.1.103.25, SIP Server adds the ability to monitor statistics related to SIP Feature Server interactions.

Starting with version 8.1.103.35, SIP Server adds the ability to monitor statistics related to Extended Services (XS) components.

Starting with version 8.1.103.72, SIP Server adds the following statistics in separate tables for Trunk, Softswitch, MSML, and Trunk Group devices: error statistics, the total number of calls created on the device, the number of out-of-service detection instances per device, and location matching instances.

Starting with version 8.1.103.95, SIP Server adds monitoring of state and quantitative statistics for T-Library client connections of the following SIP Server threads: Session Controller (the main T-Server thread), T-Controller, Interaction Proxy, and Smart Proxy. See SIP Server threads statistics for details.

This topic covers the following:

SIP Server Statistics

SIP Server collects statistics for the following internal modules:

Trunk, Softswitch, MSML, and Trunk Group Statistics

SIP Server collects the following statistics (sipTrunkStatistics) for each configured Trunk, Softswitch, MSML, and Trunk Group DNs:

  • Current statistics (real-time)
    • Number of calls currently established via this trunk (sum of incoming and outgoing calls)
    • Current call rate (average number of calls per second, both incoming and outgoing). Averaging interval is specified by the Application-level option operational-stat-timeout (in seconds, default value is 10 seconds).
    • Capacity (maximum number of calls allowed on a trunk, both incoming and outgoing)
    • Capacity group (to which a trunk belongs)
    • In Service status
    • Error code counters: 4xx, 5xx, 6xx. Error codes that are received by SIP Server for the device since SIP Server started.
    • Number of calls created since SIP Server started. It is a summary of incoming and outgoing calls created at the device. It includes failed to establish calls due to an error response or timeout.
    • Number of out-of-service detection instances since SIP Server started.
    • Statistics for Trunk, MSML, and Softswitch devices only: If location matching is enabled, devices that don't satisfy the location matching configuration are not displayed. Location matching is configured by the enable-strict-location-match, overflow-location-map, and find-trunk-by-location configuration options.
  • Summary statistics (over a period of time, such as an hour or a day)
    • Peak number of calls
    • Peak call rate
    • Call attempts (total number of new calls)
    • Total number of released calls
    • Summary period start time
    • Summary period end time

SIP Server collects the following statistics (sipTrunkStatistics) for each configured Capacity Group:

  • Current statistics
    • Current number of calls (combined number for all the trunks in a group)
    • Current call rate (combined rate for all the trunks in a group)
    • Capacity
  • Summary statistics
    • Peak number of calls
    • Peak call rate
    • Call attempts
    • Total number of released calls
    • Summary period start time
    • Summary period end time
    • Number of calls created since SIP Server started

The period of summary statistics calculation is configurable with the Application-level option summary-stat-timeout.

How to Monitor Statistics

There are two possible ways to monitor the collected statistics:

Monitoring Statistics via HTTP interface

To enable the HTTP interface, set the http-port option to a valid and unoccupied port number in the range of 1024-65535 (values lower than 1024 are system reserved ports). Only the HTTP interface is available on the configured port (HTTPS is not available).

To get the statistics data, the following URL must be retrieved with the HTTP GET request:
http://<SIP Server IP address>:<configured HTTP port> (for example, http://192.168.0.1:8088)

Depending on the path used in the URL, the statistics page can be provided in the HTML or XML format:

  • To get the statistics in the HTML format, use an empty path ("" or "/") or path /server.
    For example: http://192.168.0.1:8088 or http://192.168.0.1:8088/server.
    The above URL returns a root statistics page with the list of statistics for SIP Server internal modules, with each list item being a link to the statistic data for that module.
  • To get the statistics in the XML format, use path /serverx.
    For example: http://192.168.0.1:8088/serverx.
    The above URL returns a root statistics page with the full statistics dump, including data for each SIP Server internal module.

To get the statistics page for one module, add the URL parameter with the module name. For example, to get the Trunk Statistics page in the HTML format, the following URL must be used: http://192.168.0.1:8088/server?sipTrunkStatistics.

Trunk Statistics Examples

  • In the HTML format:

[+] Show HTML source

Example of statistics displayed in the HTML format in a browser:

Trunk Statistics in HTML Format

  • In the XML format:

[+] Show XML source

Monitoring Statistics via Log File

SIP Server always creates a dedicated log file for a statistics output despite of the values set in the log options. The name of the file contains suffix 1536 (for example, server1-1536.20160201_195851_685.log).

Statistics are written to the log file periodically, with a period specified by the Application-level option operational-stat-timeout (default value is 10 seconds).

Statistics log format

A statistics log file uses the following line format for statistics output:

<Timestamp in ISO 8601 format> <Statistics module name> <Parameter>=<Value> [<Parameter>=<Value>]

For example, for the Call Manager module, the output for the "Number of devices" statistics is as follows:

2016-01-01T12:00:01.001 sipCallManager NDEVICES=150

For more sophisticated Trunk statistics, the output is as follows:

2016-01-01T12:00:01.001 sipTrunkStatistics TRUNK=21001 CURRENT_CALLS=100 CURRENT_CALL_RATE=15 PEAK_CALLS=150 PEAK_CALL_RATE=20 CALL_ATTEMPTS=5000 RELEASED_CALLS=4900 SUMMARY_START=2016-01-01T12:00:01 SUMMARY_END=2016-01-01T13:00:01 CAPACITY=500 CAPACITY_GROUP=MyTrunks IN_SERVICE=Yes

SIP Server Statistics Details

Each SIP Server internal module has its own statistics section. Each section has the identifier that is used in the log and XML output to distinguish one section from another. This identifier is also used as a URL parameter if a user wishes to get an HTML/XML page with only one statistics section.

The statistics for each SIP Server module is described below. Each statistic record is described in a table that has the following columns:

  • ID—The ID that is used in the log and XML output.
  • Description—The general description of the statistic record. It is also used in the HTML output.
  • Comments—Additional information if the description of the record is not self-explanatory.

General SIP Server Statistics

Section ID: sipServer

ID

Description

Comments

SIPS_PROCESS_ID

Process identifier

Identifier that is assigned by an Operating System to a SIP Server process

SIPS_MEMORY_USAGE

Memory usage

In bytes

SIPS_CPU_USAGE

CPU usage

In percent

PROCESS_INFO_TIME

Process info time

Time range on which statistics are gathered

NAME

Name

Name of the SIP Server Application in the Configuration Layer

BIT

Bit data model

32 or 64 bit

PLATFORM

Platform

 

SERVER_VERSION

Server version

 

COMPILED_DATE

Compiled date

 

XS_CPU_USAGE CPU usage Extended Service CPU usage on a singe CPU core. Applies only to SIP Cluster deployments starting with SIP Server release 8.1.103.29.
XS_REQUESTS_RATE Requests rate Rate of requests processed by the Extended Services component. Applies only to SIP Cluster deployments starting with SIP Server release 8.1.103.29.

SIP Stack Statistics

Section ID: sipStackObjects

ID Description Comments
DIALOG_CREATED SIP Dialogs created A cumulative metric that is reset to zero on restart.
DIALOG_DELETED SIP Dialogs deleted A cumulative metric that is reset to zero on restart.
MESSAGES_CREATED SIP Messages created A cumulative metric that is reset to zero on restart.
MESSAGES_DELETED SIP Messages deleted A cumulative metric that is reset to zero on restart.
CLIENT_TRANSACTION_CREATED SIP Client transactions created A cumulative metric that is reset to zero on restart.
CLIENT_TRANSACTION_DELETED SIP Client transactions deleted A cumulative metric that is reset to zero on restart.
SERVER_TRANSACTION_CREATED SIP Server transactions created A cumulative metric that is reset to zero on restart.
SERVER_TRANSACTION_DELETED SIP Server transactions deleted A cumulative metric that is reset to zero on restart.
TRANSPORT_CREATED Transports created A cumulative metric that is reset to zero on restart.
TRANSPORT_DELETED Transports deleted A cumulative metric that is reset to zero on restart.
DATASENT Data sent In bytes. If a value is not increased, it might be used as indication of the backup mode of SIP Server.
DATARECEIVED Data received In bytes. If a value is not increased, it might be used as indication of the backup mode of SIP Server.
RESPONSE_TIME_LESS20 Response time less than 20 ms Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_20TO50 Response time 20 to 50 ms Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_50TO100 Response time 50 to 100 ms Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_100TO200 Response time 100 to 200 ms Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_200TO500 Response time 200 to 500 ms Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_500TO1SEC Response time 500 ms to 1 sec Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_1TO5SEC Response time 1 to 5 sec Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_5TO10SEC Response time 5 to 10 sec Response time to SIP messages sent by SIP Server.
RESPONSE_TIME_MORE10SEC Response time more than 10 sec Significant increase of the value might indicate network problems or a SIP Server overload condition.

Call and Client Statistics

Section ID: sipCallManager

ID

Description

Comments

CM_THREAD_ID

Thread ID

CM_CPU_USAGE

CPU usage of call manager thread

In percent.

NCALLSCREATED

Number of calls created

A cumulative metric that is reset to zero on restart.

NDIALOGS

Number of dialogs

Current active SIP dialogs.

NCALLS

Number of calls

Current active calls.

NPARTIES

Number of parties

Current active parties. If the number of calls is zero, a nonzero value of the number of parties might indicate a leak of resources which might lead to an abnormal behavior of SIP Server.

HADATASENT

HA sync data sent

In bytes.

HADATARECEIVED

HA sync data received

In bytes.

NROUTINGTIMEOUTS

Number of routing timeouts

Increases when the routing timer expires (specified by the option router-timeout). Significant increase of the value might indicate issues in the routing strategies or router overload.

NLOGGEDAGENTS

Number of logged on agents

 

NREGISTEREDDNS

Number of registered DNs

Number of registered DNs by using a TRegisterAddress request.

NTLIBCLIENTS

Number of T-Library clients

Number of T-Library clients currently connected to SIP Server.

NCALLSABANDONED

Number of abandoned calls

Number of calls released by a caller before the call was answered.

NCALLRECORDINGFAILED

Number of failed call recording sessions

Significant increase of the value might indicate problems with the recording services.

NSIPREGISTEREDEP

Number of active SIP registrations

 

NSIPEXPIREDREGS

Number of expired SIP registrations

Number of SIP registrations that were expired since SIP Server startup. Significant increase of the value might indicate problems with the network.

NCALLSOVRLREJECTED

Number of rejected calls due to overload control

Related to the static overload control feature.

NMSMLLOCATIONFAILED

Number of MSML location resolution failed

Significant increase of the value might indicate problems with configuration of geo-location of MSML services.

ISCC Statistics

Section ID: sipTSCP

ID

Description

Comments

ISCC_ACTIVE_ORIG_TRANSACTIONS

ISCC active orig transactions

 

ISCC_ACTIVE_DEST_TRANSACTIONS

ISCC active destination transactions

 

ISCC_SUCCEEDED_ORIG_TRANSACTIONS

ISCC succeeded origination transactions

 

ISCC_SUCCEEDED_DEST_TRANSACTIONS

ISCC succeeded destination transactions

 

ISCC_FAILED_ORIG_TRANSACTIONS

ISCC failed origination transactions

Significant increase of the value might indicate issues in the communication between sites in a multisite environment.

ISCC_FAILED_DEST_TRANSACTIONS

ISCC failed destination transactions

Significant increase of the value might indicate issues in the communication between sites in a multisite environment.

T-Library Statistics

Section ID: sipSessionController

ID

Description

Comments

MAIN_THREAD_ID

Thread ID

 

MAIN_CPU_USAGE

CPU usage

In percent.

NAPPLY_TREATMENTS

Number of Apply Treatments

Number of processed TApplyTreatment requests. A cumulative metric that is reset to zero on restart.

URS_RESPONSE_LESS50

URS Response less than 50 ms

URS response time means the time passed between the moment the call is queued on a Routing Point (indicated by EventQueued) and the moment the call is routed to a destination, including the default destination in case of URS failure/timeout (indicated by EventRouteUsed). Long response times might indicate problems with the network or URS overload. Response times corresponding to the router-timeout option value most likely indicate a URS failure/timeout (e.g. as a result of incorrect routing strategy).

URS_RESPONSE_50TO100

URS Response 50 to 100 ms

 

URS_RESPONSE_100TO200

URS Response 100 to 200 ms

 

URS_RESPONSE_200TO500

URS Response 200 to 500 ms

 

URS_RESPONSE_500TO1SEC

URS Response 500 ms to 1sec

 

URS_RESPONSE_1TO5SEC

URS Response 1 to 5 sec

 

URS_RESPONSE_MORE5SEC

URS Response more than 5 sec

NUSER_DATA_UPDATES

Number of User Data updates

Number of processed requests related to UserData (TAttachUserData, TUpdateUserData, TDeletePair, TDeleteUserData). A cumulative metric that is reset to zero on restart.

NTREQUESTS

Number of T-Requests

Number of all processed T-Library requests (of any type). A cumulative metric that is reset to zero on restart.

Out-Of-Service Detection Statistics

Section ID: sipServiceChecker

ID

Description

Comments

SRVC_THREAD_ID

Thread ID

 

SRVC_CPU_USAGE

CPU usage

 

SRVC_NOOS_DEVICES

Number Out Of Service devices

Significant increase of the value might indicate problems with the network

SIP Server threads statistics

To enable statistics for T-Library client connections of the SIP Server threads, set the t-library-stats-enabled configuration option to true. SIP Server provides state and quantitative statistics for the following SIP Server threads:

  • Session Controller
  • T-Controller
  • Interaction Proxy
  • Smart Proxy

Table name: tLibClientsStatistics
Table ID: tLibClientsPage

For each thread, SIP Server displays the client connection statistics and error statistics. See also a limitation for this feature.

Client connection statistics

Statistics tables for client connections have the following names and IDs per SIP Server thread:

  • Session Controller: SC_clientsStatistics and id='SC_clientsDataTable'
  • T-Controller: TC_clientsStatistics and id='TC_clientsDataTable'
  • Interaction Proxy: IP_clientsStatistics and id='IP_clientsDataTable'
  • Smart Proxy: SP_clientsStatistics and id='SP_clientsDataTable'
ID Description Comment
CLIENT Client The name of the connected client.
CURRENT_CONN_STATE Client Connection State The connection state of the client either 1 (connected) or 0 (disconnected).
NSTATE_DISCONNECTED Number of client disconnects The number of client disconnections since SIP Server is started.
NCLIENT_REQUESTS Accumulated number of requests The accumulated number of requests sent by the client since SIP Server is started.
NCLIENT_EVENTS Accumulated number of events (errors including) The accumulated number of events sent by SIP Server to the client since SIP Server is started.
NCLIENT_ERROR_EVENTS Accumulated number of errors The accumulated number of errors among events sent by SIP Server to the client since SIP Server is started.
CLIENT_OUTPUT_QUEUE Output queue size (bytes) The output queue size (the connection output buffer), in bytes.
CLIENT_DATA_RX_BYTES Accumulated incoming bytes The accumulated incoming bytes received by SIP Server from the client since SIP Server is started.
CLIENT_DATA_TX_BYTES Accumulated outgoing bytes The accumulated outgoing bytes sent by SIP Server to the application since SIP Server is started.

Error statistics

Error statistics tables contain embedded tables. There are as many tables as different types of errors that are received by a particular thread since SIP Server is started. The error statistics tables have the following names and IDs per SIP Cluster thread:

  • Session Controller: SC_errorsStatistics and id='SC_errorsDataTable'
  • T-Controller: TC_errorsStatistics and id='TC_errorsDataTable'
  • Interaction Proxy: IP_errorsStatistics and id='IP_errorsDataTable'
  • Smart Proxy: SP_errorsStatistics and id='SP_errorsDataTable'
ID Description Comment
ERROR_CODE ErrorCode The digital error code.
ERROR_TEXT Error Meaning The error description.
N_ERRORS Accumulated number of errors The accumulated number of errors of this particular type thread that are received since SIP Server is started.

Example of Session Controller client and error statistics in the XML format:

[+] Show XML source

SIP Feature Server Statistics

SIP Server collects the following statistics related to SIP Feature Server (sipFeatureServer) interactions:

ID Description
FS_STATE The Current service state. Values: 0 is out of service, 1 is in service.
FS_QUEUE_SIZE Current number of XS requests in a queue (requests that have not been sent to Feature Server).
FS_AVERAGE_QUEUE_TIME Average request in queue time in milliseconds during the period of statistic's summary.
FS_CONNECTIONS Number of connections for each URL.
Statistics for each configured URL
FS_URL URL of Feature Server.
FS_ACTIVE_CONNECTIONS Current number of active connections.
FS_REQUEST_RATE Requests rate during the period of statistic's summary (request/sec).
FS_TIMEOUTS Number of timeouts for requests (sent or in queue) during the period of statistic's summary.
FS_400_ERRORS Number of 400 error responses during the period of statistic's summary.
FS_404_ERRORS Number of 404 error responses during the period of statistic's summary.
FS_4XX_ERRORS Number of 4xx error responses during the period of statistic's summary.
FS_500_ERRORS Number of 500 error responses during the period of statistic's summary.
FS_501_ERRORS Number of 501 error responses during the period of statistic's summary.
FS_5XX_ERRORS Number of 5xx error responses during the period of statistic's summary.
FS_AVERAGE_RESPONSE_LATENCY Average response latency during the period of statistic's summary (in milliseconds).

SIP Feature Server Statistics Examples

  • In the HTML format:

[+] Show HTML source

The code above displays these results in a browser:

Sips-httpmonitor-fs.png
  • The URL-related statistic in the HTML format:

[+] Show HTML source

The code above displays these results in a browser:

Sips-httpmonitor-fs-url.png
  • The URL-related statistic in the XML format:

[+] Show XML source

Configuration Options

http-port

Section: [TServer]
Default Value: 0
Valid Values: 0, 1024-65535
Changes Take Effect: After SIP Server restart

Specifies the HTTP interface port number. When set to 0, the HTTP server is disabled. The port numbers in the range of 1 through 1023 are the system ports and must not be used.

operational-stat-timeout

Section: [TServer]
Default Value: 10
Valid Values: 3-65535
Changes Take Effect: After SIP Server restart

Specifies how often, in seconds, a local LCA is queried for system information such as CPU and memory usage. This information is then written into the SIP Server Operational Information log as defined in the SIP Server configuration.

summary-stat-timeout

Section: [TServer]
Default Value: 60
Valid Values: Integer value 1-65535
Changes Take Effect: After SIP Server restart

Specifies how often, in minutes, the summary statistics are calculated.

t-library-stats-enabled

Section: [TServer]
Default Value: false
Valid Values: true, false
Changes Take Effect: After SIP Server restart

When set to true, SIP Server collects T-Library client statistics and embeds them inside HTTP monitoring statistics. When set to false (the default, this feature is disabled.

Feature Limitations

  • The CSS style for the HTML statistics page is hardcoded.
  • For the SIP Server threads statistics feature, each client connected to SIP Server must have a unique name. Two or more clients with the same name must not connect to SIP Server simultaneously.
This page was last edited on October 5, 2020, at 21:25.
blog comments powered by Disqus