Jump to: navigation, search

Elasticsearch Connector

Starting with Release 8.1.400.40, Orchestration Server uses Elasticsearch 2.x to store data, such as operational and performance data. The data may then be used with Kibana (default) or a custom visualization tool to monitor Orchestration Server performance and routing session processing in near real time.

ORS814Elastic5.png


Starting with release 8.1.400.64, ORS supports Elasticsearch releases 5.x, starting from 5.3.0. There have been significant changes between major Elasticsearch releases, in relation to APIs and settings. ORS functionality has been enhanced such that it dynamically adjusts in accordance with the version of the Elasticsearch node that it operates with.

General Setup/Configuration

  1. Download and install Elasticsearch.
  2. Configure Elasticsearch for Orchestration.
  3. Configure the ORS Elasticsearch options for session, performance, and node reporting. You do not have to configure each type of reporting (although you can); you can configure only the reporting options that you wish to use.
  4. Configure Security options.

Configuring Elasticsearch for Orchestration

[+] Configuring Elasticsearch for Orchestration

Configuring ORS Options for Elasticsearch

Session Reporting

ORS stores SCXML session data into Elasticsearch where documents are created in a “session” daily index. The Orchestration session ID is used as index key. ORS saves information about session states if _es_store=true in the state description in the strategy. Example: <state id="routing" _es_store=”true”>.

[+] Session Reporting Document

Notes:

  • If "<state name>" contains dot symbols (.), the state name stored in Elasticsearch will contain underscore symbols instead of dots.
  • Starting with release 8.1.400.43, ORS stores two new Session Reporting fields into Elasticsearch: ani and duration.

Session reporting uses the following options:

ors-es-session-report

Option section: elasticsearch

Configuration object: ORS Application object

Default value: false

Valid values: true, false

Value changes: Take effect immediately upon notification.

This option specifies if session reporting is enabled or disabled. When set to true, session reporting is enabled. When set to false, session reporting is disabled.

If an ors-es-session-report value is changed from false to true in the middle of a session, ORS will send an update request for a non-existing session document. Elasticsearch will respond with a "document missing" exception, which ORS will ignore.

ors-es-bulk-write-period

Option section: elasticsearch

Configuration object: ORS Application object

Default value: 5 seconds

Valid values: An Integer value in the range of 5 to 120 seconds.

Value changes: Take effect immediately upon notification.

Used for both session and performance reporting. The word "bulk" refers to the possibility of having ORS accumulate a configurable amount reporting data and then sending the data to Elasticsearch instead of sending one reporting request at a time.

This option specifies the maximum time interval between the sending of two bulk requests to Elasticsearch. The bulk mechanism is based on an internal buffer that stores all requests that should be sent to Elasticsearch. In cases where there are no buffered requests, the actual time between two bulk requests could exceed the ors-es-bulk-write-period. However, when there is a steady stream of the requests to be handled by single bulk operation, the actual time between two bulk requests will be equal to the time defined by this option.

ors-es-bulk-max-size

Option section: elasticsearch

Configuration object: ORS Application object

Default value: 10000

Valid values: Integer value in range from 1000 to 10000000

Value changes: Take effect immediately upon notification.

Used for both session and performance reporting. This option specifies the maximum number of actions inside a single bulk request. If reporting is enabled, ORS stores all requests in its internal bulk requests cache. This option can be set to control cache overgrowth. If the maximum number of actions in the cache is reached, ORS removes the oldest actions and inserts the newest.

Note: Genesys recommends clustering Elasticsearch nodes with several master nodes to prevent a potential loss of reporting data should an extended node disconnect occur (unless all available Master nodes in an Elasticsearch cluster are excluded).

Performance Reporting

If performance monitoring is enabled, ORS stores performance data into Elasticsearch where documents are created in a “performance” daily index.

[+] Performance Reporting Document

Performance reporting uses the following options:

ors-es-perfsnapshot-report

Option section: elasticsearch

Configuration object: ORS Application object

Default value: false

Valid values: true, false

Value changes: Take effect immediately upon notification.

This option specifies if performance reporting is enabled via Elasticsearch.

  • If set to true, performance reporting via the Elasticsearch is enabled.
  • If set to false), performance reporting via Elasticsearch is disabled.

ors-es-bulk-write-period

Used for both session and performance reporting. Option ors-es-bulk-write-period allows you to submit session or performance reporting to Elasticsearch via a "bulk" mode. For information on this option, see the description in Session Reporting.

Node Reporting

ORS stores node information into Elasticsearch where a document is created in a “nodes” index. The document is created during node startup and updated when ORS switches to primary mode.

[+] Node Reporting Document

Node reporting uses the following options:

ors-es-node-info-report

Option section: elasticsearch

Configuration object: ORS Application object

Default value: false

Valid values: true, false

Value changes: Take effect immediately upon notification.

This option specifies if node reporting via the Elasticsearch engine is enabled. If, upon ORS startup, ors-es-node-info-report has a value of false, and the value is subsequently changed to true, the report information will not be sent.

ors-es-nodes

Option section: elasticsearch

Configuration object: ORS Application object

Default value: <empty string>

Valid values: String containing a semicolon-separated list of http(s)://host:port of ElasticSearch servers.

Value changes: Take effect after restart.

This option is used for connectivity to Elasticsearch. It specifies the addresses of Elasticsearch nodes. Configure each ORS to work with an Elasticsearch node. For valid values:

  • If the node is specified as http://<ES host>:<port>, then a non-encrypted connection will be used for communication with Elasticsearch.
  • If the node is specified as https://<ES host>:<port>, then an encrypted connection will be used for communication with Elasticsearch.

Reporting for ORS Elasticsearch Clusters

ORS nodes can send report requests to either a single Elasticsearch node or to an Elasticsearch cluster. It does not matter whether Elasticsearch is installed in single mode or in cluster mode. ORS communicates with Elasticsearch using the host:port specified in option ors_es_nodes.

  • If Elasticsearch is configured as a single node setup, then ORS sends report requests directly to this node.
  • In the case of an Elasticsearch cluster, each ORS Application can communicate with a specific node of the cluster, or all ORS Applications can communicate with a dedicated load-balancing node.
  • The dedicated load-balancing node is the Elasticsearch node that is configured to receive documents and distribute them to the data nodes of the cluster.
  • Upon startup, ORS connects to the first node in the ors-es-nodes list. If a disconnect from this Elasticsearch node occurs, ORS will try to connect to the next Elasticsearch node specified in the list.

ORS uses the following options for connectivity:

ors-es-reconnect-timeout

Option section: elasticsearch

Configuration object: ORS Application object

Default value: 5 seconds

Valid value: 1 to 20 seconds.

Valid changes: Take effect immediately upon notification.

This option specifies the timeout period that ORS uses when trying to reconnect to an Elasticsearch node. If a connection to Elasticsearch is lost, then ORS will try to reconnect to the next Elasticsearch node in the ors-es-nodes list.

ors-es-node-exclude-timeout

Option section: elasticsearch

Configuration object: ORS Application object

Default value: 600 seconds

Valid values: Integer value in range from 5 to 86400 seconds.

Valid changes: Take effect immediately upon notification.

After successfully connecting to an Elasticsearch node, ORS checks the node status. In case of a red status, ORS disconnects from the node and excludes the node from the reconnection procedure for the time period defined by the option ors-es-node-exclude-timeout.

Security Configuration

ORS communicates with Elastic search via the Elasticsearch REST API. The following security features are supported:

  • TLS/SSL—ORS connects to Elasticsearch using SSL or TLS. The connection is supported via proprietary security layer in Genesys Framework libraries. For details, see the Genesys Security Deployment Guide, chapter "Protection of Data in Transit".

To enable a TLS/SSL connection to an Elasticsearch node, specify it in the ors-es-nodes option with the https:// prefix.

  • Authentication (basic)—ORS supports authorization and authentication based on standard HTTP protocol mechanisms.

Configure the following basic authentication options:

username

Option section: elasticsearch

Configuration object: ORS Application object

Default value: <empty string>

Valid value: String value

Valid changes: Take effect immediately upon notification.

This option specifies the username that should be used for basic authentication on the Elasticsearch server.

password

Option section: elasticsearch

Configuration object: ORS Application object

Default value: <empty string>

Valid value: String value

Valid changes: Take effect immediately upon notification.

This option specifies the password that should be used for basic authentication on the Elasticsearch server.

Limitations

  • All ORS Applications that belong to the same ORS cluster must be connected to master nodes of the same Elasticsearch cluster.

Configuring ORS Options for Elasticsearch 5.3

Users can configure the following options as required, if Elasticsearch 5.x is being used. All other existing options are supported for Elasticsearch 5.3 also.

Dynamic Index Settings

Starting from Elasticsearch 5.3, configuration of the index module for each Elasticsearch node in the elasticsearch.yml file is not possible. Settings for the index module can only be applied per-index via the Elasticsearch index or settings API. ORS uses the API to explicitly define the required index module settings for each index it creates.

The following option is available to configure the index module settings:

index-settings-<version>

Option section: elasticsearch

Default value: {"index":{"translog":{"durability":"async"},"refresh_interval":"5s"}}

Valid values: Elasticsearch API request content in the form of a JSON string.

Value changes: After ORS restarts.

This option is to define the Elasticsearch version-specific index settings based on which settings will be applied to the indexes that ORS creates. Each ORS application dynamically updates index settings after any operation that may cause the creation of a new index.

Important
Elasticsearch version in the above option can be specified completely, as in "5.4.0", or partially, as in "5" or "5.3". For example, if it is specified as "5", the index settings from this option’s value will be applied if the version of Elasticsearch starts with "5.".

Index Settings for Custom Indexes

A new attribute, settings, has been added to specify index settings for custom indexes created using the <elasticconnector:createdoc> or <elasticconnector:updatedoc> actions. If not specified, ORS uses the settings value specified in the index-settings-<version> option. If the Elasticsearch version is less than 5.x, this attribute is ignored.

If the <elasticconnector:createdoc> action or the <elasticconnector:updatedoc> action is executed without the settings attribute, the result is the same as executing the action with the value defined by the current Elasticsearch node's index-settings-<version> option.

Important
New index settings are sent for the custom index only if they are different from the previous settings sent for that particular index, except if the Elasticsearch node was disconnected or ORS was restarted.

Orchestration Plugin

Starting with Release 8.1.400.49, ORS installation includes a plugin for a real-time view of active sessions currently being executed. With the Orchestration Plug-in integrated into Kibana you can:

  • Display a list of active sessions.
  • Use different filter criteria to refine the list of active sessions.
  • View session-related information.
  • Terminate an active session (for example, a "stuck" session).

You can filter using the following criteria:

  • time interval (from/to)
  • session age
  • node (one or more)
  • cluster, (one or more)
  • device (routing point or interaction queue)
  • media type (voice, email, chat, and so on)

ORS Application Configuration

In the ORS Application object, Options tab, configure the Elasticsearch options in the elasticsearch section as described above.

Plugin Installation

  • Install Elasticsearch 2.4.1 and Kibana 4.5.0.
  • Install ORS 8.1.400.49.
  • Navigate to elasticserach\kibana-plugins\Orchestration subfolder and unpack the plugin.zip file.
  • Follow the instructions provided in the installation guide PDF to launch Elasticsearch and Kibana.
[+] Plugin Installation Guide
  • Access the Kibana dashboard through your browser. Kibana adds an orchestration icon.

Kibana4.png

  • Select the orchestration icon to bring up the plugin interface. The example below shows a list of active sessions.

width

  • To the left, you can filter by session start time interval, session age, node, cluster, device or media type. You can also save and load filters.
  • To the right, the interface displays search results: the Start Time and Session ID of active sessions. Session ID is the link to the Session information dialog box (shown below), which contains a control to terminate a session.
  • An Auto-refresh on/off button appears at the top right.
  • Forward and back buttons appear for navigating through search results that exceed the page.

Applying a Filter

Below is an example of filtering for sessions within a specific time frame.

  1. Select the Start time filter check box.
  2. Define the from/to time frame using Start time interval button.
  3. Select the Apply filter button. Sessions filtered within this time frame display under the Session ID column.

Retrieving Session Information

Click particular session identifier. A Session information dialog box summarizes the selected session (see example below).

Terminating a Session

Click the Terminate link in the Session Information dialog box. An HTTP request to terminate the session is sent to the corresponding ORS node.

width

Plugin Limitations

The ability to terminate a session via the Orchestration Plugin feature is not supported:

  1. In a deployment using Recovery of Voice Calls Without Persistence.
  2. If an ORS node is restarted.

Elasticsearch Connector Enhancements

Starting with 8.1.400.58, ORS enhances its Elasticsearch real-time reporting capabilities with the addition of the following new features:

  • Explicit mapping of session and performance indexes
  • A new Orchestration SCXML extension, elasticconnector
  • Samples of Kibana dashboards and visualizations

Explicit Mapping

ORS now uses Explicit mapping instead of Dynamic mapping for session and performance indexes via the creation of index templates (tpl_session and tpl_performance). From a performance standpoint, Explicit mapping is more optimal and supports all data types. ORS creates Session and Reporting indexes as soon as it starts and switches to Primary mode.

New SCXML Extension

A new Orchestration SCXML extension, elasticconnector, provides simplified access to Elasticsearch APIs which allows communication with Elasticsearch without the usage of <session:fetch>. You can add new properties into a session document or create custom documents right from a strategy. To use elasticconnector actions, configure the corresponding namespace manually in Composer.

ORS supports the following new Action elements that can be used with Composer's SCXML State block:

[+] createindextemplate
[+] createdoc
[+] updatedoc
[+] deletedoc

Events

The following Events are supported:

[+] Events

Configuration for Elasticsearch

To configure these enhancements in Elasticsearch, edit the elasticsearch.yml file:

  • es.script.engine.groovy.indexed.update: true.

See Configuring Elasticsearch for Orchestration.

Session and Performance Enhancement Limitations

Elasticsearch v 1.7.3 is not supported for these enhancements as it does not support the Date Format that ORS uses.

Kibana Visualization Samples

Starting with Release 8.1.400.58, ORS installation includes samples of Kibana dashboards and visualizations of data stored by ORS Elasticsearch Connector into Elasticsearch. ORS installation brings two sets of samples in folders located in the directory where ORS is installed:

  1. Composer Project sample located in \elasticsearch\kibana_dashboard_sample\composer. The provided Composer project contains SCXML strategies configured to store data into Elasticsearch that will be retrieved and presented in the Kibana visualizations.
  2. Samples of Kibana dashboards and visualizations located in \elasticsearch\kibana_dashboard_sample\kibana.

To load the dashboards/visualizations, execute script \elasticsearch\kibana_dashboard_sample\kibana

  • Unix: ./load.sh -url "http://<elasticsearch host>:9200"
  • Windows: .\load.ps1 -url "http://<elasticsearch host>:9200"

The performance visualizations are based on predefined <alarm-name>(s) configured in the performance-alarms section imported from the ORS Application template.

Click to open expanded views of the samples. Click the "X" to close.

Performance Dashboard
Session Dashboard

Feedback

Comment on this article:

blog comments powered by Disqus
This page was last modified on January 10, 2018, at 06:56.