About Jobs

Genesys Info Mart provides several jobs that you can run when needed, or schedule to run on a periodic basis. These jobs perform routine or special-purpose functions, as follows:

• Routine functions — Genesys Info Mart provides jobs to perform the following routine functions:
  • Extract data from your source databases.
  • Cleanse and transform the data.
  • Load the data into the fact and dimension tables of the Info Mart dimensional model.
  • Optionally, calculate and load aggregated data into the Aggregate tables.
  • Purge old data from the Info Mart database.
  • Maintain calendar dimension tables.
  • Add and delete partitions for partitioned tables.
  • Starting with release 8.5.010, process General Data Protection Regulation (GDPR) "export" or "forget" requests.
• Special-purpose functions — Genesys Info Mart provides jobs to perform the following special-purpose functions:
  • Initialize the Info Mart database.
  • Migrate your existing version 8.x Info Mart database so that it is ready for use by the current 8.x release of Genesys Info Mart.
  • In PostgreSQL deployments, perform supplementary database maintenance.
  • Export data from the Info Mart database to make it available for further import into a data warehouse.

Understanding the jobs

The following table summarizes the jobs that are provided with Genesys Info Mart. Click a job name to link to more information about the job. For more information about how the Genesys Info Mart jobs function, see the part about how Genesys Info Mart works in the Genesys Info Mart 8.1 Deployment Guide. For information about using the jobs to extract and transform data, job interdependencies, and a sample schedule, see Managing and scheduling jobs.

Genesys Info Mart Jobs Summary

Job	Frequency	Notes
Job_InitializeGIM Populates many of the dimension tables in the Info Mart database with fixed information. Adds partitions to partitioned tables for partitioned databases, and automatically updates Interaction Databases (IDBs) for use with Genesys Info Mart.	Once	This job automatically executes once during the first run of Genesys Info Mart after the initial deployment.
Job_ExtractICON Extracts new and changed records from one or more IDBs, and stores those records in Global Interaction Database (GIDB) tables in the Info Mart database.	Intraday, as scheduled
Job_TransformGIM Transforms and loads previously extracted IDB data and data from other data streams into the fact and dimension tables of the Info Mart database.	Intraday, depending on Job_ExtractICON
Job_AggregateGIM Aggregates or re-aggregates the facts based on data that was added or changed during the last transformation job. Stores the data in historical Aggregate tables.	Continuous, within a configured daily time period	This job is available in Genesys Info Mart deployments with the Genesys historical reporting presentation layer—Genesys CX Insights (GCXI) reports—or Reporting and Analytics Aggregates (RAA).
Job_MaintainGIM Maintains the Info Mart database.	Daily, with parts of the job running as configured or as needed
Job_UpdateStats In PostgreSQL deployments, performs supplementary maintenance on the Info Mart database.	Intraday, as scheduled	This job cannot be run from Genesys Info Mart Manager.
Job_MigrateGIM Runs all scripts and makes any other updates necessary to prepare your Info Mart database for the new release of Genesys Info Mart, and automatically updates IDB(s) for use with Genesys Info Mart, if required.	Once during the process of moving from an earlier 8.x release to the current one	This job must be run from Genesys Info Mart Manager except in the special circumstance where you are using it to create optional export views (see Creating or updating export views, below).
Job_ExportGIM Periodically copies the data that is stored in the Info Mart database into local .csv files, one file per table, so that the data is available for further import into a data warehouse.	Intraday, as scheduled	This job cannot be run from Genesys Info Mart Manager. This job, which was originally introduced for Genesys Engage cloud deployments, is supported in on-premises deployments starting with release 8.5.011.22.

Job_InitializeGIM

Genesys Info Mart Server automatically launches Job_InitializeGIM to initialize the Info Mart database during the first run of Genesys Info Mart after the initial deployment. Job_InitializeGIM performs the following functions:

• Populates the following dimensions with fixed information:
  • ATTEMPT_DISPOSITION
  • CALL_RESULT
  • CAMPAIGN_GROUP_STATE
  • CONTACT_INFO_TYPE
  • DATE_TIME
  • DIALING_MODE
  • INTERACTION_RESOURCE_STATE
  • INTERACTION_TYPE
  • MEDIA_TYPE
  • RECORD_STATUS
  • RECORD_TYPE
  • RESOURCE_STATE
  • TECHNICAL_DESCRIPTOR
• In a partitioned database, creates the first set of partitions to be populated during the first extract, transform, and load (ETL) cycle.
• Automatically modifies the IDB schema(s) for use with Genesys Info Mart.

Starting with release 8.5.006, the job automatically creates any missing IDB indexes that Genesys Info Mart requires for the particular DAP role(s). Starting with release 8.5.007, the job also automatically creates any missing views, if database links are used. In earlier releases, the job automatically executed SQL scripts to modify the IDB schema(s) as Genesys Info Mart requires.

Job_ExtractICON

Job_ExtractICON extracts data from one or more IDBs in discrete chunks and stores it either directly in the Global Interaction Database (GIDB) tables or, for voice interaction data, initially in the Merge tables within GIDB. As part of the extraction process for Voice details, Job_ExtractICON merges related data in the Merge tables, and then moves the data to the GIDB tables.

In particular, Job_ExtractICON:

• Populates the START_DATE_TIME_KEY field in the GIDB and Merge tables.
• Merges call data in the Merge tables.
• Creates audit log records in the CTL_AUDIT_LOG table for each chunk.
• Starting with release 8.5.006, automatically creates any missing IDB indexes that Genesys Info Mart requires for the particular DAP role(s). Starting with release 8.5.007, the job also automatically creates any missing views, if database links are used.

In high availability (HA) deployments, the extraction job also analyzes the Interaction Concentrator (ICON)–provided session information in the redundant IDBs that store the same type of data (Configuration, Voice, Outbound Contact, or Multimedia details), to evaluate which IDB to use for data extraction in a particular extraction cycle. This analysis occurs prior to data extraction in a particular extraction cycle.

After the successful completion of the extraction job, the Genesys Info Mart Server launches the job that transforms all the extracted ICON data.

Important

By default, all time dimension data is calculated in Coordinated Universal Time (UTC) format.

Extraction Roles

The algorithm that Job_ExtractICON uses to extract data depends on the extraction role that you configured in the database access point (DAP) that enables Genesys Info Mart to access IDB. The extraction roles are:

• ICON_CFG
• ICON_CORE
• ICON_OCS
• ICON_MM

The extraction algorithms use high-water mark (HWM) timestamps, configured chunk sizes, and configured stuck thresholds to determine an extraction window (in other words, a time span for which data will be extracted) for each data domain during a particular ETL cycle. For more information about the extraction algorithms and how the extraction job processes data, see the chapter about ETL processing in the Genesys Info Mart 8.1 Deployment Guide.

ICON_CFG

For the ICON_CFG role, Job_ExtractICON extracts:

• All new and changed data from IDB tables that store the contact center configuration history, and stores the data in the GIDB tables of the Genesys Info Mart database. The job extracts all available configuration data in one extraction cycle so that the transformation of other data can proceed.
• Object relationship records from IDB tables. New relationship fact records are inserted into GIDB, and updates are simply merged into existing historical records.

ICON_CORE

For the ICON_CORE role, Job_ExtractICON extracts:

• Completed virtual queue details
• Completed voice interaction details, such as calls and user data (including call-based attached data and UserEvent-based key-value pair [KVP] data)
• Both active and completed voice agent login session details
• Both active and completed voice agent states
• Completed voice agent state reason codes
• Both active and completed voice do-not-disturb (DND) modes

The job stores the information in the GIDB tables of the Genesys Info Mart database. The job also merges voice data in the Merge tables before transformation.

ICON_OCS

For the ICON_OCS role, Job_ExtractICON extracts all new and changed data from IDB tables that store Outbound Contact data from Outbound Contact Server (OCS), and stores the data in GIDB.

Important

The IDB GC_TIME_ZONE table is extracted as part of the ICON_CFG ETL process. The GC_TIME_ZONE table is used by Genesys Info Mart by the transformation job to create the Info Mart table TIME_ZONE, which is used within Genesys Info Mart OCS processing to associate OCS interactions with specifically defined Configuration Layer–based time zones (this includes custom time zones).

ICON_MM

For the ICON_MM role, Job_ExtractICON extracts:

• All new and changed data from IDB tables that store multimedia interactions. Both active and completed multimedia interactions are extracted along with user data (including interaction-based attached data and eServices/Multimedia-specific attributes). These interactions do not need to be merged.
• Both active and completed virtual queue details.
• Both active and completed multimedia agent login session details.
• Both active and completed multimedia agent states.
• Completed multimedia agent state reason codes.
• Both active and completed multimedia do-not-disturb (DND) modes.

The extraction job stores the extracted multimedia data in the GIDB tables of the Genesys Info Mart database.

Job_TransformGIM

Job_TransformGIM transforms the data that has been extracted from all IDBs or that is available to Genesys Info Mart through other data streams such as Elasticsearch databases or Apache Kafka. The Genesys Info Mart Server launches this job during each ETL cycle after it has extracted data from all IDBs. Job_TransformGIM transforms GIDB and non-IDB data and then loads it into the Info Mart database as the last step of the transformation process.

The transformation logic implements a dependency between the data in primary and secondary tables. This concept is referred to as horizontal transform. For more discussion of horizontal transform, including transformation behavior when there is delayed or missing data, see the section about data transformation in the chapter about ETL processing in the Genesys Info Mart 8.1 Deployment Guide.

Error Handling

Genesys Info Mart handles errors differently, depending on the type of data and the reasons for the error. The main categories of error that the transformation job might encounter are:

• Missing configuration data
• Partially merged voice interactions
• Data inconsistencies

For more information, see Troubleshooting Genesys Info Mart jobs. For more information about the behavior of the transformation job when it encounters errors, see the section about error handling in the chapter about ETL processing in the Genesys Info Mart 8.1 Deployment Guide. For more information about the configuration options that control error handling, see the error-policy Section in the Genesys Info Mart Configuration Options Reference.

Job_AggregateGIM

In deployments that include GCXI or RAA, Job_AggregateGIM calculates or recalculates the historical Aggregate tables in the Info Mart database based on:

• Data that changed since the last load of the historical fact tables
• New settings for configuration options that control aggregation

In release 8.x, Job_AggregateGIM runs continuously within a time window that you specify.

During ETL processing, before the transformation is committed, the transformation job notifies the aggregation engine that there is new or changed data. The aggregation engine writes the data to an auxiliary table. The aggregation job, which is implemented as a plug-in inside the Genesys Info Mart Server process, reads the data from the auxiliary table, aggregates new data and recalculates historical aggregates, and updates the aggregate tables in the Info Mart database.

For more information about the aggregation package and running the aggregation process, see the Reporting and Analytics Aggregates User’s Guide. For information about managing the aggregation job through the Genesys Info Mart Manager, see Managing jobs with Genesys Info Mart Manager.

Scheduling the Aggregation Job

Job_AggregateGIM is an optional job. You can run Job_AggregateGIM from the Genesys Info Mart Manager if you plan to use GCXI reports. You can also aggregate directly from the command prompt, in autonomous aggregation mode. For more information about these aggregation modes, see the RAA documentation.

Configuration options enable you to specify:

• Whether Job_AggregateGIM will run the aggregation engine within the Genesys Info Mart Server process, under the control of the scheduler.
• The start times and duration of the daily intervals within which Job_AggregateGIM will run. Within these intervals, Job_AggregateGIM will run continuously.

For more information about the scheduling options, see the schedule Section in the Genesys Info Mart Options Reference.

You can calculate or recalculate the aggregates for a certain time span using Genesys Info Mart Manager (see Procedure: Re-aggregating data).

Job_MaintainGIM

Job_MaintainGIM performs the following tasks:

• Purges the Info Mart database in accordance with configurable data-retention policies, as described below (see Purging the Info Mart database).
• Populates the calendar table(s) for future reports, as described below (see Maintaining calendar tables).
• If you are using partitions in a partitioned database, adds partitions as necessary to process incoming data, as described below (see Maintaining database partitions).
• Starting with release 8.5.010, processes GDPR requests, as described below (see Processing GDPR requests).

Purging the Info Mart database

Job_MaintainGIM purges:

• Completed and artificially terminated fact data from GIDB.
• Completed and artificially terminated fact data from the dimensional model.
• Discarded operational data from discard tables.
• Outdated information from the AUDIT_LOG and History tables.
• Configuration fact data from GIDB and relevant fact tables.
• For partitioned tables, partitions that contain only completed and artificially terminated fact data that is eligible to be purged.

Job_MaintainGIM uses different algorithms to purge different categories of data from various areas of the Info Mart schema. Separate configuration options enable you to configure different retention policies for the different categories of data. For more information about the various days-to-keep-* configuration options that control data retention, see the gim-etl Section in the Genesys Info Mart Options Reference. For more information about data retention policies in general and the purging algorithms that Job_MaintainGIM uses, see the chapter about maintenance and other activities in the Genesys Info Mart 8.1 Deployment Guide. For a list of the tables Job_MaintainGIM purges, see Info Mart Tables Purged by the Maintenance Job.

Important

Job_MaintainGIM does not purge old aggregate data or dimension data.

Purging mechanism

The actual SQL commands that Job_MaintainGIM issues depend on whether the tables are partitioned.

• When Job_MaintainGIM deletes rows in nonpartitioned tables, the job issues SQL DELETE operations against the tables. Running this job daily results in a small percentage of the table being deleted, which minimizes the time that it takes the RDBMS server to find the rows, delete them, and make index adjustments.
• When Job_MaintainGIM purges partitioned tables, the job issues the appropriate SQL commands against the tables, as required by the RDBMS implementation, to drop partitions.

Scheduling purging

To use Job_MaintainGIM to purge data, configure the time of day that you want Genesys Info Mart Server to launch this job. The job is run once a day. For more information about enabling or disabling a purging schedule, see Setting scheduling options for Genesys Info Mart Server, particularly Step 10.

Maintaining calendar tables

The calendar tables are the default DATE_TIME dimension table and any custom calendar tables that you create to support your reporting. Job_InitializeGIM initially populates the calendar table(s) for the period of time that you specify in the date-time-max-days-ahead option, so that calendar dimensions are available for your reports. Job_MaintainGIM continues to populate the calendar tables when the next batch of calendars is required.

Tip

Genesys does not recommend that you populate the calendar tables more than a year in advance.

For information about:

• The configuration options that control population of the calendar dimensions, see the date-time Section in the Genesys Info Mart Options Reference.
• Creating custom calendars, see Creating Custom Calendars in the Genesys Info Mart Deployment Guide.
• Modifying existing calendar dimensions, see Changing calendar dimension values.

Maintaining database partitions

You can use partitioning on Oracle (range partitioning only), Microsoft SQL Server, and PostgreSQL databases. Fact tables and associated indexes in GIDB and the dimensional model are partitioned. Configuration object tables, configuration relationship fact tables, and dimension tables are not partitioned.

During initialization, Job_InitializeGIM creates the first set of partitions, and Job_MaintainGIM subsequently creates additional partitions as required to be populated during ETL cycles.

You can configure Genesys Info Mart to specify the size of the partitions in GIDB and the dimensional model and to control how far ahead the Genesys Info Mart jobs (Job_InitializeGIM in the first instance, then Job_MaintainGIM on an ongoing basis) will create partitions, in preparation for future ETL cycles. For more information, see the descriptions of the partitioning-* options in the gim-etl Section in the Genesys Info Mart Options Reference.

Processing GDPR requests

Starting with release 8.5.010, the maintenance job processes input JSON files that customers provide to comply with Right to Access ("export") or Right of Erasure ("forget") requests from their customers ("consumers"). Starting with release 8.5.010.16, Genesys Info Mart also supports customer compliance with GDPR requests relating to employee data.

As described in Genesys Engage Premise Support for GDPR in the Genesys Security Deployment Guide, customers place the input files in a tenant-specific, configurable location. The JSON files identify the consumers or employees who have made GDPR requests. Job_MaintainGIM processes any "export" or "forget" JSON files that have been added or modified since the job last ran. For "forget" requests, the data is redacted in Info Mart fact tables. For both export and "forget" requests, the unredacted data is stored for a configurable amount of time (maximum 30 days) in the CTL_GDPR_HISTORY table.

The personally identifiable information (PII) that Genesys Info Mart exports or redacts is specified in the input JSON files in:

• The phone and email attributes that identify the requesting consumer
• The username attribute that identifies the requesting employee
• Custom user data KVPs and custom Outbound Contact Server (OCS) fields customers might specify in the "gim-attached-data" element

For more information, see Genesys Info Mart Support for GDPR in the Genesys Security Deployment Guide.

Job_UpdateStats

Job_UpdateStats performs important aspects of database maintenance to improve query performance in PostgreSQL deployments. The job uses a combination of Genesys Info Mart and default PostgreSQL functionality to:

• Detect tables for which statistics are out of date and update them
• Run a vacuum process, which supplements autovacuum, to reclaim storage space from updated or deleted rows

Important

Terminology Note: Although Job_UpdateStats performs maintenance activities, in the Genesys Info Mart documentation, the term maintenance job refers to Job_MaintainGIM only.

Scheduling Job_UpdateStats

Genesys recommends that you configure the Genesys Info Mart Server to run Job_UpdateStats frequently throughout the day. You must use configuration options to schedule the job; you cannot schedule or run the job manually from the Genesys Info Mart Manager.

For more information about enabling or disabling the schedule for Job_UpdateStats, see Setting scheduling options for Genesys Info Mart Server. For more information about the scheduling options, see the schedule Section in the Genesys Info Mart Options Reference.

Job_MigrateGIM

When you need to migrate from an existing Genesys Info Mart 8.x deployment to a later release of Genesys Info Mart, you run Job_MigrateGIM as a part of the transition process. Job_MigrateGIM automatically runs all of the scripts necessary to prepare your existing Info Mart database for use with the current release of Genesys Info Mart 8.x. If required, Job_MigrateGIM automatically modifies the IDB schema(s) for use with the upgraded Genesys Info Mart.

• When you restart an upgraded Genesys Info Mart Server application with an unmigrated Info Mart database, Genesys Info Mart automatically detects an out-of-date Info Mart database schema version and puts the Genesys Info Mart Server into migration state. In this state, you cannot run any jobs other than Job_MigrateGIM.
• Starting with release 8.5.006, Job_MigrateGIM automatically creates any missing IDB indexes that Genesys Info Mart requires for the particular DAP role(s). Starting with release 8.5.007, the job also automatically creates any missing views, if database links are used. In earlier releases, each time the job was run, the job automatically executed update_idb_*.sql scripts to modify the IDB schema(s) as Genesys Info Mart requires.
• When Genesys Info Mart checks the deployment configuration before the start of the extraction job during normal functioning, it detects any IDBs in the Genesys Info Mart application connections that are not the correct version for Genesys Info Mart. This situation might arise when you upgrade Interaction Concentrator or when you add a new IDB to an existing deployment. In this situation, Genesys Info Mart Server goes into the migration state and will not run any jobs until you manually run Job_MigrateGIM, which automatically executes the required scripts to update the IDB(s).

Important

Job_MigrateGIM migrates only existing 8.x deployments to the later 8.x release of Genesys Info Mart. There is no migration path from 7.x to 8.x.

Migration scheduling considerations

Ensure that no queries or other activities are performed against the Info Mart database while Job_MigrateGIM runs. Be sure to take this into account when you plan migration.

Because Job_MigrateGIM might be required to execute the scripts to update IDB(s), consider all ways in which you can minimize or prevent contention between Genesys Info Mart and ICON activity on IDB while Job_MigrateGIM runs. For more information, see "Preventing Deadlocks on IDB During Genesys Info Mart Migration" in the “Genesys Info Mart 8.x Migration Procedures” chapter in the Genesys Migration Guide.

Tip

The on-demand-migration configuration option enables you to configure Genesys Info Mart to run Job_MigrateGIM automatically when the Genesys Info Mart schema version is not up to date. However, Genesys recommends that you use this method only if you have completed all relevant pre-migration and post-migration steps.

For complete migration preparations and procedures, see the Genesys Info Mart 8.x section of the Genesys Migration Guide.

Creating or updating export views

The "About Data Export" page in the Genesys Info Mart Physical Data Model for your RDBMS (for Microsoft SQL Server, Oracle, or PostgreSQL, respectively) describes how you can use Job_ExportGIM to export Info Mart data for import into another data warehouse and why you might want Job_ExportGIM to export your data using export views.

To create or update (refresh) the export views for Job_ExportGIM to use, execute Job_MigrateGIM from the command line with the make-export-views parameter. For example:

gim_etl_server.bat -host localhost -port 8000 -app <app> -job Job_MigrateGIM -make-export-views

Genesys Info Mart will create export views of the schema that was in effect before the migration job was run. The following table summarizes considerations for when to run the migration job for this purpose.

I want to...	What must I do?
Set up export views or update existing export views to match an Info Mart schema that was migrated some time ago.	Verify that the target database schema and import and consumption queries are ready to process data from tables and columns in the current Info Mart schema. Execute the migration job from the command line, with the make-export-views parameter, at any time before the first export (or the first export that you want to use the updated views). The new export views will reflect the current Info Mart schema.
Migrate Genesys Info Mart without updating existing export views (in other words, the post-migration export will export the same tables and columns as the pre-migration exports).	Run the migration job from Genesys Info Mart Manager in the usual way during the migration process, to migrate the Info Mart schema.
Migrate Genesys Info Mart and update existing views to reflect the post-migration schema.	Verify that the target database schema and import and consumption queries are ready to process the new data. Run the migration job from Genesys Info Mart Manager in the usual way during the migration process, to migrate the Info Mart schema. Run the migration job again from the command line, with the make-export-views parameter. The new export views will reflect the migrated Info Mart schema.

Job_ExportGIM

Job_ExportGIM exports data from fact and dimension tables that are part of the Genesys Info Mart dimensional model, including fact extension tables in the Info Mart database, and creates a .zip archive containing individual .csv files for each table. You can configure various aspects of the export function, including the frequency with which the job runs and the amount of data that it exports. You can also configure Genesys Info Mart to export the data using export views, which represent a frozen snapshot of the Info Mart schema, so that the export will always include the same tables and columns even if future migrations introduce schema changes.

See the page about Data Export capability in the Genesys Info Mart Physical Data Model for your RDBMS (for Microsoft SQL Server, Oracle, or PostgreSQL, respectively) for full details, including information about:

• Using export views
• Using the Genesys-provided update_target_*.sql script to create a compatible target database schema
• The export file/directory structure and export metadata
• Configuration options that control aspects of Job_ExportGIM behavior

Scheduling data export

Use configuration options to configure the Genesys Info Mart Server to run Job_ExportGIM; you cannot schedule or run the job manually from the Genesys Info Mart Manager.

For more information about enabling or disabling the schedule for Job_ExportGIM, see Setting scheduling options for Genesys Info Mart Server.