ETL Audit Dashboard
Use the (CX Insights for iWD folder >) ETL Audit Dashboard to efficiently audit/analyze iWD ETL processes. The dashboard provides an overview of job execution statistics, including duration, status, and other information drawn from the ETL_AUDIT table.
The dashboard report organizes data on the following tabs:
- Summary Intraday tab — Use this tab to analyze job execution statistics for intraday jobs, which are jobs that run on a regular interval (every 15 minutes by default, or at user-configurable intervals).
- Summary Historical tab — Use this tab to analyze job execution statistics for historical jobs, which are jobs that run on a daily basis.
Note that the term dashboard is used interchangeably with the term dossier. Dashboards provide an interactive, intuitive data visualization, summarizing key business indicators (KPIs). You can change how you view the data in most reports and dashboards by using interactive features such as selectors, grouping, widgets, and visualizations, and explore data using multiple paths, through text and data filtering, and layers of organization.
To get a better idea of what this dashboard looks like, view sample output from the report:
SampleETLAuditDashboard.pdf
The following table explains the prompts you can select when you generate the ETL Audit Dashboard:
| Prompt | Description |
|---|---|
| Start Time Min | Choose the low filter value for the Start Time attribute. |
| Start Time Max | Choose the high filter value for the Start Time attribute. |
| Data Source Type | Optionally, select a data source type on which to focus the report. |
| Process Name | Optionally, select a business process on which to focus the report. |
| Status | Optionally, select a status on which to focus the report. |
The following table explains the attributes used in the ETL Audit Dashboard:
| Attribute | Description | Data Mart Column |
|---|---|---|
| ETL Audit Key | Enables data to be organized based on the technical key which identifies DataMart job. | ETL_AUDIT.ETL_AUDIT_KEY |
| Data Source Type | Enables data to be organized based on the type of data source. | V_DIM_DATA_SOURCE_TYPE.DATA_SOURCE_TYPE |
| Data Source Name | Enables data to be organized based on the name of the data source. | ETL_AUDIT.DATA_SOURCE_NAME |
| Process Name | Enables data to be organized based on the process name. | V_DIM_PROCESS_NAME.PROCESS_NAME |
| Status | Enables data to be organized based on job status. | V_DIM_STATUS.STATUS |
| Job Type | Enables data to be organized based on the the type of job. | ETL_AUDIT.JOB_TYPE |
| Start Time | Enables data to be organized based on the time when the job started. | ETL_AUDIT.ETL_AUDIT_START_TIME |
| Finish Time | Enables data to be organized based on the time when the job finished. | ETL_AUDIT.ETL_AUDIT_FINISH_TIME |
| Batch ID | Enables data to be organized based on the batch number. Applicable only to jobs that process data in batches. | ETL_AUDIT.BATCH_ID |
| First Extracted Event ID | Enables data to be organized based on the ID that starts the range of event IDs that are processed. | ETL_AUDIT.FIRST_EXTRACTED_EVENT_ID |
| Last Extracted Event ID | Enables data to be organized based on the ID that ends the range of event IDs that are processed. | ETL_AUDIT.LAST_EXTRACTED_EVENT_ID |
| Batch Last Event ID | Enables data to be organized based on the lD of the last event in the batch. | ETL_AUDIT.BATCH_LAST_EVENT_ID |
| Last Interval Date Key | Enables data to be organized based on the last date interval that is processed by the aggregate ETL scripts. | ETL_AUDIT.LAST_INTERVAL_DATE_KEY |
| Last Interval Time Key | Enables data to be organized based on the last time interval that is processed by the aggregate ETL scripts. | ETL_AUDIT.LAST_INTERVAL_TIME_KEY |
The following table explains the metrics used in the ETL Audit Dashboard:
| Metric | Description | Source or Calculation |
|---|---|---|
| Duration | The total amount number of seconds that job execution lasted. This value is calculated based on the creation timestamp for the job during the given time interval (ETL_AUDIT_START_TIME), and the finish time for the job (ETL_AUDIT_FINISH_TIME). | ETL_AUDIT.DURATION |
| Extracted Events Amount | The number of loaded events during a job or batch run. | ETL_AUDIT.EXTRACTED_EVENTS_AMOUNT |