Jump to: navigation, search

Troubleshooting

What Happens if MCP fails to send recordings to Recording Processor?

If you have scheduled maintenance for an extended period of time which could cause lost recordings, Genesys recommends that you reconfigure Media Control Platform (MCP) to stop posting recording, so that you do not have to handle the failed recordings.

Important
Genesys recommends that your deployment be sized to accommodate 48 hours worth of recording on disk, and that there is plenty of physical memory available on the MCP machine, so that the recording can reload efficiently when MCP restarts.

To reconfigure MCP:

  1. Stop MCP.
  2. Set the recordnumparallelpost parameter in the [mcp] section to 0.
  3. Restart MCP.
    This disables the MCP posting functionality and all recording/metadata files will continue to accumulate in the cache/record directory.
  4. When suitable, stop MCP, set the recordnumparallelpost parameter back to the recommended/default value and restart MCP.
    MCP will now scan the cache/record directory and post the requests for the recording files that are present.

Handling Failed Recordings

For handling MCP post failures in the cache directory:

  1. Find the MCP's $InstallationRoot$/cache/record/failed directory for the following file types:
    • [audiofilename]-htccmetadata.json
    • [audiofilename]
    • [audiofilename].pem (provided only when encryption is enabled)
    The audio file name can have extensions of .mp3 or .mp3.bin. For example:
    • 743CTLVFHH0TH7HHB4D1PTS5US00000F_2014-10-04_11-47-25-00780142-10004349-00000001.mp3-htccmetadata.json
    • 743CTLVFHH0TH7HHB4D1PTS5US00000F_2014-10-04_11-47-25-00780142-10004349-00000001.mp3
  2. Submit these files to:
    1. Recording Processor
    2. Recording storage (WebDAV)
  3. Recording post failures can be due to incorrect provisioning in the IVR Profile, so before posting the metadata file to Recording Processor, look at the metadata file to make sure provisioning is correct. [+] See example

    Check for following items in the json file:

    • mediaFiles[0].mediaDescriptor.path—Make sure this is the correct WebDAV address
    • mediaFiles[0].parameters['rp.speechminer_uri']—Make sure this is the correct location of SpeechMiner Interaction Receiver
    • mediaFiles[0].parameters['rp.speechminer_auth']—Make sure this is the correct authorization of SpeechMiner Interaction Receiver
  4. When encryption is enabled, edit the json file and manually add the pkcs7 headers which is the content of the .pem file (make sure that you replace the line break with \n). [+] See example
  5. When you are ready to submit the files, do the following steps in order:
    1. Issue an HTTP PUT to WebDAV:
      curl -u [webdavauth] -T [audiofilename] [mediaFiles[0].mediaDescriptor.path]

      Using this example above and assuming the WebDAV authorization is webdav:password, you will see the following returned:

      curl -u webdav:password -T 743CTLVFHH0TH7HHB4D1PTS5US00000G_2014-10-07_15-46-21-00780143-100043F9-00000001.mp3 
      http://ca-to-irp02/recordings/743CTLVFHH0TH7HHB4D1PTS5US00000G_2014-10-07_15-46-21-00780143-100043F9-00000001.mp3
    2. Issue an HTTP POST to Recording Processor:
      curl -u [rpauth] -X POST -d @[metadatafile] --header "Content-Type: application/json" [rpuri]

      where:

      • [rpauth] is the Recording Processor authorization as per the IVR Profile
      • [metadatafile] is the metadata json file in MCP's cache/failed directory
      • [rpuri] is the Recording Processor URI as per the IVR Profile

      Using the example above, you should see the following returned:

      curl -u user:password -X POST -d @743CTLVFHH0TH7HHB4D1PTS5US00000F_2014-10-04_11-47-25-00780142-10004349-00000001.mp3
      -htccmetadata.json --header "Content-Type: application/json" http://ca-to-macon2:8889/api/contact-centers//recordings/

What Happens if Recording Processor fails to send recordings to SpeechMiner or Interaction Recording Web Services (Web Services)?

If Recording Processor fails to send recordings to SpeechMiner or Interaction Recording Web Services (Web Services if you're using version 8.5.210.02 or earlier) for any reason, Recording Processor will continue trying to send these recordings until it exhausts the number of retries configured. When the number of retries have been exhausted, Recording Processor logs an INFO message to the log file with the corresponding JSON content that is not submitted to SpeechMiner or Interaction Recording Web Services (Web Services). The following steps show how to search for the specific messages in the log file to recover the post failure to SpeechMiner and Interaction Recording Web Services (Web Services).

Prerequisites:

  • Make sure that the Recording Processor logging is set to the INFO level.


Important
Metadata for the same call could be sent multiple times to Interaction Recording Web Services (Web Services) or SpeechMiner if the call is transferred. Therefore, even if a given metadata record is not sent upstream, the call may still exist in the Interaction Recording Web Services (Web Services) or SpeechMiner database.

Handling Failed Recordings to Interaction Receiver (SpeechMiner)

  1. Search the Recording Processor log files for the follow message: "Metadata will not be sent to SM". For example:
    grep "Metadata will not be sent to SM" recordingProcessor_071614_112548.log
    [+] Show code snippet returned
  2. Save the json string after "Metadata will not be sent to SM:" as a text file.
  3. Issue an HTTP POST to Interaction Receiver for the saved json text file. For example:
  4. curl -u <user:password> -X POST -d @json.txt --header "Content-Type: application/json" <http://speechminer/interactionreceiver>

    Where:

  • <user:password> is the user/password authorization for Interaction Receiver
  • <http://speechminer/interactionreceiver> is the URI of Interaction Receiver

Handling Failed Recordings to Interaction Recording Web Services (Web Services)

  1. Search the Recording Processor files for the following message: "Metadata will not be sent to HTCC". For example,
    grep "Metadata will not be sent to HTCC" recordingProcessor_071714_054043.log
    [+] Show code snippet returned
  2. Save the json string after "Metadata will not be sent to HTCC:" as a text file.
  3. Issue an HTTP POST to Interaction Recording Web Services (or Web Services if you're using version 8.5.210.02 or earlier) for the saved json text file. For example,
    curl -u <user:password> -X POST -d @json.txt --header "Content-Type: application/json" <http://web-services/internal-api/contact-centers/{ccid}/recordings>

    Where:

    • <user:password> is the user/password authorization for Interaction Recording Web Services (Web Services)
    • <http://web-services/internal-api/contact-centers/{ccid}/recordings> is the URI of Interaction Recording Web Services (Web Services)
    • {ccid} is the contact center identifier on Interaction Recording Web Services (Web Services)

    You can find the contact center identifier from the initial Interaction Recording Web Services (Web Services) setup, or you can look up the contact center id from the recording processor log file. In the first few lines of the log file you will find a GUID string as follows:

    2014-07-17 05:40:43,746 contact_centers INFO Contact center (:79566087-0535-4f30-bb07-2c95a22aaa89) discovered.

    The following example can be used to return the contact center ID:

    curl -u user:password -X POST -d @json.txt --header "Content-Type: application/json" http://web-services/internal-api/contact-centers/79566087-0535-4f30-bb07-2c95a22aaa89/recordings


How do I collect logs to debug issues relating to login to the SpeechMiner UI?

If you are having a problem logging into the SpeechMiner UI, you will need to collect the Network and Console log from the Chrome Developer Tools before and after the login attempts with the Preserve log checkbox enabled—for example:
Devtool.png

Make sure that you have captured the requests for the following applications after you click the login button:

  • Web Services
  • Recording Crypto Server
  • SpeechMiner
Important
Uncheck these options after you have collected the necessary information.

Why do only some of my calls have screen recordings associated with them?

Screen Recording only occurs when an agent logs into a workstation that has Screen Recording Service installed.

To solve this issue try one of the following:

  • If the Screen Recording Service is installed on your workstation and this problem persists, execute the following command to check the screen recording trigger condition settings for a given agent (for example, julie@genesys.com):

    curl -u julie@genesys.com:123456 http://10.10.15.59/api/v2/me/settings/screen-recording-client

    If the code below appears with Value = off, Screen Recording is disabled. If Value = random_voice(x) (where x is 0-100, representing the percentage of calls to record) then, the calls that do not fall into the selected percentage will not be recorded. The SRS is instructed to either record or not-record on a per-call basis. If you are not seeing screen recordings for your deployment, or, for a particular agent, you may wish to temporarily disable ‘random_voice’ or set to 100 to ensure that all interactions are instructed to have the screens recorded.

    "statusCode": 0,
    "settings": [
    {
    "name": "recordingWhen",
    "value": <to-be-checked>
    } ]

    To reconfigure Screen Recording, refer to Advanced Configuration for the Screen Recording Service.
  • Go to C:\Genesys\SRC\Problem Videos. If the Problem Videos folder contains files, verify that you have the following information and contact your Genesys Professional:
    • GSR.log
    • config.json
    • The agent's name.
    • Directory listing

How do I restore the Find Recordings capability?

Interaction Recording Web Services (or Web Services if you're using version 8.5.210.02 or earlier) uses Elasticsearch for metadata indexing, which is used by MLM, RCS archiving and the SpeechMiner Screen Recording grid.

If your Elasticsearch index is out of date, follow these steps to rebuild the index and restore the Find Recordings capability:

  1. Select one Interaction Recording Web Services (Web Services) node, and do one of the following:
    • If you have a regular maintenance window, select an existing Interaction Recording Web Services (Web Services) node and proceed to step #2.

    • If your business runs 24*7, prepare a new dedicated Interaction Recording Web Services (Web Services) node as follows:
      1. Install Interaction Recording Web Services (Web Services) in the same way a regular Interaction Recording Web Services (Web Services) node is installed. Do not add this node to the Interaction Recording Web Services (Web Services) Load Balancer.
      2. Edit the Interaction Recording Web Services application.yaml file (if you are using Web Services and Application version 8.5.201.09 or earlier modify the server-settings.yaml file instead), by adding the following configuration. Verify that you add lines under nodes for all the existing Interaction Recording Web Services (Web Services) nodes in your deployment:
           
        elasticSearchSettings:
          useTransportClient: true
          transportClient:
              nodes:
                  - {host: <elastic-search-node1>, port: 9300}
                  - {host: <elastic-search-node2>, port: 9300}
                  - {host: <elastic-search-node3>, port: 9300}
              useSniff: true
              ignoreClusterName: true
              pingTimeout: 10000
              nodesSamplerInterval: 10000
          enableIndexVerificationAtStartUp: false
          indexPerContactCenter: true
  2. Important
    By default, the Interaction Recording Web Services (Web Services) node will also serve as the Elasticsearch node. This step is performed to avoid any change to the existing Elasticsearch Cluster.
  3. Increase the Hystrix timeout for RecordingOperationApiTaskV2 by adding the following line to the Hystrix configuration:
    hystrix.command.RecordingOperationApiTaskV2.execution.isolation.thread.timeoutInMilliseconds=<max time acceptable in milliseconds>
  4. Important
    <max time acceptable in milliseconds> should be 3 times the restored period or more.
  5. Restart the selected Interaction Recording Web Services (Web Services) node, navigate to the following URL and wait until it returns the Interaction Recording Web Services (Web Services) version:
    http://<selected-web-services-node>:<web-services-listening-port>/api/v2/diagnostics/version
  6. Perform Reindexing:
    1. Determine the contact center ID using the following command:
       
      curl -u <ops-user>:<ops-pass> http://< selected-web-services-node>: <web-services-listening-port>/api/v2/ops/contact-centers; echo

      The following output is returned:

       
      {"statusCode":0,"uris":["http://< selected-web-services-node>: <web-services-listening-port>/api/v2/ops/contact-centers/<contact-center-id>"]}


    2. Run the following to regenerate indexing for Voice Recording if required. If you do not need to regenerate indexing for voice recording, skip this step and proceed to 4c.
         

      curl -u <ops-user>:<ops-pass> -XPOST -H "Content-Type:application/json" "http://<selected-web-services-node>:<web-services-listening-port>/api/v2/ops/contact-centers/<contact-center-id>/recordings" -d '{
       "operationName":"forceIndex",
       "from": <start-range-in-milliseconds>,
       "to": <stop-range-in-milliseconds>,
       "purgeOld":false
      }'
    3. Run the following command to regenerate indexing for Screen Recording:
    4.    

      curl -u <ops-user>:<ops-pass> -XPOST -H "Content-Type:application/json" "http://<selected-web-services-node>:<web-services-listening-port>/api/v2/ops/contact-centers/<contact-center-id>/screen-recordings" -d '{
       "operationName":"forceIndex",
       "from": <start-range-in-milliseconds>,
       "to": <stop-range-in-milliseconds>,
       "purgeOld":false
      }'
    Important
    • If purgeOld is true, the entire index will be purged. For this reason, set purgeOld to false when only a period of your index is lost. The typical scenario is nightly indexing for a multi-tenancy deployment. When your existing ES map is wrong and you must rebuild the entire index, you should set purgeOld to true. If your restore range is too long and you want to restore each period at a time, verify that you set purgeOld to false after the first period is restored.
    • Verify that the operation is only executed during the period at which the MLM or RCS archive job is scheduled.


  7. Once all re-indexing operations are complete:
    • Remove the change to the Hystrix timeout for the RecordingOperationApiTaskV2 that was added in step 2.
    • Restart the selected Interaction Recording Web Services (Web Services) node.

Feedback

Comment on this article:

blog comments powered by Disqus
This page was last modified on 29 March 2017, at 16:11.