Jump to: navigation, search

GDPR Compliance

Feature Server release 8.1.202.10 includes the following python script:

  • forgetMe.py—Deletes the voicemail data of a customer when requested.

You can run this python script on the master Feature Server instance to delete the voicemail data. The customer-related information received from the common Web UI will be transformed into a JSON input file. The forgetMe.py script will fetch the JSON files (that were added since the last execution time to fetch the ANI) added to the gdpr-directory option configured in the [gdpr] section of the master Feature Server. The script will then set the expiration time to 21 days for voicemails that correspond to the ANI obtained. The voicemails will be deleted after the expiration time.

From release 8.1.202.16, Forget Me will fetch files based on the last script execution time.

Feature Server release 8.1.2 includes the following python script:

  • exportMe.py—Exports voicemail data in .wav format when requested by the customer, from the Cassandra database in a client-understandable format.

Data, such as name, phone number, email address, bank details, and IP address are considered as Personally Identifiable Information (PII). Anything that is likely to identify an individual, or a combination of other held data to identify an individual is considered as PII. According to EU GDPR, when a client requests to access his/her personal data that is available with the contact center, the PII associated with the client should be exported from the database in client-understandable format.

Forget Me

The forgetMe.py script will parse the input JSON file and obtain the ANI. It will check whether the ANI has any voicemail information associated in the Feature Server Cassandra database. If any data is present, then the respective records will be queried in the column families: mailmessage, message_bytes, and mailboxmessages using its key and the expiration time will be set to 21 days. If no messages are present for an ANI, the operation will be skipped and the appropriate status messages will be written to the log file. The log file is in the format: sipfs-forgetMe-<DDMMYYYY>.log. The output file includes the input file information, and the execution results, success and failures with appropriate error information.

Notes

  • Feature Server does not delete the data upon manual script execution. Instead, it will set the expiration time which will make the data inaccessible.
  • The Feature Server script will query the Cassandra database based on the ANI which is received as one of the inputs.
  • The expiration time (TTL) of the Voicemail messages and the metadata of the corresponding ANI will be set to 21 days.
  • Voicemail messages corresponding to an ANI residing in different mailboxes will also be set with the new expiration time.
  • The expiration time will be set for the corresponding records of the column families: mailmessage, mailboxmessages, and message_bytes from sipfs keyspace.
  • The script will be invoked once a day scheduled by automatic triggering within Feature Server.

Sample input JSON file

The following is a sample input JSON file: 

{
   "caseid":"123456789",
   "consumers":[
      {"consumer":
         [
            {"name":"John Doe"},
            {"name": "John Q. Doe"},
            {"phone":"555551212"}
         ]
      },
      {"consumer":
         [
            {"name":"Dan Akroyd"},
            {"phone":"555556161"},
            {"email":"danny@hollywood.com"},
            {"fbid":"Dan Akroyd"}
         ]
      }],
   "gim-attached-data":{"kvlist":["AcctNum", "SSN"]}
 }

If the retention limit is already set for voicemail messages, then the retention limit of the voicemail will be set as the expiration time, provided the retention limit is less than 21 days.

If the customer makes a second request to delete the voicemails associated with their ANI, then voicemail set with expiration time during the first request will be skipped during execution. The voicemails deposited after the first request alone will be set with the new expiration time.

Sample Output File

====================================================================
| forgetMe.py script run |
====================================================================

<input.json>//actual content

execution start time = 2018-04-19 13:23:22
[debug]ANI '555556161' has voicemail data
[debug]Expiration time set for 555556161 in mailmessage
[debug]Expiration time set for 555556161 in mailboxmessages
[debug]Expiration time set for 555556161 in message_bytes
[debug]ANI '555556162' has no voicemail data 
execution end time = 2018-04-19 13:23:22

Scheduling the forget-me Task

The forgetMe.py script execution should be scheduled once a day from SIP Feature Server. You can schedule the forget-me task as described in Scheduled maintenance tasks.

Important
The "update-mailbox-counters" task should run once a day to overcome the mailbox counter issue caused by running the "forget-me" task.

Configuration Options

The following options enable automatic script execution of Forget Me:

  • forget-me.active = true (activate/deactivate)
  • forget-me.cmd = forgetMe.py --dbhost <host> --dbport <port>
  • forget-me.schedule = 0 0 4 ? * * (Schedule)

Export Me

SIP Feature stores voicemail recordings of the client in the Cassandra database. The requirement is to develop a script that queries the Cassandra database based on the ANI fetched from the input file and export the voicemail recordings. This script should be triggered and executed automatically from SIP Feature Server once a day. The contact center administrator receives the client details from a common web user interface as specified in the GDPR compliance page and generates the input JSON file that is common for all components/solutions.

Input source - export-<DDMMYYYY>.json

Output naming - sipfs-exportMe-<DDMMYYYY>-execution.log (UTC is the time standard to be used for naming)

  • SIP Feature Server stores the client's voicemail recordings in Cassandra DB.
  • The customer care administrator will receive the customer details through a common web user interface and generate the input JSON file required for Export Me.
  • SIP Feature Server will contain the exportMe script that fetches the ANI from the input file that is added to the gdpr-directory.
  • The script will parse all the input files that were added since the last execution time to fetch the ANI.
  • The script will query the Cassandra DB for voicemail recordings associated with each ANI and export them as .wav files.
  • Voicemail recordings for an ANI residing in different mailboxes will also be exported.
  • The script will be triggered and executed automatically once a day using scheduled tasks.

Input and Output Formats

The following arguments are mandatory. If any of the following arguments are missing, the script will fail after writing the appropriate error messages to the output file. Only the Cassandra host and port will be configured in the Scheduled tasks command. Other parameters will be added to command line arguments from the SIP Feature Server application when the task is triggered.

Script run-command - python exportMe.py --dbhost <host> --dbport <port> --fileLocation <gdpr-directory>

Where:

  • <host>: Cassandra DB host name of SIP Feature Server
  • <port>: Cassandra DB port of SIP Feature Server
  • <gdpr-directory>: Absolute path to the directory where input files are stored

Sample run-command - python exportMe.py --dbhost 10.31.12.99 --dbport 9160 --fileLocation "C:\Users\joanselm\Documents\privacy gdpr

Sample Input File

export-<DDMMYYYY>-<any optional content>.json

The phone number here refers to the ANI that SIP Feature Server will use to query the SIP Feature Server DB.

The following is a sample input JSON file: 

{
   "caseid":"123456789",
   "consumers":[
      {"consumer":
         [
            {"name":"John Doe"},
            {"name": "John Q. Doe"},
            {"phone":"55551011"}
         ]
      },
      {"consumer":
         [
            {"name":"Dan Akroyd"},
            {"phone":"55551012"},
            {"phone":"555556162"}, 
            {"email":"danny@hollywood.com"},
            {"email":"funnyguy@comedy.org"},
            {"fbid":"Dan Akroyd"}
         ]
      }],
   "gim-attached-data":{"kvlist":["AcctNum", "SSN"]}
 }

There are two primary output files:

  • <component>-<script name>-<DDMMYYYY>-access.json
    This will include input file name at the top, JSON format, and references to recordings available for each ANI. The recordings will be exported as .wav files. For each input file fetched, access.json will be generated individually.
  • <component>-<script name>-<DDMMYYYY>-execution.log
    This will include input file at the top. The log will contain the execution results, and success and failure messages with appropriate error information.

The naming convention for the exported recording file will contain the following format:

ANI-<Date from timestamp>.wav

Sample Output Files

access.json 

{
   "inputfile":"export.json",
   "caseid":"123456789",
   "consumers":[
      {"55551011":
         [
            {"file":"55551011-11072018.wav"},
            {"file":"55551011-23072018.wav"},
          ]
      },
      {"55551012":
         [
            {"file":"55551012-11072018.wav"},
            {"file":"55551012-23072018.wav"},
            {"msg":"Message marked for deletion"}, 
         ]
      }],
 }

execution.log 

//input file

====================================================================

| exportMe.py script run |
====================================================================
[INFO] Execution start time = 2018-10-01 18:22:14
[INFO] DBHOST:172.24.131.63 DBPORT:9160
[INFO] KEYSPACE:sipfs
[INFO] ANI list obtained from inputfile:['55551011', '55551012']
[DEBUG] Message with key:65b2073e-4191-49e6-a013-a324255010d2 and callerid:55551011 is present in message_bytes of keyspace sipfs
[DEBUG] ANI '55551011' has voicemail data
[DEBUG] Voicemail exported as 55551011-03072018-072321.wav
[DEBUG] ANI 55551012 has no new voicemail data
[INFO] Execution end time = 2018-10-01 18:22:14

Scheduling the export-me Task

The feature also requires automatic script execution from SIP Feature Server once a day. This will be achieved using scheduled maintenance tasks. Scheduled maintenance tasks can be executed only from the master Feature Server. Export Me will be added as a task under ScheduledTasks that is available in SIP Feature Server application options.

task-name - export-me

Configuration Options

The following options enable automatic script execution of Export Me:

  • export-me.active = true (Activate/Deactivate)
  • export-me.cmd = exportMe.py --dbhost <host> --dbport <port> (Command line: The other parameters are added from SIP Feature Server)
  • export-me.schedule = 0 0 4 ? * * (Schedule)

Capture.png

The tasks can be started/stopped using the web APIs, which are already available for scheduled tasks.

This page was last edited on March 6, 2019, at 16:37.
Comments or questions about this documentation? Contact us for support!