Jump to: navigation, search

Kafka Capture Point Configuration Options

This section describes the Kafka capture point configuration options.

The following options should be configured in the default-values section of the Capture Point application:

  • InteractionType (optional) – Specifies the interaction type. The possible values of this option are any of the types defined. The default value is Inbound.
  • InteractionSubtype (optional) – Specifies the interaction subtype. The possible values of this option are any of the subtypes defined. The default value is InboundNew.
  • MediaType (optional) – Specifies the interaction media type. The possible values of this option are any of the media types defined. The default value is workitem.

The following options should be configured in the settings section of the Capture Point application:

  • capture-point-type (mandatory) – Specifies the Capture Point type. The following types are supported:
    • jms - JMS Capture Point
    • file - File Capture Point
    • db - DB Capture Point
    • webservice - Web Service Capture Point
    • kafka - Kafka Capture Point
    The values of this option are jms, file, db, webservice, and kafka. The default value is kafka. The value must be set to kafka so the Capture Point is configured to work via Apache Kafka.
  • kafka-server (mandatory) – A list of host:port pairs to use for establishing the initial connection to the Kafka cluster. The value of this option is passed to the underlying Java Kafka client in the bootstrap.servers option as is.
  • consumer-group (optional) – A unique string that identifies the consumer group this Capture Point is assigned to. The consumer groups functionality is used for balancing partitions when receiving threads and keeping track of the processed messages. To work correctly, this option should be set once and never changed.
    The valid value for this option is any valid string. The default value is InteractionServer.
  • inbound-topic-name (mandatory) – Specifies the inbound topic for the capture point. The valid value for this option is a valid Kafka topic name. There is no default value.
  • error-topic-name (optional) – Specifies the error topic for the capture point. The valid value for this option is a valid Kafka topic name. There is no default value.
  • processed-topic-name (optional) – Specifies the topic for processed messages for the capture point. The valid value for this option is a valid Kafka topic name. There is no default value.
  • notification-topic-name (optional) – Specifies the topic for notification messages for the capture point. The valid value for this option is a valid Kafka topic name. There is no default value.
  • xsl-inbound-transform-path (optional) – Specifies the location of the inbound transformation script. The valid value for this option is the transformation script file name that is relative to the working directory. The default value is ./iwd_scripts/iWD2IxnServerTransformer.groovy
  • xsl-outbound-transform-path (optional) – Specifies the location of the outbound transformation script. The valid value for this option is the transformation script file name that is relative to the working directory. The default value is ./iwd_scripts/IxnServer2iWDTransformer.groovy
  • after-rollback-delay (optional) – Specifies the timeout (in seconds) to wait before attempting to process inbound messages after the previous rollback. This option controls how long Interaction Server waits before retrying message processing after a recoverable error. The default value is 30, the minimum value is 1, and the maximum value is 300.
  • rollback-on-transformation-fail (optional) – Specifies how Interaction Server should behave when inbound message transformation fails. If the option is set to true, Interaction Server rolls back the transaction and attempts processing later. This option is set to false by default.
  • output-queue-size (optional) – Specifies the maximum number of unsolicited notification messages a capture point can buffer. The default value is 5000, the minimum value is 1000, and the maximum value is 20000.
  • consumer-receive-timeout (optional) – Specifies the timeout (in milliseconds) on the message consumer blocking receive method. The minimum value is 200, the maximum value is 5000, and the default value is 1000.
  • number-outbound-threads (optional) – Specifies the number of threads Interaction Server allocates to transform and send unsolicited notification messages. The default value is 4, the minimum value is 1, and the maximum value is 20.
  • number-receiving-sessions (optional) – Specifies the number of receiving sessions consuming from the inbound Kafka topic. The default value is 4, the minimum value is 1, and the maximum value is 20.
  • include-ids-in-duplicate-error (optional) – If set to true, the duplicate id error includes interaction id and external id, if available. The value for this option is set to false by default.
  • correlation-id-header-key (optional) – Specifies the key on which the correlation ID is read from the message header. The valid value for this option is any string. The default value is CorrelationID.
  • copy-original-headers-in-reply (optional) – When this option is set to true, all Kafka message headers of the request will be copied to the Kafka message headers of the reply. The default value of this option is false. When the value is false, the replies to requests have only the message ID and correlation ID headers (if present).
  • reconnect-timeout (optional) – Specifies the time interval (in seconds) between the reconnect attempts in case a connection with the corresponding Kafka cluster is broken. The default value is 10, the minimum value is 3, and the maximum value is 30.

The following options should be configured in the consumer-options section of the Capture Point application:

  • max.poll.interval.ms (optional) – The maximum delay between invocations of poll(). This places an upper bound on the amount of time that the consumer can be idle before fetching more records. If poll() is not called before expiration of this timeout, then the consumer is considered failed and the group will rebalance in order to reassign the partitions to another member. For this option, the valid value is any integer greater than 1. The default value is 600000.
  • max.poll.records (optional) – The maximum number of records returned in a single call to poll(). The valid value for this option is any integer greater than 1. The default value is 20.
  • auto.offset.reset (optional) – Specifies what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server.
    • earliest: automatically reset the offset to the earliest offset
    • latest: automatically reset the offset to the latest offset
    • none: throw exception to the consumer if no previous offset is found for the consumer's group
    • anything else: throw exception to the consumer
    The valid values are latest, earliest, and none. The default value is earliest.

The following option should be configured in the producer-options section of the Capture Point application:

  • retries (optional) – Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. The valid value for this option is any integer from zero to 2147483647. The default value is 10.

The following options should be configured in the inbound-transformer-parameters section of the Capture Point application:

  • AllowAnyAttributes (optional) – Specifies whether the provided iWD inbound transformation script should copy any unknown message attributes along with all of the known message attributes into the transformed inbound message. If the value is set to true, all attributes from the inbound iWD message are copied into the transformed message. If the value is set to false, only known attributes are copied and all of the other attributes are ignored. The list of known attributes is defined in the inbound iWD transformation script and can be customized. The value is set to false by default.
  • CompleteQueues (optional) – Specifies a list of queue names designated for interaction completion. This parameter is used by iWD specific transformation to transform request CompleteTask by providing queue name to [update] request. For this option, the valid value is a comma-separated list of queue names. The default value is iWD_Completed.
  • CancelQueues (optional) – Specifies a list of queue names designated for interaction cancellation. This parameter is used by iWD specific transformation to transform request CancelTask by providing queue name to [update] request. For this option, the valid value is a comma-separated list of queue names. The default value is iWD_Canceled.
  • RestartQueues (optional) – Specifies a list of queue names designated for interaction restart. This parameter is used by iWD specific transformation to transform request RestartTask by providing queue name to [update] request. For this option, the valid value is a comma-separated list of queue names. The default value is iWD_New.
  • ExtendedAttributes (optional) – Specifies a list of attribute names that are expected under the [Ext] tag in CreateTask request. This parameter is used by iWD specific transformation. For this option, the valid value is a comma-separated list of queue names. There is no default value.
  • SchemaDocumentPath (optional) – If configured, this option specifies the file name that contains xml schema document that the inbound transformation script should use for validation. The valid value is a path to xsd document. The default value is ./iwd_scripts/iwd_messages.xsd
  • CaseSensitiveAttributes (optional) – If the value is set to false, the transformation script ignores the letter case of known attribute names (including Ext and Data section names). The value is set to false by default.
  • CaseSensitiveActions (optional) – If the value is set to false, the transformation script ignores the letter case of action names. The default value is false.

The following options should be configured in the outbound-transformer-parameters section of the Capture Point application:

  • RejectQueues (optional) – (iWD specific) Specifies a comma-separated list of queue names for rejected interactions. The valid value is an empty string or any valid queue name. The default value is iWD_Rejected.
  • ErrorHeldQueues (optional) – (iWD specific) Specifies a comma-separated list of queue names for interactions that are held because of a configuration error (such as incomplete rules). The valid value is an empty string or any valid queue name. The default value is iWD_ErrorHeld.
  • CompleteQueues (optional) – Specifies a list of queue names designated for interaction completion. This parameter is used by iWD specific transformation to transform [moved] notification to [TaskCompleted] notification. The valid value is a comma-separated list of queue names. The default value is iWD_Completed.
  • CancelQueues (optional) – Specifies a list of queue names designated for interaction cancellation. This parameter is used by iWD specific transformation to transform [moved] notification to [TaskCancelled] notification. The valid value is a comma-separated list of queue names. The default value is iWD_Canceled.
  • RestartQueues (optional) – Specifies a list of queue names designated for interaction restart. This parameter is used by iWD specific transformation to transform [moved] notification to [TaskRestarted] notification. The valid value is a comma-separated list of queue names. The default value is iWD_New.
  • ExtendedAttributes (optional) – Specifies a list of attribute names that are expected under the [Ext] tag in [TaskCreated] notification. This parameter is used by iWD specific transformation. The valid value is a comma-separated list of attribute names. There is no default value.

The following options should be configured in the notification-filtering section of the Capture Point application:

  • notify-created (optional) – If the value is set to false or no, the capture point will not store notifications about newly submitted interactions. The default value is set to true.
  • notify-route-requested (optional) – If the value is set to false or no, the capture point will not store notifications about a strategy being added as a party on an interaction. The default value is true.
  • notify-error (optional) – If the value is set to false or no, the capture point will not store notifications about capture point requests resulting in errors. The valid values are true, false, yes, and no. The default value is true.
  • disable-unsolicited-notifications (optional) – If the value is set to true or yes, the capture point will not store any unsolicited notifications about the interactions that are submitted by this capture point. The valid values are true, false, yes, and no. The default value is false.
  • notify-changed (optional) – If the value is set to false or no, the capture point will not store notifications about interaction property changes. The default value is true.
  • notify-moved (optional) – If the value is set to false or no, the capture point will not store notifications about interactions being placed in a queue or workbin. The default value is true.
  • notify-assigned (optional) – If the value is set to false or no, the capture point will not store notifications about an agent being added as a party on an interaction. The default value is true.
  • notify-stopped (optional) – If the value is set to false or no, the capture point will not store notifications about interactions being stopped (terminated). The default value is true.
  • notify-held (optional) – If the value is set to false or no, the capture point will not store notifications about interactions being put on hold. The default value is true.
  • notify-resumed (optional) – If the value is set to false or no, the capture point will not store notifications about interactions being resumed. The default value is true.

Advanced Java Kafka Client Options

Java Kafka client used for communication with the Kafka cluster has many configurable options. The vast majority of these options can be specified in the following Capture Point option sections:

  • consumer-options - Options specified in these sections are passed to the Java Kafka consumers as is. Consumers are used to read inbound message from the Kafka cluster. The list of options can be found on the official page.
  • producer-options - Options specified in these sections are passed to the Java Kafka producers as is. Producers are used to send outbound message to the Kafka cluster. The list of options can be found on the official page.

The following are options that cannot be overridden in the consumer-options and producer-options sections:

  • bootstrap.servers (settings\kafka-server) - The value of kafka-server under the settings section is passed to boostrap.servers as Interaction Server uses the same broker for producer and consumer.
  • group.id - The value of consumer-group under the settings section is passed to group.id and its value is set to InteractionServer by default.
  • enable.auto.commit - The value of this option is hardcoded to false as Interaction Server guarantees “at least once” delivery.

Interaction Server ignores any value that is given for the bootstrap.servers, group.id, and enable.auto.commit options under the producer-options and consumer-options sections of Kafka Capture Point. The remainder of the producer and consumer configuration options of Kafka can be customized by the end user under the producer-options and consumer-options sections of Kafka Capture Point.

Security Configuration

Secured communications are provided by means of the used Java Kafka client library. Refer to official Kafka documentation. Necessary options mentioned in this documentation can be set in the consumer-options and producer-options sections.

Kafka protected options

The following sensitive Kafka client options can be hidden by putting them into the password option in the following sections:

Important
Secure "password" options are limited with 120 symbols. This is an internal limitation in Configuration Server.
Original Section Original Option Protected Section Protected Option
consumer-options ssl.keystore.password consumer-options:ssl.keystore.password password
consumer-options ssl.truststore.password consumer-options:ssl.truststore.password password
consumer-options sasl.jaas.config consumer-options:sasl.jaas.config password
producer-options ssl.keystore.password producer-options:ssl.keystore.password password
producer-options ssl.truststore.password producer-options:ssl.truststore.password password
producer-options sasl.jaas.config producer-options:sasl.jaas.config password
This page was last edited on October 24, 2020, at 09:20.
blog comments powered by Disqus