org.apache.kafka.connect.transforms.InsertField

Insert field(s) using attributes from the record metadata or a configured static value.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.InsertField$Key) or value (org.apache.kafka.connect.transforms.InsertField$Value).

• offset.field

  Field name for Kafka offset - only applicable to sink connectors.
  Suffix with ! to make this a required field, or ? to keep it optional (the default).

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

• partition.field

  Field name for Kafka partition. Suffix with ! to make this a required field, or ? to keep it optional (the default).

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

• static.field

  Field name for static data field. Suffix with ! to make this a required field, or ? to keep it optional (the default).

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

• static.value

  Static field value, if field name configured.

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

• timestamp.field

  Field name for record timestamp. Suffix with ! to make this a required field, or ? to keep it optional (the default).

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

• topic.field

  Field name for Kafka topic. Suffix with ! to make this a required field, or ? to keep it optional (the default).

  Type:	string
  Default:	null
  Valid Values:
  Importance:	medium

org.apache.kafka.connect.transforms.ReplaceField

Filter or rename fields.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.ReplaceField$Key) or value (org.apache.kafka.connect.transforms.ReplaceField$Value).

• exclude

  Fields to exclude. This takes precedence over the fields to include.

  Type:	list
  Default:	""
  Valid Values:
  Importance:	medium

• include

  Fields to include. If specified, only these fields will be used.

  Type:	list
  Default:	""
  Valid Values:
  Importance:	medium

• renames

  Field rename mappings.

  Type:	list
  Default:	""
  Valid Values:	list of colon-delimited pairs, e.g. foo:bar,abc:xyz
  Importance:	medium

• blacklist

  Deprecated. Use exclude instead.

  Type:	list
  Default:	null
  Valid Values:
  Importance:	low

• whitelist

  Deprecated. Use include instead.

  Type:	list
  Default:	null
  Valid Values:
  Importance:	low

org.apache.kafka.connect.transforms.MaskField

Mask specified fields with a valid null value for the field type (i.e. 0, false, empty string, and so on).

For numeric and string fields, an optional replacement value can be specified that is converted to the correct type.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.MaskField$Key) or value (org.apache.kafka.connect.transforms.MaskField$Value).

• fields

  Names of fields to mask.

  Type:	list
  Default:
  Valid Values:	non-empty list
  Importance:	high

• replacement

  Custom value replacement, that will be applied to all 'fields' values (numeric or non-empty string values only).

  Type:	string
  Default:	null
  Valid Values:	non-empty string
  Importance:	low

org.apache.kafka.connect.transforms.ValueToKey

Replace the record key with a new key formed from a subset of fields in the record value.

• fields

  Field names on the record value to extract as the record key.

  Type:	list
  Default:
  Valid Values:	non-empty list
  Importance:	high

org.apache.kafka.connect.transforms.HoistField

Wrap data using the specified field name in a Struct when schema present, or a Map in the case of schemaless data.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.HoistField$Key) or value (org.apache.kafka.connect.transforms.HoistField$Value).

• field

  Field name for the single field that will be created in the resulting Struct or Map.

  Type:	string
  Default:
  Valid Values:
  Importance:	medium

org.apache.kafka.connect.transforms.ExtractField

Extract the specified field from a Struct when schema present, or a Map in the case of schemaless data. Any null values are passed through unmodified.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.ExtractField$Key) or value (org.apache.kafka.connect.transforms.ExtractField$Value).

• field

  Field name to extract.

  Type:	string
  Default:
  Valid Values:
  Importance:	medium

org.apache.kafka.connect.transforms.SetSchemaMetadata

Set the schema name, version or both on the record's key (org.apache.kafka.connect.transforms.SetSchemaMetadata$Key) or value (org.apache.kafka.connect.transforms.SetSchemaMetadata$Value) schema.

• schema.name

  Schema name to set.

  Type:	string
  Default:	null
  Valid Values:
  Importance:	high

• schema.version

  Schema version to set.

  Type:	int
  Default:	null
  Valid Values:
  Importance:	high

org.apache.kafka.connect.transforms.TimestampRouter

Update the record's topic field as a function of the original topic value and the record timestamp.

This is mainly useful for sink connectors, since the topic field is often used to determine the equivalent entity name in the destination system(e.g. database table or search index name).

• timestamp.format

  Format string for the timestamp that is compatible with java.text.SimpleDateFormat.

  Type:	string
  Default:	yyyyMMdd
  Valid Values:
  Importance:	high

• topic.format

  Format string which can contain ${topic} and ${timestamp} as placeholders for the topic and timestamp, respectively.

  Type:	string
  Default:	${topic}-${timestamp}
  Valid Values:
  Importance:	high

org.apache.kafka.connect.transforms.RegexRouter

Update the record topic using the configured regular expression and replacement string.

Under the hood, the regex is compiled to a java.util.regex.Pattern. If the pattern matches the input topic, java.util.regex.Matcher#replaceFirst() is used with the replacement string to obtain the new topic.

• regex

  Regular expression to use for matching.

  Type:	string
  Default:
  Valid Values:	valid regex
  Importance:	high

• replacement

  Replacement string.

  Type:	string
  Default:
  Valid Values:
  Importance:	high

org.apache.kafka.connect.transforms.Flatten

Flatten a nested data structure, generating names for each field by concatenating the field names at each level with a configurable delimiter character. Applies to Struct when schema present, or a Map in the case of schemaless data. Array fields and their contents are not modified. The default delimiter is '.'.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.Flatten$Key) or value (org.apache.kafka.connect.transforms.Flatten$Value).

• delimiter

  Delimiter to insert between field names from the input record when generating field names for the output record

  Type:	string
  Default:	.
  Valid Values:
  Importance:	medium

org.apache.kafka.connect.transforms.Cast

Cast fields or the entire key or value to a specific type, e.g. to force an integer field to a smaller width. Cast from integers, floats, boolean and string to any other type, and cast binary to string (base64 encoded).

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.Cast$Key) or value (org.apache.kafka.connect.transforms.Cast$Value).

• spec

  List of fields and the type to cast them to of the form field1:type,field2:type to cast fields of Maps or Structs. A single type to cast the entire value. Valid types are int8, int16, int32, int64, float32, float64, boolean, and string. Note that binary fields can only be cast to string.

  Type:	list
  Default:
  Valid Values:	list of colon-delimited pairs, e.g. foo:bar,abc:xyz
  Importance:	high

org.apache.kafka.connect.transforms.TimestampConverter

Convert timestamps between different formats such as Unix epoch, strings, and Connect Date/Timestamp types.Applies to individual fields or to the entire value.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.TimestampConverter$Key) or value (org.apache.kafka.connect.transforms.TimestampConverter$Value).

• target.type

  The desired timestamp representation: string, unix, Date, Time, or Timestamp

  Type:	string
  Default:
  Valid Values:
  Importance:	high

• field

  The field containing the timestamp, or empty if the entire value is a timestamp

  Type:	string
  Default:	""
  Valid Values:
  Importance:	high

• format

  A SimpleDateFormat-compatible format for the timestamp. Used to generate the output when type=string or used to parse the input if the input is a string.

  Type:	string
  Default:	""
  Valid Values:
  Importance:	medium

org.apache.kafka.connect.transforms.Filter

Drops all records, filtering them from subsequent transformations in the chain. This is intended to be used conditionally to filter out records matching (or not matching) a particular Predicate.

org.apache.kafka.connect.transforms.InsertHeader

Add a header to each record.

• header

  The name of the header.

  Type:	string
  Default:
  Valid Values:	non-null string
  Importance:	high

• value.literal

  The literal value that is to be set as the header value on all records.

  Type:	string
  Default:
  Valid Values:	non-null string
  Importance:	high

org.apache.kafka.connect.transforms.DropHeaders

Removes one or more headers from each record.

• headers

  The name of the headers to be removed.

  Type:	list
  Default:
  Valid Values:	non-empty list
  Importance:	high

org.apache.kafka.connect.transforms.HeaderFrom

Moves or copies fields in the key/value of a record into that record's headers. Corresponding elements of fields and headers together identify a field and the header it should be moved or copied to. Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.HeaderFrom$Key) or value (org.apache.kafka.connect.transforms.HeaderFrom$Value).

• fields

  Field names in the record whose values are to be copied or moved to headers.

  Type:	list
  Default:
  Valid Values:	non-empty list
  Importance:	high

• headers

  Header names, in the same order as the field names listed in the fields configuration property.

  Type:	list
  Default:
  Valid Values:	non-empty list
  Importance:	high

• operation

  Either move if the fields are to be moved to the headers (removed from the key/value), or copy if the fields are to be copied to the headers (retained in the key/value).

  Type:	string
  Default:
  Valid Values:	[move, copy]
  Importance:	high