Class SourceRecord


public class SourceRecord
extends ConnectRecord<SourceRecord>

SourceRecords are generated by SourceTasks and passed to Kafka Connect for storage in Kafka. In addition to the standard fields in ConnectRecord which specify where data is stored in Kafka, they also include a sourcePartition and sourceOffset.

The sourcePartition represents a single input sourcePartition that the record came from (e.g. a filename, table name, or topic-partition). The sourceOffset represents a position in that sourcePartition which can be used to resume consumption of data.

These values can have arbitrary structure and should be represented using org.apache.kafka.connect.data objects (or primitive values). For example, a database connector might specify the sourcePartition as a record containing { "db": "database_name", "table": "table_name"} and the sourceOffset as a Long containing the timestamp of the row.

  • Constructor Details

  • Method Details

    • sourcePartition

      public Map<String,​?> sourcePartition()
    • sourceOffset

      public Map<String,​?> sourceOffset()
    • newRecord

      public SourceRecord newRecord​(String topic, Integer kafkaPartition, Schema keySchema, Object key, Schema valueSchema, Object value, Long timestamp)
      Description copied from class: ConnectRecord
      Create a new record of the same type as itself, with the specified parameter values. All other fields in this record will be copied over to the new record. Since the headers are mutable, the resulting record will have a copy of this record's headers.
      Specified by:
      newRecord in class ConnectRecord<SourceRecord>
      Parameters:
      topic - the name of the topic; may be null
      kafkaPartition - the partition number for the Kafka topic; may be null
      keySchema - the schema for the key; may be null
      key - the key; may be null
      valueSchema - the schema for the value; may be null
      value - the value; may be null
      timestamp - the timestamp; may be null
      Returns:
      the new record
    • newRecord

      public SourceRecord newRecord​(String topic, Integer kafkaPartition, Schema keySchema, Object key, Schema valueSchema, Object value, Long timestamp, Iterable<Header> headers)
      Description copied from class: ConnectRecord
      Create a new record of the same type as itself, with the specified parameter values. All other fields in this record will be copied over to the new record.
      Specified by:
      newRecord in class ConnectRecord<SourceRecord>
      Parameters:
      topic - the name of the topic; may be null
      kafkaPartition - the partition number for the Kafka topic; may be null
      keySchema - the schema for the key; may be null
      key - the key; may be null
      valueSchema - the schema for the value; may be null
      value - the value; may be null
      timestamp - the timestamp; may be null
      headers - the headers; may be null or empty
      Returns:
      the new record
    • equals

      public boolean equals​(Object o)
      Overrides:
      equals in class ConnectRecord<SourceRecord>
    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class ConnectRecord<SourceRecord>
    • toString

      public String toString()
      Overrides:
      toString in class ConnectRecord<SourceRecord>