Package org.apache.kafka.connect.storage

Interface OffsetStorageReader

public interface OffsetStorageReader

OffsetStorageReader provides access to the offset storage used by sources. This can be used by connectors to determine offsets to start consuming data from. This is most commonly used during initialization of a task, but can also be used during runtime, e.g. when reconfiguring a task.

Offsets are always defined as Maps of Strings to primitive types, i.e. all types supported by Schema other than Array, Map, and Struct.

  • Method Details

    • offset

      <T> Map<String,Object> offset(Map<String,T> partition)
      Get the offset for the specified partition. If the data isn't already available locally, this gets it from the backing store, which may require some network round trips.
      Parameters:
      partition - object uniquely identifying the partition of data
      Returns:
      object uniquely identifying the offset in the partition of data

    • offsets

      <T> Map<Map<String,T>,Map<String,Object>> offsets(Collection<Map<String,T>> partitions)

      Get a set of offsets for the specified partition identifiers. This may be more efficient than calling offset(Map) repeatedly.

      Note that when errors occur, this method omits the associated data and tries to return as many of the requested values as possible. This allows a task that's managing many partitions to still proceed with any available data. Therefore, implementations should take care to check that the data is actually available in the returned response. The only case when an exception will be thrown is if the entire request failed, e.g. because the underlying storage was unavailable.

      Parameters:
      partitions - set of identifiers for partitions of data
      Returns:
      a map of partition identifiers to decoded offsets