@InterfaceStability.Unstable
public interface OffsetStorageReader
OffsetStorageReader provides access to the offset storage used by sources. This can be used by connectors to determine offsets to start consuming data from. This is most commonly used during initialization of a task, but can also be used during runtime, e.g. when reconfiguring a task.
Offsets are always defined as Maps of Strings to primitive types, i.e. all types supported by
Schema other than Array, Map, and Struct.
| Modifier and Type | Method and Description |
|---|---|
<T> Map<String,Object> |
offset(Map<String,T> partition)
Get the offset for the specified partition.
|
<T> Map<Map<String,T>,Map<String,Object>> |
offsets(Collection<Map<String,T>> partitions)
Get a set of offsets for the specified partition identifiers.
|
<T> Map<String,Object> offset(Map<String,T> partition)
partition - object uniquely identifying the partition of data<T> Map<Map<String,T>,Map<String,Object>> offsets(Collection<Map<String,T>> partitions)
Get a set of offsets for the specified partition identifiers. This may be more efficient
than calling offset(Map) repeatedly.
Note that when errors occur, this method omits the associated data and tries to return as many of the requested values as possible. This allows a task that's managing many partitions to still proceed with any available data. Therefore, implementations should take care to check that the data is actually available in the returned response. The only case when an exception will be thrown is if the entire request failed, e.g. because the underlying storage was unavailable.
partitions - set of identifiers for partitions of data