org.apache.kafka.connect.sink

Interface SinkTaskContext

public interface SinkTaskContext

Context passed to SinkTasks, allowing them to access utilities in the Kafka Connect runtime.

Method Summary

Modifier and Type	Method and Description
java.util.Set<TopicPartition>	assignment() Get the current set of assigned TopicPartitions for this task.
void	offset(java.util.Map<TopicPartition,java.lang.Long> offsets) Reset the consumer offsets for the given topic partitions.
void	offset(TopicPartition tp, long offset) Reset the consumer offsets for the given topic partition.
void	pause(TopicPartition... partitions) Pause consumption of messages from the specified TopicPartitions.
void	requestCommit() Request an offset commit.
void	resume(TopicPartition... partitions) Resume consumption of messages from previously paused TopicPartitions.
void	timeout(long timeoutMs) Set the timeout in milliseconds.

Method Detail

offset

void offset(java.util.Map<TopicPartition,java.lang.Long> offsets)

Reset the consumer offsets for the given topic partitions. SinkTasks should use this if they manage offsets in the sink data store rather than using Kafka consumer offsets. For example, an HDFS connector might record offsets in HDFS to provide exactly once delivery. When the SinkTask is started or a rebalance occurs, the task would reload offsets from HDFS and use this method to reset the consumer to those offsets. SinkTasks that do not manage their own offsets do not need to use this method.

Parameters:

offsets - map of offsets for topic partitions

offset

void offset(TopicPartition tp,
            long offset)

Reset the consumer offsets for the given topic partition. SinkTasks should use if they manage offsets in the sink data store rather than using Kafka consumer offsets. For example, an HDFS connector might record offsets in HDFS to provide exactly once delivery. When the topic partition is recovered the task would reload offsets from HDFS and use this method to reset the consumer to the offset. SinkTasks that do not manage their own offsets do not need to use this method.

Parameters:

tp - the topic partition to reset offset.

offset - the offset to reset to.

timeout

void timeout(long timeoutMs)

Set the timeout in milliseconds. SinkTasks should use this to indicate that they need to retry certain operations after the timeout. SinkTasks may have certain operations on external systems that may need to retry in case of failures. For example, append a record to an HDFS file may fail due to temporary network issues. SinkTasks use this method to set how long to wait before retrying.

Parameters:

timeoutMs - the backoff timeout in milliseconds.

assignment

java.util.Set<TopicPartition> assignment()

Get the current set of assigned TopicPartitions for this task.

Returns:

the set of currently assigned TopicPartitions

pause

void pause(TopicPartition... partitions)

Pause consumption of messages from the specified TopicPartitions.

Parameters:

partitions - the partitions which should be paused

resume

void resume(TopicPartition... partitions)

Resume consumption of messages from previously paused TopicPartitions.

Parameters:

partitions - the partitions to resume

requestCommit

void requestCommit()

Request an offset commit. Sink tasks can use this to minimize the potential for redelivery by requesting an offset commit as soon as they flush data to the destination system. It is only a hint to the runtime and no timing guarantee should be assumed.