org.apache.kafka.connect.source

Class SourceTask

java.lang.Object

org.apache.kafka.connect.source.SourceTask

All Implemented Interfaces:

Task

public abstract class SourceTask
extends Object
implements Task

SourceTask is a Task that pulls records from another system for storage in Kafka.

Field Summary

Fields

Modifier and Type	Field and Description
protected SourceTaskContext	context

Constructor Summary

Constructors

Constructor and Description
SourceTask()

Method Summary

Methods

Modifier and Type	Method and Description
void	commit() Commit the offsets, up to the offsets that have been returned by poll().
void	commitRecord(SourceRecord record) Commit an individual SourceRecord when the callback from the producer client is received, or if a record is filtered by a transformation.
void	initialize(SourceTaskContext context) Initialize this SourceTask with the specified context object.
abstract List<SourceRecord>	poll() Poll this SourceTask for new records.
abstract void	start(Map<String,String> props) Start the Task.
abstract void	stop() Signal this SourceTask to stop.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.kafka.connect.connector.Task

version

Field Detail

context

protected SourceTaskContext context

Constructor Detail

SourceTask

public SourceTask()

Method Detail

initialize

public void initialize(SourceTaskContext context)

Initialize this SourceTask with the specified context object.

start

public abstract void start(Map<String,String> props)

Start the Task. This should handle any configuration parsing and one-time setup of the task.

Specified by:

start in interface Task

Parameters:

props - initial configuration

poll

public abstract List<SourceRecord> poll()
                                 throws InterruptedException

Poll this SourceTask for new records. This method should block if no data is currently available.

Returns:

a list of source records

Throws:

InterruptedException

commit

public void commit()
            throws InterruptedException

Commit the offsets, up to the offsets that have been returned by poll(). This method should block until the commit is complete.

SourceTasks are not required to implement this functionality; Kafka Connect will record offsets automatically. This hook is provided for systems that also need to store offsets internally in their own system.

Throws:

InterruptedException

stop

public abstract void stop()

Signal this SourceTask to stop. In SourceTasks, this method only needs to signal to the task that it should stop trying to poll for new data and interrupt any outstanding poll() requests. It is not required that the task has fully stopped. Note that this method necessarily may be invoked from a different thread than poll() and commit(). For example, if a task uses a Selector to receive data over the network, this method could set a flag that will force poll() to exit immediately and invoke wakeup() to interrupt any ongoing requests.

Specified by:

stop in interface Task

commitRecord

public void commitRecord(SourceRecord record)
                  throws InterruptedException

Commit an individual SourceRecord when the callback from the producer client is received, or if a record is filtered by a transformation.

SourceTasks are not required to implement this functionality; Kafka Connect will record offsets automatically. This hook is provided for systems that also need to store offsets internally in their own system.

Parameters:

record - SourceRecord that was successfully sent via the producer.

Throws:

InterruptedException