org.apache.kafka.connect.connector

Class Connector

java.lang.Object

org.apache.kafka.connect.connector.Connector

All Implemented Interfaces:

Versioned

Direct Known Subclasses:

MockConnector, SinkConnector, SourceConnector

public abstract class Connector
extends Object
implements Versioned

Connectors manage integration of Kafka Connect with another system, either as an input that ingests data into Kafka or an output that passes data to an external system. Implementations should not use this class directly; they should inherit from SourceConnector or SinkConnector.

Connectors have two primary roles. First, given some configuration, they are responsible for creating configurations for a set of Tasks that split up the data processing. For example, a database Connector might create Tasks by dividing the set of tables evenly among tasks. Second, they are responsible for monitoring inputs for changes that require reconfiguration and notifying the Kafka Connect runtime via the ConnectorContext. Continuing the previous example, the connector might periodically check for new tables and notify Kafka Connect of additions and deletions. Kafka Connect will then request new configurations and update the running Tasks.

Field Summary

Fields

Modifier and Type	Field and Description
protected ConnectorContext	context

Constructor Summary

Constructors

Constructor and Description
Connector()

Method Summary

Modifier and Type	Method and Description
abstract ConfigDef	config() Define the configuration for the connector.
protected ConnectorContext	context() Returns the context object used to interact with the Kafka Connect runtime.
void	initialize(ConnectorContext ctx) Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.
void	initialize(ConnectorContext ctx, List<Map<String,String>> taskConfigs) Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations.
void	reconfigure(Map<String,String> props) Reconfigure this Connector.
abstract void	start(Map<String,String> props) Start this Connector.
abstract void	stop() Stop this connector.
abstract Class<? extends Task>	taskClass() Returns the Task implementation for this Connector.
abstract List<Map<String,String>>	taskConfigs(int maxTasks) Returns a set of configurations for Tasks based on the current configuration, producing at most maxTasks configurations.
Config	validate(Map<String,String> connectorConfigs) Validate the connector configuration values against configuration definitions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.kafka.connect.components.Versioned

version

Field Detail

context

protected ConnectorContext context

Constructor Detail

Connector

public Connector()

Method Detail

initialize

public void initialize(ConnectorContext ctx)

Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.

Parameters:

ctx - context object used to interact with the Kafka Connect runtime

initialize

public void initialize(ConnectorContext ctx,
                       List<Map<String,String>> taskConfigs)

Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations. This version is only used to recover from failures.

The default implementation ignores the provided Task configurations. During recovery, Kafka Connect will request an updated set of configurations and update the running Tasks appropriately. However, Connectors should implement special handling of this case if it will avoid unnecessary changes to running Tasks.

Parameters:

ctx - context object used to interact with the Kafka Connect runtime

taskConfigs - existing task configurations, which may be used when generating new task configs to avoid churn in partition to task assignments

context

protected ConnectorContext context()

Returns the context object used to interact with the Kafka Connect runtime.

Returns:

the context for this Connector.

start

public abstract void start(Map<String,String> props)

Start this Connector. This method will only be called on a clean Connector, i.e. it has either just been instantiated and initialized or stop() has been invoked.

Parameters:

props - configuration settings

reconfigure

public void reconfigure(Map<String,String> props)

Reconfigure this Connector. Most implementations will not override this, using the default implementation that calls stop() followed by start(Map). Implementations only need to override this if they want to handle this process more efficiently, e.g. without shutting down network connections to the external system.

Parameters:

props - new configuration settings

taskClass

public abstract Class<? extends Task> taskClass()

Returns the Task implementation for this Connector.

taskConfigs

public abstract List<Map<String,String>> taskConfigs(int maxTasks)

Returns a set of configurations for Tasks based on the current configuration, producing at most maxTasks configurations.

Parameters:

maxTasks - maximum number of configurations to generate

Returns:

configurations for Tasks

stop

public abstract void stop()

Stop this connector.

validate

public Config validate(Map<String,String> connectorConfigs)

Validate the connector configuration values against configuration definitions.

Parameters:

connectorConfigs - the provided configuration values

Returns:

a parsed and validated Config containing any relevant validation errors with the raw connectorConfigs which should prevent this configuration from being used.

config

public abstract ConfigDef config()

Define the configuration for the connector.

Returns:

The ConfigDef for this connector; may not be null.