public abstract class Connector extends Object implements Versioned
Connectors manage integration of Kafka Connect with another system, either as an input that ingests
data into Kafka or an output that passes data to an external system. Implementations should
not use this class directly; they should inherit from SourceConnector
or SinkConnector.
Connectors have two primary tasks. First, given some configuration, they are responsible for
creating configurations for a set of Tasks that split up the data processing. For
example, a database Connector might create Tasks by dividing the set of tables evenly among
tasks. Second, they are responsible for monitoring inputs for changes that require
reconfiguration and notifying the Kafka Connect runtime via the ConnectorContext. Continuing the
previous example, the connector might periodically check for new tables and notify Kafka Connect of
additions and deletions. Kafka Connect will then request new configurations and update the running
Tasks.
| Modifier and Type | Field and Description |
|---|---|
protected ConnectorContext |
context |
| Constructor and Description |
|---|
Connector() |
| Modifier and Type | Method and Description |
|---|---|
abstract ConfigDef |
config()
Define the configuration for the connector.
|
void |
initialize(ConnectorContext ctx)
Initialize this connector, using the provided ConnectorContext to notify the runtime of
input configuration changes.
|
void |
initialize(ConnectorContext ctx,
List<Map<String,String>> taskConfigs)
Initialize this connector, using the provided ConnectorContext to notify the runtime of
input configuration changes and using the provided set of Task configurations.
|
void |
reconfigure(Map<String,String> props)
Reconfigure this Connector.
|
abstract void |
start(Map<String,String> props)
Start this Connector.
|
abstract void |
stop()
Stop this connector.
|
abstract Class<? extends Task> |
taskClass()
Returns the Task implementation for this Connector.
|
abstract List<Map<String,String>> |
taskConfigs(int maxTasks)
Returns a set of configurations for Tasks based on the current configuration,
producing at most count configurations.
|
Config |
validate(Map<String,String> connectorConfigs)
Validate the connector configuration values against configuration definitions.
|
protected ConnectorContext context
public void initialize(ConnectorContext ctx)
ctx - context object used to interact with the Kafka Connect runtimepublic void initialize(ConnectorContext ctx, List<Map<String,String>> taskConfigs)
Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations. This version is only used to recover from failures.
The default implementation ignores the provided Task configurations. During recovery, Kafka Connect will request an updated set of configurations and update the running Tasks appropriately. However, Connectors should implement special handling of this case if it will avoid unnecessary changes to running Tasks.
ctx - context object used to interact with the Kafka Connect runtimetaskConfigs - existing task configurations, which may be used when generating new task configs to avoid
churn in partition to task assignmentspublic abstract void start(Map<String,String> props)
stop() has been invoked.props - configuration settingspublic void reconfigure(Map<String,String> props)
stop() followed by start(Map).
Implementations only need to override this if they want to handle this process more
efficiently, e.g. without shutting down network connections to the external system.props - new configuration settingspublic abstract Class<? extends Task> taskClass()
public abstract List<Map<String,String>> taskConfigs(int maxTasks)
maxTasks - maximum number of configurations to generatepublic abstract void stop()
public Config validate(Map<String,String> connectorConfigs)
connectorConfigs - the provided configuration valuespublic abstract ConfigDef config()