Class Connector
- All Implemented Interfaces:
Versioned
- Direct Known Subclasses:
SinkConnector
,SourceConnector
Connectors manage integration of Kafka Connect with another system, either as an input that ingests
data into Kafka or an output that passes data to an external system. Implementations should
not use this class directly; they should inherit from SourceConnector
or SinkConnector
.
Connectors have two primary tasks. First, given some configuration, they are responsible for
creating configurations for a set of Task
s that split up the data processing. For
example, a database Connector might create Tasks by dividing the set of tables evenly among
tasks. Second, they are responsible for monitoring inputs for changes that require
reconfiguration and notifying the Kafka Connect runtime via the ConnectorContext
. Continuing the
previous example, the connector might periodically check for new tables and notify Kafka Connect of
additions and deletions. Kafka Connect will then request new configurations and update the running
Tasks.
-
Field Summary
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionabstract ConfigDef
config()
Define the configuration for the connector.protected ConnectorContext
context()
Returns the context object used to interact with the Kafka Connect runtime.void
Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.void
initialize
(ConnectorContext ctx, List<Map<String, String>> taskConfigs) Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations.void
reconfigure
(Map<String, String> props) Reconfigure this Connector.abstract void
Start this Connector.abstract void
stop()
Stop this connector.Returns the Task implementation for this Connector.taskConfigs
(int maxTasks) Returns a set of configurations for Tasks based on the current configuration, producing at most count configurations.Validate the connector configuration values against configuration definitions.
-
Field Details
-
context
-
-
Constructor Details
-
Connector
public Connector()
-
-
Method Details
-
initialize
Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.- Parameters:
ctx
- context object used to interact with the Kafka Connect runtime
-
initialize
Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations. This version is only used to recover from failures.
The default implementation ignores the provided Task configurations. During recovery, Kafka Connect will request an updated set of configurations and update the running Tasks appropriately. However, Connectors should implement special handling of this case if it will avoid unnecessary changes to running Tasks.
- Parameters:
ctx
- context object used to interact with the Kafka Connect runtimetaskConfigs
- existing task configurations, which may be used when generating new task configs to avoid churn in partition to task assignments
-
context
Returns the context object used to interact with the Kafka Connect runtime.- Returns:
- the context for this Connector.
-
start
Start this Connector. This method will only be called on a clean Connector, i.e. it has either just been instantiated and initialized orstop()
has been invoked.- Parameters:
props
- configuration settings
-
reconfigure
Reconfigure this Connector. Most implementations will not override this, using the default implementation that callsstop()
followed bystart(Map)
. Implementations only need to override this if they want to handle this process more efficiently, e.g. without shutting down network connections to the external system.- Parameters:
props
- new configuration settings
-
taskClass
Returns the Task implementation for this Connector. -
taskConfigs
Returns a set of configurations for Tasks based on the current configuration, producing at most count configurations.- Parameters:
maxTasks
- maximum number of configurations to generate- Returns:
- configurations for Tasks
-
stop
public abstract void stop()Stop this connector. -
validate
Validate the connector configuration values against configuration definitions.- Parameters:
connectorConfigs
- the provided configuration values- Returns:
- List of Config, each Config contains the updated configuration information given the current configuration values.
-
config
Define the configuration for the connector.- Returns:
- The ConfigDef for this connector; may not be null.
-