Package org.apache.kafka.connect.connector

Class Connector

java.lang.Object
org.apache.kafka.connect.connector.Connector
All Implemented Interfaces:
Versioned
Direct Known Subclasses:
MockConnector, SinkConnector, SourceConnector
public abstract class Connector extends Object implements Versioned

Connectors manage integration of Kafka Connect with another system, either as an input that ingests data into Kafka or an output that passes data to an external system. Implementations should not use this class directly; they should inherit from SourceConnector or SinkConnector.

Connectors have two primary roles. First, given some configuration, they are responsible for creating configurations for a set of Tasks that split up the data processing. For example, a database Connector might create Tasks by dividing the set of tables evenly among tasks. Second, they are responsible for monitoring inputs for changes that require reconfiguration and notifying the Kafka Connect runtime via the ConnectorContext. Continuing the previous example, the connector might periodically check for new tables and notify Kafka Connect of additions and deletions. Kafka Connect will then request new configurations and update the running Tasks.

  • Field Details

  • Constructor Details

    • Connector

      public Connector()

  • Method Details

    • initialize

      public void initialize(ConnectorContext ctx)
      Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.
      Parameters:
      ctx - context object used to interact with the Kafka Connect runtime

    • initialize

      public void initialize(ConnectorContext ctx, List<Map<String,String>> taskConfigs)

      Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations. This version is only used to recover from failures.

      The default implementation ignores the provided Task configurations. During recovery, Kafka Connect will request an updated set of configurations and update the running Tasks appropriately. However, Connectors should implement special handling of this case if it will avoid unnecessary changes to running Tasks.

      Parameters:
      ctx - context object used to interact with the Kafka Connect runtime
      taskConfigs - existing task configurations, which may be used when generating new task configs to avoid churn in partition to task assignments

    • context

      protected ConnectorContext context()
      Returns the context object used to interact with the Kafka Connect runtime.
      Returns:
      the context for this Connector.

    • start

      public abstract void start(Map<String,String> props)
      Start this Connector. This method will only be called on a clean Connector, i.e. it has either just been instantiated and initialized or stop() has been invoked.
      Parameters:
      props - configuration settings

    • reconfigure

      public void reconfigure(Map<String,String> props)
      Reconfigure this Connector. Most implementations will not override this, using the default implementation that calls stop() followed by start(Map). Implementations only need to override this if they want to handle this process more efficiently, e.g. without shutting down network connections to the external system.
      Parameters:
      props - new configuration settings

    • taskClass

      public abstract Class<? extends Task> taskClass()
      Returns the Task implementation for this Connector.

    • taskConfigs

      public abstract List<Map<String,String>> taskConfigs(int maxTasks)
      Returns a set of configurations for Tasks based on the current configuration, producing at most maxTasks configurations.
      Parameters:
      maxTasks - maximum number of configurations to generate
      Returns:
      configurations for Tasks

    • stop

      public abstract void stop()
      Stop this connector.

    • validate

      public Config validate(Map<String,String> connectorConfigs)
      Validate the connector configuration values against configuration definitions.
      Parameters:
      connectorConfigs - the provided configuration values
      Returns:
      a parsed and validated Config containing any relevant validation errors with the raw connectorConfigs which should prevent this configuration from being used.

    • config

      public abstract ConfigDef config()
      Define the configuration for the connector.
      Returns:
      The ConfigDef for this connector; may not be null.