Package org.apache.kafka.connect.connector

Class Connector

  • All Implemented Interfaces:
    Versioned
    Direct Known Subclasses:
    SinkConnector, SourceConnector
    public abstract class Connector
extends Object
implements Versioned

    Connectors manage integration of Kafka Connect with another system, either as an input that ingests data into Kafka or an output that passes data to an external system. Implementations should not use this class directly; they should inherit from SourceConnector or SinkConnector.

    Connectors have two primary tasks. First, given some configuration, they are responsible for creating configurations for a set of Tasks that split up the data processing. For example, a database Connector might create Tasks by dividing the set of tables evenly among tasks. Second, they are responsible for monitoring inputs for changes that require reconfiguration and notifying the Kafka Connect runtime via the ConnectorContext. Continuing the previous example, the connector might periodically check for new tables and notify Kafka Connect of additions and deletions. Kafka Connect will then request new configurations and update the running Tasks.

    • Constructor Detail

      • Connector

        public Connector()

    • Method Detail

      • initialize

        public void initialize​(ConnectorContext ctx)
        Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes.
        Parameters:
        ctx - context object used to interact with the Kafka Connect runtime

      • initialize

        public void initialize​(ConnectorContext ctx,
                       List<Map<String,​String>> taskConfigs)

        Initialize this connector, using the provided ConnectorContext to notify the runtime of input configuration changes and using the provided set of Task configurations. This version is only used to recover from failures.

        The default implementation ignores the provided Task configurations. During recovery, Kafka Connect will request an updated set of configurations and update the running Tasks appropriately. However, Connectors should implement special handling of this case if it will avoid unnecessary changes to running Tasks.

        Parameters:
        ctx - context object used to interact with the Kafka Connect runtime
        taskConfigs - existing task configurations, which may be used when generating new task configs to avoid churn in partition to task assignments

      • context

        protected ConnectorContext context()
        Returns the context object used to interact with the Kafka Connect runtime.
        Returns:
        the context for this Connector.

      • start

        public abstract void start​(Map<String,​String> props)
        Start this Connector. This method will only be called on a clean Connector, i.e. it has either just been instantiated and initialized or stop() has been invoked.
        Parameters:
        props - configuration settings

      • reconfigure

        public void reconfigure​(Map<String,​String> props)
        Reconfigure this Connector. Most implementations will not override this, using the default implementation that calls stop() followed by start(Map). Implementations only need to override this if they want to handle this process more efficiently, e.g. without shutting down network connections to the external system.
        Parameters:
        props - new configuration settings

      • taskClass

        public abstract Class<? extends Task> taskClass()
        Returns the Task implementation for this Connector.

      • taskConfigs

        public abstract List<Map<String,​String>> taskConfigs​(int maxTasks)
        Returns a set of configurations for Tasks based on the current configuration, producing at most count configurations.
        Parameters:
        maxTasks - maximum number of configurations to generate
        Returns:
        configurations for Tasks

      • stop

        public abstract void stop()
        Stop this connector.

      • validate

        public Config validate​(Map<String,​String> connectorConfigs)
        Validate the connector configuration values against configuration definitions.
        Parameters:
        connectorConfigs - the provided configuration values
        Returns:
        List of Config, each Config contains the updated configuration information given the current configuration values.

      • config

        public abstract ConfigDef config()
        Define the configuration for the connector.
        Returns:
        The ConfigDef for this connector; may not be null.