https://kafka.apache.org/43/configuration/Recent content in Configuration on Apache KafkaHugo -- gohugo.ioenhttps://kafka.apache.org/43/configuration/broker-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/broker-configs/The essential configurations are the following: node.id log.dirs process.roles controller.quorum.bootstrap.servers controller.listener.names Broker configurations and defaults are discussed in more detail below. controller.listener.names A comma-separated list of the names of the listeners used by the controller. This is required when communicating with the controller quorum, the broker will always use the first listener in this list. Type:list Default: Valid Values: Importance:high Update Mode:read-only node.id The node ID associated with the roles this process is playing when process.https://kafka.apache.org/43/configuration/topic-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/topic-configs/Configurations pertinent to topics have both a server default as well an optional per-topic override. If no per-topic configuration is given the server default is used. The override can be set at topic creation time by giving one or more --config options. This example creates a topic named my-topic with a custom max message size and flush rate: $ bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic my-topic --partitions 1 \ --replication-factor 1 --config max.https://kafka.apache.org/43/configuration/group-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/group-configs/Below is the group configuration: consumer.assignment.interval.ms The interval between assignment updates for a consumer group. Type:int Default:1000 (1 second) Valid Values:[0,...] Importance:medium consumer.heartbeat.interval.ms The heartbeat interval given to the members of a consumer group. Type:int Default:5000 (5 seconds) Valid Values:[1,...] Importance:medium consumer.session.timeout.ms The timeout to detect client failures when using the consumer group protocol. Type:int Default:45000 (45 seconds) Valid Values:[1,...] Importance:medium share.assignment.interval.ms The interval between assignment updates for a share group.https://kafka.apache.org/43/configuration/producer-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/producer-configs/Below is the producer configuration: bootstrap.servers A list of host/port pairs used to establish the initial connection to the Kafka cluster. Clients use this list to bootstrap and discover the full set of Kafka brokers. While the order of servers in the list does not matter, we recommend including more than one server to ensure resilience if any servers are down. This list does not need to contain the entire set of brokers, as Kafka clients automatically manage and update connections to the cluster efficiently.https://kafka.apache.org/43/configuration/consumer-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/consumer-configs/Below is the consumer and share consumer configuration: bootstrap.servers A list of host/port pairs used to establish the initial connection to the Kafka cluster. Clients use this list to bootstrap and discover the full set of Kafka brokers. While the order of servers in the list does not matter, we recommend including more than one server to ensure resilience if any servers are down. This list does not need to contain the entire set of brokers, as Kafka clients automatically manage and update connections to the cluster efficiently.https://kafka.apache.org/43/configuration/kafka-connect-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/kafka-connect-configs/Below is the Kafka Connect framework configuration. bootstrap.servers A list of host/port pairs used to establish the initial connection to the Kafka cluster. Clients use this list to bootstrap and discover the full set of Kafka brokers. While the order of servers in the list does not matter, we recommend including more than one server to ensure resilience if any servers are down. This list does not need to contain the entire set of brokers, as Kafka clients automatically manage and update connections to the cluster efficiently.https://kafka.apache.org/43/configuration/kafka-streams-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/kafka-streams-configs/Below is the Kafka Streams client library configuration. application.id An identifier for the stream processing application. Must be unique within the Kafka cluster. It is used as 1) the default client-id prefix, 2) the group-id for membership management, 3) the changelog topic prefix. Type:string Default: Valid Values: Importance:high bootstrap.servers A list of host/port pairs used to establish the initial connection to the Kafka cluster. Clients use this list to bootstrap and discover the full set of Kafka brokers.https://kafka.apache.org/43/configuration/admin-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/admin-configs/Below is the Kafka Admin client library configuration. bootstrap.controllers A list of host/port pairs to use for establishing the initial connection to the KRaft controller quorum. This list should be in the form host1:port1,host2:port2,.... Type:list Default:"" Valid Values: Importance:high bootstrap.servers A list of host/port pairs used to establish the initial connection to the Kafka cluster. Clients use this list to bootstrap and discover the full set of Kafka brokers. While the order of servers in the list does not matter, we recommend including more than one server to ensure resilience if any servers are down.https://kafka.apache.org/43/configuration/mirrormaker-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/mirrormaker-configs/Below is the configuration of the connectors that make up MirrorMaker 2. MirrorMaker Common Configs Below is the common configuration that applies to all three connectors. source.cluster.alias Alias of source cluster Type:string Default: Valid Values: Importance:high ssl.key.password The password of the private key in the key store file or the PEM key specified in 'ssl.keystore.key'. Type:password Default:null Valid Values: Importance:high ssl.keystore.certificate.chain Certificate chain in the format specified by 'ssl.keystore.type'. Default SSL engine factory supports only PEM format with a list of X.https://kafka.apache.org/43/configuration/system-properties/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/system-properties/Kafka supports some configuration that can be enabled through Java system properties. System properties are usually set by passing the -D flag to the Java virtual machine in which Kafka components are running. Below are the supported system properties. org.apache.kafka.sasl.oauthbearer.allowed.files This system property is used to determine which files, if any, are allowed to be read by the SASL OAUTHBEARER plugin. This property accepts comma-separated list of files. By default the value is an empty list.https://kafka.apache.org/43/configuration/tiered-storage-configs/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/tiered-storage-configs/Below is the Tiered Storage configuration. log.local.retention.bytes The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. Default value is -2, it represents log.retention.bytes value to be used. The effective value should always be less than or equal to log.retention.bytes value. Type:long Default:-2 Valid Values:[-2,...] Importance:medium log.local.retention.ms The number of milliseconds to keep the local log segments before it gets eligible for deletion.https://kafka.apache.org/43/configuration/configuration-providers/Mon, 01 Jan 0001 00:00:00 +0000https://kafka.apache.org/43/configuration/configuration-providers/Use configuration providers to load configuration data from external sources. This might include sensitive information, such as passwords, API keys, or other credentials. You have the following options: Use a custom provider by creating a class implementing the ConfigProvider interface and packaging it into a JAR file. Use a built-in provider: DirectoryConfigProvider EnvVarConfigProvider FileConfigProvider To use a configuration provider, specify it in your configuration using the config.providers property. Using Configuration Providers Configuration providers allow you to pass parameters and retrieve configuration data from various sources.