org.apache.kafka.streams.processor

Interface PartitionGrouper

All Known Implementing Classes:

DefaultPartitionGrouper

public interface PartitionGrouper

A partition grouper that generates partition groups given the list of topic-partitions. This grouper also acts as the stream task creation function along with partition distribution such that each generated partition group is assigned with a distinct TaskId; the created task ids will then be assigned to Kafka Streams instances that host the stream processing application.

Method Summary

Methods

Modifier and Type	Method and Description
Map<TaskId,Set<TopicPartition>>	partitionGroups(Map<Integer,Set<String>> topicGroups, Cluster metadata) Returns a map of task ids to groups of partitions.

Method Detail

partitionGroups

Map<TaskId,Set<TopicPartition>> partitionGroups(Map<Integer,Set<String>> topicGroups,
                                              Cluster metadata)

Returns a map of task ids to groups of partitions. A partition group forms a task, thus, partitions that are expected to be processed together must be in the same group. Note that the grouping of partitions need to be sticky such that for a given partition, its assigned task should always be the same regardless of the input parameters to this function. This is to ensure task's local state stores remain valid through workload rebalances among Kafka Streams instances. The default partition grouper implements this interface by assigning all partitions across different topics with the same partition id into the same task. See DefaultPartitionGrouper for more information.

Parameters:

topicGroups - The map from the TopologyBuilder.topicGroups(String) topic group} id to topics

metadata - Metadata of the consuming cluster

Returns:

a map of task ids to groups of partitions