Package org.apache.kafka.streams.state

Class Stores

java.lang.Object

org.apache.kafka.streams.state.Stores

public final class Stores
extends Object

Factory for creating state stores in Kafka Streams.

When using the high-level DSL, i.e., StreamsBuilder, users create StoreSuppliers that can be further customized via Materialized. For example, a topic read as KTable can be materialized into an in-memory store with custom key/value serdes and caching disabled:

 StreamsBuilder builder = new StreamsBuilder();
 KeyValueBytesStoreSupplier storeSupplier = Stores.inMemoryKeyValueStore("queryable-store-name");
 KTable<Long,String> table = builder.table(
   "topicName",
   Materialized.<Long,String>as(storeSupplier)
               .withKeySerde(Serdes.Long())
               .withValueSerde(Serdes.String())
               .withCachingDisabled());
 

When using the Processor API, i.e., Topology, users create StoreBuilders that can be attached to Processors. For example, you can create a windowed RocksDB store with custom changelog topic configuration like:

 Topology topology = new Topology();
 topology.addProcessor("processorName", ...);

 Map<String,String> topicConfig = new HashMap<>();
 StoreBuilder<WindowStore<Integer, Long>> storeBuilder = Stores
   .windowStoreBuilder(
     Stores.persistentWindowStore("queryable-store-name", ...),
     Serdes.Integer(),
     Serdes.Long())
   .withLoggingEnabled(topicConfig);

 topology.addStateStore(storeBuilder, "processorName");
 

Constructor Summary

Constructors

Constructor	Description
Stores()

Method Summary

Modifier and Type	Method	Description
static KeyValueBytesStoreSupplier	inMemoryKeyValueStore(String name)	Create an in-memory KeyValueBytesStoreSupplier.
static SessionBytesStoreSupplier	inMemorySessionStore(String name, Duration retentionPeriod)	Create an in-memory SessionBytesStoreSupplier.
static WindowBytesStoreSupplier	inMemoryWindowStore(String name, Duration retentionPeriod, Duration windowSize, boolean retainDuplicates)	Create an in-memory WindowBytesStoreSupplier.
static <K,V> StoreBuilder<KeyValueStore<K,V>>	keyValueStoreBuilder(KeyValueBytesStoreSupplier supplier, Serde<K> keySerde, Serde<V> valueSerde)	Creates a StoreBuilder that can be used to build a KeyValueStore.
static KeyValueBytesStoreSupplier	lruMap(String name, int maxCacheSize)	Create a LRU Map KeyValueBytesStoreSupplier.
static KeyValueBytesStoreSupplier	persistentKeyValueStore(String name)	Create a persistent KeyValueBytesStoreSupplier.
static SessionBytesStoreSupplier	persistentSessionStore(String name, long retentionPeriodMs)	Deprecated. since 2.1 Use persistentSessionStore(String, Duration) instead
static SessionBytesStoreSupplier	persistentSessionStore(String name, Duration retentionPeriod)	Create a persistent SessionBytesStoreSupplier.
static KeyValueBytesStoreSupplier	persistentTimestampedKeyValueStore(String name)	Create a persistent KeyValueBytesStoreSupplier.
static WindowBytesStoreSupplier	persistentTimestampedWindowStore(String name, Duration retentionPeriod, Duration windowSize, boolean retainDuplicates)	Create a persistent WindowBytesStoreSupplier.
static WindowBytesStoreSupplier	persistentWindowStore(String name, long retentionPeriod, int numSegments, long windowSize, boolean retainDuplicates)	Deprecated. since 2.1 Use persistentWindowStore(String, Duration, Duration, boolean) instead
static WindowBytesStoreSupplier	persistentWindowStore(String name, Duration retentionPeriod, Duration windowSize, boolean retainDuplicates)	Create a persistent WindowBytesStoreSupplier.
static <K,V> StoreBuilder<SessionStore<K,V>>	sessionStoreBuilder(SessionBytesStoreSupplier supplier, Serde<K> keySerde, Serde<V> valueSerde)	Creates a StoreBuilder that can be used to build a SessionStore.
static <K,V> StoreBuilder<TimestampedKeyValueStore<K,V>>	timestampedKeyValueStoreBuilder(KeyValueBytesStoreSupplier supplier, Serde<K> keySerde, Serde<V> valueSerde)	Creates a StoreBuilder that can be used to build a TimestampedKeyValueStore.
static <K,V> StoreBuilder<TimestampedWindowStore<K,V>>	timestampedWindowStoreBuilder(WindowBytesStoreSupplier supplier, Serde<K> keySerde, Serde<V> valueSerde)	Creates a StoreBuilder that can be used to build a TimestampedWindowStore.
static <K,V> StoreBuilder<WindowStore<K,V>>	windowStoreBuilder(WindowBytesStoreSupplier supplier, Serde<K> keySerde, Serde<V> valueSerde)	Creates a StoreBuilder that can be used to build a WindowStore.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Stores

public Stores()

Method Detail

persistentKeyValueStore

public static KeyValueBytesStoreSupplier persistentKeyValueStore(String name)

Create a persistent KeyValueBytesStoreSupplier.

This store supplier can be passed into a keyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde). If you want to create a TimestampedKeyValueStore you should use persistentTimestampedKeyValueStore(String) to create a store supplier instead.

Parameters:

name - name of the store (cannot be null)

Returns:

an instance of a KeyValueBytesStoreSupplier that can be used to build a persistent key-value store

persistentTimestampedKeyValueStore

public static KeyValueBytesStoreSupplier persistentTimestampedKeyValueStore(String name)

Create a persistent KeyValueBytesStoreSupplier.

This store supplier can be passed into a timestampedKeyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde). If you want to create a KeyValueStore you should use persistentKeyValueStore(String) to create a store supplier instead.

Parameters:

name - name of the store (cannot be null)

Returns:

an instance of a KeyValueBytesStoreSupplier that can be used to build a persistent key-(timestamp/value) store

inMemoryKeyValueStore

public static KeyValueBytesStoreSupplier inMemoryKeyValueStore(String name)

Create an in-memory KeyValueBytesStoreSupplier.

This store supplier can be passed into a keyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde) or timestampedKeyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde).

Parameters:

name - name of the store (cannot be null)

Returns:

an instance of a KeyValueBytesStoreSupplier than can be used to build an in-memory store

lruMap

public static KeyValueBytesStoreSupplier lruMap(String name,
                                                int maxCacheSize)

Create a LRU Map KeyValueBytesStoreSupplier.

This store supplier can be passed into a keyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde) or timestampedKeyValueStoreBuilder(KeyValueBytesStoreSupplier, Serde, Serde).

Parameters:

name - name of the store (cannot be null)

maxCacheSize - maximum number of items in the LRU (cannot be negative)

Returns:

an instance of a KeyValueBytesStoreSupplier that can be used to build an LRU Map based store

Throws:

IllegalArgumentException - if maxCacheSize is negative

persistentWindowStore

@Deprecated
public static WindowBytesStoreSupplier persistentWindowStore(String name,
                                                             long retentionPeriod,
                                                             int numSegments,
                                                             long windowSize,
                                                             boolean retainDuplicates)

Deprecated.

since 2.1 Use persistentWindowStore(String, Duration, Duration, boolean) instead

Create a persistent WindowBytesStoreSupplier.

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length of time to retain data in the store (cannot be negative) (note that the retention period must be at least long enough to contain the windowed data's entire life cycle, from window-start through window-end, and for the entire grace period)

numSegments - number of db segments (cannot be zero or negative)

windowSize - size of the windows that are stored (cannot be negative). Note: the window size is not stored with the records, so this value is used to compute the keys that the store returns. No effort is made to validate this parameter, so you must be careful to set it the same as the windowed keys you're actually storing.

retainDuplicates - whether or not to retain duplicates. Turning this on will automatically disable caching and means that null values will be ignored.

Returns:

an instance of WindowBytesStoreSupplier

persistentWindowStore

public static WindowBytesStoreSupplier persistentWindowStore(String name,
                                                             Duration retentionPeriod,
                                                             Duration windowSize,
                                                             boolean retainDuplicates)
                                                      throws IllegalArgumentException

Create a persistent WindowBytesStoreSupplier.

This store supplier can be passed into a windowStoreBuilder(WindowBytesStoreSupplier, Serde, Serde). If you want to create a TimestampedWindowStore you should use persistentTimestampedWindowStore(String, Duration, Duration, boolean) to create a store supplier instead.

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length of time to retain data in the store (cannot be negative) (note that the retention period must be at least long enough to contain the windowed data's entire life cycle, from window-start through window-end, and for the entire grace period)

windowSize - size of the windows (cannot be negative)

retainDuplicates - whether or not to retain duplicates. Turning this on will automatically disable caching and means that null values will be ignored.

Returns:

an instance of WindowBytesStoreSupplier

Throws:

IllegalArgumentException - if retentionPeriod or windowSize can't be represented as long milliseconds

IllegalArgumentException - if retentionPeriod is smaller than windowSize

persistentTimestampedWindowStore

public static WindowBytesStoreSupplier persistentTimestampedWindowStore(String name,
                                                                        Duration retentionPeriod,
                                                                        Duration windowSize,
                                                                        boolean retainDuplicates)
                                                                 throws IllegalArgumentException

Create a persistent WindowBytesStoreSupplier.

This store supplier can be passed into a timestampedWindowStoreBuilder(WindowBytesStoreSupplier, Serde, Serde). If you want to create a WindowStore you should use persistentWindowStore(String, Duration, Duration, boolean) to create a store supplier instead.

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length of time to retain data in the store (cannot be negative) (note that the retention period must be at least long enough to contain the windowed data's entire life cycle, from window-start through window-end, and for the entire grace period)

windowSize - size of the windows (cannot be negative)

retainDuplicates - whether or not to retain duplicates. Turning this on will automatically disable caching and means that null values will be ignored.

Returns:

an instance of WindowBytesStoreSupplier

Throws:

IllegalArgumentException - if retentionPeriod or windowSize can't be represented as long milliseconds

IllegalArgumentException - if retentionPeriod is smaller than windowSize

inMemoryWindowStore

public static WindowBytesStoreSupplier inMemoryWindowStore(String name,
                                                           Duration retentionPeriod,
                                                           Duration windowSize,
                                                           boolean retainDuplicates)
                                                    throws IllegalArgumentException

Create an in-memory WindowBytesStoreSupplier.

This store supplier can be passed into a windowStoreBuilder(WindowBytesStoreSupplier, Serde, Serde) or timestampedWindowStoreBuilder(WindowBytesStoreSupplier, Serde, Serde).

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length of time to retain data in the store (cannot be negative) Note that the retention period must be at least long enough to contain the windowed data's entire life cycle, from window-start through window-end, and for the entire grace period.

windowSize - size of the windows (cannot be negative)

retainDuplicates - whether or not to retain duplicates. Turning this on will automatically disable caching and means that null values will be ignored.

Returns:

an instance of WindowBytesStoreSupplier

Throws:

IllegalArgumentException - if retentionPeriod or windowSize can't be represented as long milliseconds

IllegalArgumentException - if retentionPeriod is smaller than windowSize

persistentSessionStore

@Deprecated
public static SessionBytesStoreSupplier persistentSessionStore(String name,
                                                               long retentionPeriodMs)

Deprecated.

since 2.1 Use persistentSessionStore(String, Duration) instead

Create a persistent SessionBytesStoreSupplier.

Parameters:

name - name of the store (cannot be null)

retentionPeriodMs - length of time to retain data in the store (cannot be negative) (note that the retention period must be at least as long enough to contain the inactivity gap of the session and the entire grace period.)

Returns:

an instance of a SessionBytesStoreSupplier

persistentSessionStore

public static SessionBytesStoreSupplier persistentSessionStore(String name,
                                                               Duration retentionPeriod)

Create a persistent SessionBytesStoreSupplier.

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length of time to retain data in the store (cannot be negative) (note that the retention period must be at least as long enough to contain the inactivity gap of the session and the entire grace period.)

Returns:

an instance of a SessionBytesStoreSupplier

inMemorySessionStore

public static SessionBytesStoreSupplier inMemorySessionStore(String name,
                                                             Duration retentionPeriod)

Create an in-memory SessionBytesStoreSupplier.

Parameters:

name - name of the store (cannot be null)

retentionPeriod - length ot time to retain data in the store (cannot be negative) (note that the retention period must be at least as long enough to contain the inactivity gap of the session and the entire grace period.)

Returns:

an instance of a SessionBytesStoreSupplier

keyValueStoreBuilder

public static <K,V> StoreBuilder<KeyValueStore<K,V>> keyValueStoreBuilder(KeyValueBytesStoreSupplier supplier,
                                                                                      Serde<K> keySerde,
                                                                                      Serde<V> valueSerde)

Creates a StoreBuilder that can be used to build a KeyValueStore.

The provided supplier should not be a supplier for TimestampedKeyValueStores.

Type Parameters:

K - key type

V - value type

Parameters:

supplier - a KeyValueBytesStoreSupplier (cannot be null)

keySerde - the key serde to use

valueSerde - the value serde to use; if the serialized bytes is null for put operations, it is treated as delete

Returns:

an instance of a StoreBuilder that can build a KeyValueStore

timestampedKeyValueStoreBuilder

public static <K,V> StoreBuilder<TimestampedKeyValueStore<K,V>> timestampedKeyValueStoreBuilder(KeyValueBytesStoreSupplier supplier,
                                                                                                            Serde<K> keySerde,
                                                                                                            Serde<V> valueSerde)

Creates a StoreBuilder that can be used to build a TimestampedKeyValueStore.

The provided supplier should not be a supplier for KeyValueStores. For this case, passed in timestamps will be dropped and not stored in the key-value-store. On read, no valid timestamp but a dummy timestamp will be returned.

Type Parameters:

K - key type

V - value type

Parameters:

supplier - a KeyValueBytesStoreSupplier (cannot be null)

keySerde - the key serde to use

valueSerde - the value serde to use; if the serialized bytes is null for put operations, it is treated as delete

Returns:

an instance of a StoreBuilder that can build a KeyValueStore

windowStoreBuilder

public static <K,V> StoreBuilder<WindowStore<K,V>> windowStoreBuilder(WindowBytesStoreSupplier supplier,
                                                                                  Serde<K> keySerde,
                                                                                  Serde<V> valueSerde)

Creates a StoreBuilder that can be used to build a WindowStore.

The provided supplier should not be a supplier for TimestampedWindowStores.

Type Parameters:

K - key type

V - value type

Parameters:

supplier - a WindowBytesStoreSupplier (cannot be null)

keySerde - the key serde to use

valueSerde - the value serde to use; if the serialized bytes is null for put operations, it is treated as delete

Returns:

an instance of StoreBuilder than can build a WindowStore

timestampedWindowStoreBuilder

public static <K,V> StoreBuilder<TimestampedWindowStore<K,V>> timestampedWindowStoreBuilder(WindowBytesStoreSupplier supplier,
                                                                                                        Serde<K> keySerde,
                                                                                                        Serde<V> valueSerde)

Creates a StoreBuilder that can be used to build a TimestampedWindowStore.

The provided supplier should not be a supplier for WindowStores. For this case, passed in timestamps will be dropped and not stored in the window-store. On read, no valid timestamp but a dummy timestamp will be returned.

Type Parameters:

K - key type

V - value type

Parameters:

supplier - a WindowBytesStoreSupplier (cannot be null)

keySerde - the key serde to use

valueSerde - the value serde to use; if the serialized bytes is null for put operations, it is treated as delete

Returns:

an instance of StoreBuilder that can build a TimestampedWindowStore

sessionStoreBuilder

public static <K,V> StoreBuilder<SessionStore<K,V>> sessionStoreBuilder(SessionBytesStoreSupplier supplier,
                                                                                    Serde<K> keySerde,
                                                                                    Serde<V> valueSerde)

Creates a StoreBuilder that can be used to build a SessionStore.

Type Parameters:

K - key type

V - value type

Parameters:

supplier - a SessionBytesStoreSupplier (cannot be null)

keySerde - the key serde to use

valueSerde - the value serde to use; if the serialized bytes is null for put operations, it is treated as delete

Returns:

an instance of StoreBuilder than can build a SessionStore