Kafka Streams Flashcards

What is the Kafka Streams DSL `groupByKey()` method used for?

Answer: Groups records by their existing key as a prerequisite for aggregation operations

groupByKey() signals that records should be grouped by their current key, enabling subsequent aggregate, count, or reduce operations.

What is a 'repartition topic' in Kafka Streams and when is it created?

Answer: An internal topic created when a key-changing operation (e.g. selectKey) precedes a join or aggregation requiring co-partitioning

When an operation changes the record key, Kafka Streams must repartition the data via an internal topic to ensure co-partitioned data lands on the same task.

How does Kafka Streams handle fault tolerance for stateless operations?

Answer: By replaying input topic data from the last committed offset

For stateless tasks, Kafka Streams simply replays the input topic from the last committed offset when a task is restarted or migrated.

What does `to()` do at the end of a Kafka Streams topology?

Answer: Writes the output records to a specified Kafka topic (terminal operation)

to() is a terminal operation that materializes the stream's output back to a Kafka topic, completing the processing pipeline.

Which Kafka Streams config controls the number of parallel stream processing threads per instance?

Answer: num.stream.threads

num.stream.threads sets how many threads within a single Kafka Streams application instance process stream tasks concurrently.

What is the Processor API in Kafka Streams used for?

Answer: Provides low-level access to define custom processing logic, state stores, and punctuations beyond the DSL

The Processor API gives developers fine-grained control over processing, state access, and scheduling punctuations that the higher-level DSL does not expose.