Mixed Deck — All Apache Kafka Topics Flashcards

What is a 'changelog topic' in Kafka Streams?

Answer: A Kafka topic used to back up local state store contents for fault tolerance

Kafka Streams writes all state store updates to a changelog topic so that state can be restored from Kafka if a task migrates to another instance.

What is a 'session window' in Kafka Streams?

Answer: A dynamic window that groups records separated by an inactivity gap smaller than a configurable threshold

Session windows group events for a key into a session as long as the gap between consecutive events is less than the inactivity gap timeout.

Can you decrease the number of partitions in an existing Kafka topic?

Answer: No, partition count can only be increased, never decreased

Kafka does not support decreasing the partition count of an existing topic because it would require redistributing data and break ordering guarantees.

What does a 'stream-table join' in Kafka Streams do?

Answer: Enriches each stream record with the current table value for the matching key

A stream-table join looks up the latest KTable value for a record's key and enriches the stream record with that value in real time.

Which Kafka CLI tool is used to create a new topic?

Answer: kafka-topics.sh

kafka-topics.sh with the --create flag is the standard tool for creating new Kafka topics from the command line.

Which of the following types of communication is performed between two nodes?

Answer: Zookeeper

Explanation: Zookeeper is a top-tier cluster coordination service that employs the most reliable synchronization techniques to keep the cluster's nodes perfectly connected. Apache Zookeeper solves distributed environment management with its simple architecture and API.

What is the Processor API in Kafka Streams used for?

Answer: Provides low-level access to define custom processing logic, state stores, and punctuations beyond the DSL

The Processor API gives developers fine-grained control over processing, state access, and scheduling punctuations that the higher-level DSL does not expose.

How does Kafka Streams handle fault tolerance for stateless operations?

Answer: By replaying input topic data from the last committed offset

For stateless tasks, Kafka Streams simply replays the input topic from the last committed offset when a task is restarted or migrated.

What happens to a failed Connect task if you POST to `/connectors/{name}/tasks/{taskId}/restart`?

Answer: The specific task is restarted and attempts to resume from its last stored offset

Restarting an individual task via the REST API causes that task to reinitialize and resume data copying from the last committed offset.

What is a 'consumer lag' in Kafka?

Answer: The difference between the latest produced offset and the consumer's current offset

Consumer lag measures how far behind a consumer group is relative to the latest messages produced in a partition.

What is the function of the Kafka `__transaction_state` internal topic?

Answer: Stores the state of ongoing and completed transactions managed by transaction coordinators

__transaction_state is Kafka's internal topic for persisting transaction status so that transaction coordinators can recover after a failure.

Which protocol is used for encrypting data in transit between Kafka clients and brokers?

Answer: TLS (Transport Layer Security)

TLS (formerly SSL) encrypts the network traffic between Kafka clients and brokers, preventing eavesdropping and man-in-the-middle attacks.

What does the `advertised.listeners` broker configuration do?

Answer: Publishes the listener addresses that clients should use to connect to the broker

advertised.listeners tells Kafka what addresses to publish in cluster metadata so clients can reach the broker, critical in containerized or cloud environments.

Which consumer API call is used to manually commit the current offset synchronously?

Answer: commitSync()

commitSync() blocks until the broker acknowledges the offset commit, providing a reliable but potentially slower commit method.

What does the `log.dirs` broker configuration specify?

Answer: One or more comma-separated directories where Kafka stores partition log data on disk

log.dirs tells Kafka brokers which directories to use for storing partition segment files; spreading across multiple paths can improve I/O throughput.

What does `flatMapValues()` do in Kafka Streams?

Answer: Transforms each record's value into zero or more values, emitting one record per resulting value

flatMapValues() applies a function to each record's value that returns an iterable, emitting a separate downstream record for each element.

When using `assign()` instead of `subscribe()`, what is the consumer responsible for?

Answer: Manually managing partition assignment without group coordinator involvement

assign() gives the consumer direct control over which partitions it reads, bypassing the group coordinator and rebalance protocol entirely.

What is the significance of `max.in.flight.requests.per.connection` in a Kafka producer?

Answer: Controls how many unacknowledged requests can be in flight simultaneously; setting it to 1 preserves ordering under retries

If retries > 0 and max.in.flight.requests.per.connection > 1, a retry can cause message reordering; setting it to 1 guarantees order at the cost of throughput.

What is a Kafka 'transaction'?

Answer: An atomic operation that writes to multiple topics/partitions and commits consumer offsets as a single all-or-nothing unit

Kafka transactions allow producers to send messages to multiple partitions and commit consumer offsets atomically, ensuring all-or-nothing semantics.

Consumers utilize Apache Kafka's control records to

Answer: filter out aborted transactional messages

Explanation: Control records in Apache Kafka are used by consumers to filter out aborted transactional messages.