Apache Kafka Kafka Architecture & Internals

Question 1

What is a Kafka 'offset'?

Accepted Answer

A sequential integer that uniquely identifies each message's position within a partition

Answer

An offset is a monotonically increasing integer assigned to each message within a partition, used by consumers to track which messages have been read.

Question 2

What does 'exactly-once semantics' (EOS) in Kafka require?

Accepted Answer

Idempotent producers plus Kafka transactions to ensure each message is processed and committed once

Answer

EOS requires idempotent producers (no duplicates) combined with Kafka's transactional API so that reads, processing, and writes are committed atomically.

Question 3

What is a Kafka 'transaction'?

Accepted Answer

An atomic operation that writes to multiple topics/partitions and commits consumer offsets as a single all-or-nothing unit

Answer

Kafka transactions allow producers to send messages to multiple partitions and commit consumer offsets atomically, ensuring all-or-nothing semantics.

Question 4

What is the role of the Kafka broker's 'log cleaner' thread?

Accepted Answer

Performs log compaction by removing older duplicate-key records from compacted topics

Answer

The log cleaner is a background thread that runs compaction on topics with cleanup.policy=compact, merging segments and removing superseded key records.

Question 5

What is 'zero-copy' in the context of Kafka's I/O performance?

Accepted Answer

Data is transferred from disk to network buffer directly in the OS kernel without copying through user space

Answer

Kafka uses the OS sendfile() system call to transfer data from the page cache to the network socket entirely in kernel space, dramatically reducing CPU and memory overhead.

Apache Kafka Practice Test

Apache Kafka Practice Test

Apache Kafka Kafka Architecture & Internals