Kafka Architecture & Internals Flashcards

What is the purpose of Kafka's `__consumer_offsets` topic compaction?

Answer: To retain only the latest committed offset per consumer group and partition, keeping the topic size manageable

Log compaction on __consumer_offsets ensures that only the most recent offset commit per (group, topic, partition) key is retained, preventing unbounded growth.

What does `isolation.level=read_committed` do for a Kafka consumer?

Answer: Causes the consumer to skip transactional messages that have not yet been committed, avoiding dirty reads

read_committed isolation level ensures consumers only see messages from committed transactions, hiding in-flight or aborted transactional records.

What is a Kafka 'broker rack' configuration used for?

Answer: Assigns a broker to a rack/availability zone so partition replicas can be spread across failure domains

The broker.rack setting tells Kafka to spread partition replicas across different racks or AZs so that a single rack failure won't take down all replicas of a partition.

What is a 'fetch session' in the Kafka broker's fetch protocol?

Answer: An incremental fetch optimization where the broker caches partition state between fetches, reducing repeated metadata overhead

Fetch sessions (incremental fetch) allow the broker and consumer to maintain a session so that the fetch request only needs to include changed partitions, reducing network overhead.

What is the significance of `max.in.flight.requests.per.connection` in a Kafka producer?

Answer: Controls how many unacknowledged requests can be in flight simultaneously; setting it to 1 preserves ordering under retries

If retries > 0 and max.in.flight.requests.per.connection > 1, a retry can cause message reordering; setting it to 1 guarantees order at the cost of throughput.

What metric best indicates a Kafka broker is under memory pressure?

Answer: High page cache miss rate (increased disk read I/O)

Kafka relies heavily on the OS page cache; if the working set exceeds available memory, page cache misses cause disk reads and severely degrade throughput.