Data Engineering Real-Time Streaming Architectures 3 — Questions and Answers
Question 1: The Kappa architecture differs from Lambda primarily because it:
- Uses a single stream-processing path for both real-time and reprocessing (Correct answer)
- Adds a third validation layer
- Eliminates the message broker
- Requires only batch processing
Correct answer: Uses a single stream-processing path for both real-time and reprocessing
Kappa removes the batch layer and reprocesses by replaying the log through one streaming path.
Question 2: What is a session window in stream processing?
- A window defined by periods of activity separated by inactivity gaps (Correct answer)
- A fixed 60-second interval
- A window that overlaps by 50%
- A window keyed only on processing time
Correct answer: A window defined by periods of activity separated by inactivity gaps
Session windows group events that occur close together, closing after a configured inactivity gap.
Question 3: In Kafka, increasing the replication factor primarily improves:
- Fault tolerance and durability (Correct answer)
- Message ordering guarantees
- Producer compression ratio
- Consumer offset commit speed
Correct answer: Fault tolerance and durability
More replicas mean the data survives more broker failures, raising durability.
Question 4: What is the function of a checkpoint in a stateful stream processor like Flink?
- Persist state snapshots for recovery after failure (Correct answer)
- Reorder out-of-order events
- Compress the operator topology
- Assign keys to partitions
Correct answer: Persist state snapshots for recovery after failure
Checkpoints periodically snapshot operator state so processing can resume consistently after a crash.
Question 5: Which serialization format is commonly used with a schema registry for streaming data?
- Avro (Correct answer)
- Plain CSV
- YAML
- INI
Correct answer: Avro
Avro pairs well with schema registries to enforce and evolve schemas across producers and consumers.
Question 6: Processing-time semantics differ from event-time semantics because processing time:
- Uses the clock of the machine processing the event (Correct answer)
- Uses the timestamp embedded in the event
- Ignores all timestamps
- Always equals the producer timestamp
Correct answer: Uses the clock of the machine processing the event
Processing time relies on the processor's wall clock, not when the event actually occurred.
Question 7: What problem does a dead-letter queue solve in a streaming system?
- Isolating messages that repeatedly fail processing (Correct answer)
- Compressing large payloads
- Balancing partitions across brokers
- Caching frequently read records
Correct answer: Isolating messages that repeatedly fail processing
A dead-letter queue captures unprocessable messages so the main pipeline keeps flowing.
Data Engineering Real-Time Streaming Architectures 3
The Kappa architecture differs from Lambda primarily because it: