Real-Time Streaming Architectures Flashcards
7 cards from real Data Engineering practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Real-Time Streaming Architectures flashcards as text
A stream-to-stream join typically requires which constraint to bound state?
Answer: A time window on the join condition
Bounding the join with a time window limits how much state must be retained for matching.
What does consumer lag measure in a streaming system?
Answer: The gap between the latest produced offset and the consumer's committed offset
Consumer lag is how far behind a consumer is relative to the newest available messages.
In Apache Flink, the purpose of allowed lateness after a watermark is to:
Answer: Update window results when late events still arrive
Allowed lateness lets windows accept and re-emit results for events arriving slightly after the watermark.
Which choice best reduces end-to-end latency in a streaming pipeline?
Answer: Smaller batch/linger sizes at the cost of throughput
Smaller batching emits records sooner, trading some throughput for lower latency.
What is the role of a partition key when producing to Kafka?
Answer: It determines which partition a message is routed to
The key is hashed to choose a partition, keeping same-key messages in order on one partition.
Schema evolution with backward compatibility allows:
Answer: New consumers to read data written with an older schema
Backward compatibility means a newer schema can still read older-format records.
Why might you choose Kappa over Lambda architecture?
Answer: To avoid maintaining two separate codebases for batch and streaming
Kappa unifies logic into one streaming path, removing the dual-codebase burden of Lambda.