Data Processing Batch Processing and Real-Time Processing 3

Question 1

What is the Lambda architecture in data processing?

Accepted Answer

A design pattern combining batch and speed (stream) layers to serve both historical and real-time queries

Answer

Lambda architecture uses a batch layer for accuracy on historical data and a speed layer for low-latency real-time results, with a serving layer merging both.

Question 2

What is the Kappa architecture in data processing?

Accepted Answer

A simplified architecture that uses only a streaming layer to handle both real-time and reprocessed historical data

Answer

Kappa architecture simplifies Lambda by using a single streaming system for all processing, reprocessing historical data by replaying the event log.

Question 3

What is backpressure in stream processing systems?

Accepted Answer

A flow control mechanism where a slow consumer signals a fast producer to slow down

Answer

Backpressure is a feedback mechanism where an overwhelmed downstream consumer signals upstream producers to reduce their send rate to prevent data loss.

Question 4

In data processing, what does throughput measure?

Accepted Answer

The volume of data or records processed per unit of time

Answer

Throughput measures how many records or how much data a system processes per second or per hour, indicating overall processing capacity.

Question 5

What is checkpointing in stream processing?

Accepted Answer

Periodically saving the processing state so the system can resume from that point after a failure

Answer

Checkpointing saves the current state of a stream processing job at intervals so it can recover and resume from the last checkpoint instead of reprocessing from the beginning.

Data Processing Practice Test

Data Processing Practice Test

Data Processing Batch Processing and Real-Time Processing 3