Data Processing Batch Processing and Real-Time Processing

Question 1

What is batch processing in data systems?

Accepted Answer

Collecting and processing large volumes of data at scheduled intervals

Answer

Batch processing collects data over a period and processes it all at once in a scheduled job, rather than handling records individually as they arrive.

Question 2

What is stream processing in data systems?

Accepted Answer

Processing data continuously as it arrives event by event in near real-time

Answer

Stream processing handles data records continuously as they arrive, enabling near real-time insights and responses with very low latency.

Question 3

Which framework is widely used in the US for large-scale batch processing on distributed systems?

Accepted Answer

Apache Hadoop MapReduce

Answer

Apache Hadoop MapReduce splits large batch jobs across a cluster, processing each partition in parallel and combining results.

Question 4

What is a key advantage of batch processing over real-time processing?

Accepted Answer

Ability to process very large volumes efficiently at lower cost

Answer

Batch processing can be highly optimized and resource-efficient for large workloads because it processes many records together with shared overhead.

Question 5

What is micro-batch processing?

Accepted Answer

Collecting small batches of data over very short intervals (seconds) for near-real-time processing

Answer

Micro-batch processing collects data in very small time windows (e.g., every few seconds) to balance between true streaming latency and batch efficiency.

Data Processing Practice Test

Data Processing Practice Test

Data Processing Batch Processing and Real-Time Processing