Data Engineering Distributed Data Processing 5

Question 1

What is the benefit of partition pruning in a distributed query engine?

Accepted Answer

It skips reading partitions irrelevant to the filter

Answer

Partition pruning avoids scanning data that cannot match the query predicate.

Question 2

In the lambda architecture, what is the speed layer responsible for?

Accepted Answer

Low-latency views of recent data

Answer

The speed layer serves real-time results while the batch layer recomputes accurate views.

Question 3

Which factor most directly limits the scalability of a global ORDER BY in distributed SQL?

Accepted Answer

All data must funnel through a single sort/merge step

Answer

Total ordering forces a global merge that becomes a bottleneck at scale.

Question 4

What does eventual consistency guarantee?

Accepted Answer

Replicas converge to the same value if updates stop

Answer

Given no new writes, all replicas eventually reflect the same data.

Question 5

Why are columnar formats often paired with predicate pushdown?

Accepted Answer

The engine can skip blocks using column statistics

Answer

Min/max statistics per column block let the engine skip non-matching data.

Data Engineering Practice Test