Data Engineering Distributed Data Processing 3

Question 1

Which delivery guarantee ensures each message is processed once with no duplicates or loss?

Accepted Answer

Exactly-once

Answer

Exactly-once semantics processes every record one time, avoiding duplicates and drops.

Question 2

In Kafka, what determines the maximum parallelism of consumers in a group?

Accepted Answer

The number of partitions

Answer

Each partition is consumed by at most one consumer in a group, capping parallelism.

Question 3

What is the main trade-off described by the CAP theorem during a network partition?

Accepted Answer

Choosing between consistency and availability

Answer

Under a partition you must sacrifice either consistency or availability.

Question 4

A windowed aggregation over event-time streams typically needs which mechanism to handle late data?

Accepted Answer

Watermarks

Answer

Watermarks define how long to wait for late events before finalizing a window.

Question 5

Why might increasing the number of partitions improve throughput up to a point?

Accepted Answer

More partitions allow more parallel tasks

Answer

More partitions enable greater parallelism, but too many add scheduling overhead.

Data Engineering Practice Test