Data Engineering Orchestrating Data Workflows 2

Question 1

In Apache Airflow, what does a DAG represent?

Accepted Answer

A directed acyclic graph defining task dependencies

Answer

A DAG (Directed Acyclic Graph) defines tasks and the order in which they run, with no cycles.

Question 2

Which Airflow concept controls how many task instances can run concurrently for a single DAG?

Accepted Answer

concurrency (max_active_tasks)

Answer

The concurrency (max_active_tasks) setting limits how many task instances run at once within a DAG.

Question 3

What is the purpose of an idempotent task in a data pipeline?

Accepted Answer

Re-running it produces the same result without side effects

Answer

Idempotent tasks can be safely retried because repeated execution yields the same end state.

Question 4

In a workflow scheduler, what is 'backfilling'?

Accepted Answer

Running a DAG for past dates that were missed or newly added

Answer

Backfilling executes a pipeline over historical intervals to populate missing data.

Question 5

Which statement best describes the difference between scheduling and orchestration?

Accepted Answer

Scheduling triggers jobs by time; orchestration manages dependencies and coordination

Answer

Scheduling decides when jobs run, while orchestration coordinates dependencies, retries, and data flow across tasks.

Data Engineering Practice Test