Data Engineering Orchestrating Data Workflows 3

Question 1

What is the main advantage of defining workflows as code (e.g., Python DAGs) over a GUI-only tool?

Accepted Answer

Version control, code review, and reproducibility

Answer

Workflows-as-code can be versioned, reviewed, and tested like any other software.

Question 2

In Airflow, what does the schedule_interval '@daily' do?

Accepted Answer

Runs the DAG once at midnight each day

Answer

'@daily' schedules the DAG to run once per day at midnight.

Question 3

Which retry strategy helps avoid overwhelming a downstream system after repeated failures?

Accepted Answer

Exponential backoff

Answer

Exponential backoff increases the delay between retries, reducing load on failing systems.

Question 4

What problem does a task dependency graph primarily prevent?

Accepted Answer

Running tasks before their inputs are ready

Answer

Dependency graphs ensure a task only runs after its upstream prerequisites complete.

Question 5

In Prefect or Dagster, what is a key advantage over a pure cron schedule?

Accepted Answer

Built-in retries, observability, and dependency management

Answer

Modern orchestrators add observability, retries, and dependency handling that cron lacks.

Data Engineering Practice Test