Data Engineering ETL and ELT Pipelines 3

Question 1

Which orchestration tool models pipelines as Directed Acyclic Graphs (DAGs) of tasks?

Accepted Answer

Apache Airflow

Answer

Airflow defines workflows as DAGs where each node is a task with explicit dependencies.

Question 2

A pipeline must process events within seconds of arrival. Which paradigm fits best?

Accepted Answer

Streaming (real-time) processing

Answer

Streaming processes records continuously as they arrive, achieving low end-to-end latency.

Question 3

In dimensional modeling, a Type 2 Slowly Changing Dimension handles updates by what method?

Accepted Answer

Adding a new row to preserve historical versions

Answer

SCD Type 2 inserts a new row with validity dates, keeping the full history of the attribute.

Question 4

What is the main risk of using SELECT * during extraction from a wide source table?

Accepted Answer

Pulling unnecessary columns and breaking when schema changes

Answer

SELECT * transfers all columns wastefully and makes the pipeline fragile to upstream schema changes.

Question 5

Which storage layer typically holds raw, unprocessed data in a modern data lakehouse pattern?

Accepted Answer

The bronze (raw) layer

Answer

The medallion architecture's bronze layer stores raw ingested data before refinement.

Data Engineering Practice Test