Data Engineering Introduction To Data Engineering 3

Question 1

In ELT, where does the transformation step occur compared to ETL?

Accepted Answer

Inside the destination system after loading raw data

Answer

ELT loads raw data into the target first, then transforms it using the destination's compute power.

Question 2

What is data partitioning primarily used for?

Accepted Answer

Improving query performance and manageability by splitting data into segments

Answer

Partitioning divides large datasets into segments so queries can scan less data and run faster.

Question 3

Which file format is columnar and commonly used in big data analytics?

Accepted Answer

Parquet

Answer

Parquet is a columnar storage format optimized for analytical queries on large datasets.

Question 4

What does 'idempotency' mean for a data pipeline operation?

Accepted Answer

Running it multiple times produces the same result as running it once

Answer

An idempotent operation yields the same outcome no matter how many times it is executed.

Question 5

Which tool is widely used to orchestrate and schedule data workflows?

Accepted Answer

Apache Airflow

Answer

Apache Airflow orchestrates workflows as directed acyclic graphs (DAGs) with scheduling and monitoring.

Data Engineering Practice Test