Data Engineering Data Quality and Testing 2

Question 1

What is 'data lineage' and why is it important for data quality?

Accepted Answer

The end-to-end tracking of data from origin through all transformations to its final destination

Answer

Data lineage maps the journey of data from source to destination, enabling root cause analysis when data quality issues occur.

Question 2

Which approach to data quality testing runs assertions directly in SQL against your data warehouse tables?

Accepted Answer

In-warehouse testing (e.g., dbt tests)

Answer

In-warehouse testing (used by dbt and similar tools) compiles assertions into SQL queries that run directly against the target tables.

Question 3

What is 'referential integrity' in the context of data warehouse quality?

Accepted Answer

Ensuring all fact table foreign keys have matching dimension table records

Answer

Referential integrity ensures every foreign key in the fact table points to an existing record in the corresponding dimension table.

Question 4

What is an 'anomaly detection' test in data quality pipelines?

Accepted Answer

A test that flags unexpected deviations from historical data patterns

Answer

Anomaly detection tests compare current data metrics against historical baselines to flag unusual volumes, distributions, or values.

Question 5

In dbt, what is the difference between a 'test' and a 'snapshot'?

Accepted Answer

Tests validate data quality assertions; snapshots capture slowly changing dimension history

Answer

dbt tests assert data quality conditions (like uniqueness or not-null), while snapshots implement SCD Type 2 by capturing row changes over time.

Data Engineering Practice Test