Data Processing ETL and Data Transformation 3 — Questions and Answers
Question 1: What is data lineage?
- The ability to trace data's origin, movements, and transformations throughout its lifecycle (Correct answer)
- The chronological order of database backups
- The number of rows added per ETL job
- The version history of a stored procedure
Correct answer: The ability to trace data's origin, movements, and transformations throughout its lifecycle
Data lineage tracks where data originates, how it has been transformed, and where it flows, enabling auditability and impact analysis.
Question 2: What is the purpose of a lookup transformation in ETL?
- To retrieve related values from a reference table during transformation (Correct answer)
- To delete records that fail validation
- To sort rows before loading into the target
- To compress data files before transmission
Correct answer: To retrieve related values from a reference table during transformation
A lookup transformation queries a reference table to fetch related data values, such as converting a product code into a full product name during processing.
Question 3: What is change data capture (CDC)?
- A technique that identifies and captures only the changes made to source data since the last extraction (Correct answer)
- A method for compressing large ETL log files
- A process for auditing user login attempts
- A way to replicate an entire database nightly
Correct answer: A technique that identifies and captures only the changes made to source data since the last extraction
Change data capture (CDC) tracks and captures insert, update, and delete operations in a source database so only changed data is propagated to downstream systems.
Question 4: What does 'data flattening' mean in ETL transformation?
- Converting hierarchical or nested data structures into a tabular row-column format (Correct answer)
- Reducing file sizes by removing whitespace
- Splitting one record into multiple rows
- Encrypting nested JSON objects
Correct answer: Converting hierarchical or nested data structures into a tabular row-column format
Data flattening converts hierarchical structures like XML or JSON into flat tabular form suitable for loading into relational databases or spreadsheets.
Question 5: Which scheduling approach runs ETL jobs at fixed time intervals regardless of data availability?
- Time-based scheduling (Correct answer)
- Event-driven scheduling
- Manual triggering
- Continuous streaming
Correct answer: Time-based scheduling
Time-based scheduling runs ETL jobs at predetermined intervals (e.g., nightly at 2 AM) independent of whether new data is available.
Question 6: What is data masking used for in ETL pipelines?
- Protecting sensitive data by replacing it with realistic but fictitious values (Correct answer)
- Hiding database tables from unauthorized users
- Compressing data during transfer
- Encoding binary data into text format
Correct answer: Protecting sensitive data by replacing it with realistic but fictitious values
Data masking replaces sensitive data (like Social Security numbers or credit card details) with realistic but non-sensitive substitute values to protect privacy.
Data Processing ETL and Data Transformation 3
What is data lineage?