ETL and ELT Pipelines Flashcards
7 cards from real Data Engineering practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 ETL and ELT Pipelines flashcards as text
In an ELT pipeline, where does the bulk of data transformation logic execute?
Answer: In the target data warehouse after loading
ELT loads raw data into the warehouse first and uses the warehouse's compute to transform it.
A pipeline reprocesses the entire source table every night instead of only changed rows. Which technique would reduce this load?
Answer: Incremental loading with change data capture
Incremental loading via CDC processes only inserted, updated, or deleted records since the last run.
Which scenario most favors ELT over traditional ETL?
Answer: A cloud warehouse with elastic, scalable compute like Snowflake or BigQuery
ELT thrives when the destination warehouse provides cheap, scalable compute to transform raw loaded data.
What is the primary purpose of a staging area in an ETL pipeline?
Answer: A temporary landing zone for raw data before transformation
Staging holds extracted raw data temporarily so transformations can run without touching source systems.
An idempotent pipeline run guarantees what behavior?
Answer: Running it multiple times produces the same final result
Idempotency means repeated executions yield identical state, preventing duplicates on retry.
Which approach best handles late-arriving data in a batch pipeline?
Answer: Reprocessing a lookback window of recent partitions
A lookback window reprocesses recent partitions so records that arrive late are still captured.
What does schema drift refer to in a data pipeline?
Answer: Unexpected changes to source column names, types, or structure over time
Schema drift is when the source structure evolves, potentially breaking downstream transformations.