Data Processing Cycle 2 — Questions and Answers
Question 1: In an ETL pipeline cycle, what does the 'T' phase primarily involve?
- Transmitting data to external systems
- Transforming raw data into a structured format (Correct answer)
- Tracking data lineage metadata
- Testing data integrity constraints
Correct answer: Transforming raw data into a structured format
The Transform phase converts raw extracted data into the required format, structure, or quality level for the target system.
Question 2: Which term describes a data processing cycle where new data is accumulated and processed at scheduled intervals rather than continuously?
- Stream processing
- Batch processing (Correct answer)
- Micro-batch processing
- Event-driven processing
Correct answer: Batch processing
Batch processing collects data over a period and processes it all at once at scheduled times, such as nightly or weekly runs.
Question 3: A processing cycle that runs every few seconds or minutes, balancing latency and throughput, is best described as:
- Pure batch processing
- Real-time streaming
- Micro-batch processing (Correct answer)
- Cold storage archiving
Correct answer: Micro-batch processing
Micro-batch processing runs at very short intervals (seconds to minutes), bridging the gap between batch and real-time streaming.
Question 4: In the data lifecycle cycle, 'data archival' typically occurs at which stage?
- After ingestion but before processing
- During active transformation
- After data is no longer frequently accessed (Correct answer)
- Before data quality checks
Correct answer: After data is no longer frequently accessed
Archival moves data to cheaper, slower storage once it is no longer actively needed but must be retained for compliance or reference.
Question 5: What is a 'hot path' in a Lambda architecture data processing cycle?
- The batch layer that reprocesses all historical data
- The serving layer that merges batch and speed views
- The speed layer that processes real-time streaming data for low-latency results (Correct answer)
- The storage layer that holds immutable master datasets
Correct answer: The speed layer that processes real-time streaming data for low-latency results
The hot path (speed layer) handles real-time data to provide low-latency query results while the batch layer catches up.
Question 6: During a data replication cycle, 'change data capture' (CDC) tracks:
- Full table snapshots at each cycle
- Only inserts to source tables
- Row-level changes (inserts, updates, deletes) in the source system (Correct answer)
- Schema migrations between cycles
Correct answer: Row-level changes (inserts, updates, deletes) in the source system
CDC identifies and captures only the rows that changed since the last replication cycle, minimizing data transfer volume.
Question 7: Which metric best measures the efficiency of a data processing cycle?
- Number of source tables ingested
- Throughput (records processed per second) (Correct answer)
- Size of the data warehouse schema
- Number of ETL developers on the team
Correct answer: Throughput (records processed per second)
Throughput measures how many records a pipeline processes per unit of time, directly reflecting cycle efficiency.
Data Processing Cycle 2
In an ETL pipeline cycle, what does the 'T' phase primarily involve?