Data Engineering Fundamentals 3 — Questions and Answers
Question 1: What is a slowly changing dimension (SCD) Type 2 designed to do?
- Overwrite old values with new ones
- Preserve full history by adding new rows for changed attributes (Correct answer)
- Delete records that change frequently
- Ignore all attribute changes
Correct answer: Preserve full history by adding new rows for changed attributes
SCD Type 2 keeps historical changes by inserting a new row each time a tracked attribute changes.
Question 2: Which guarantee does 'exactly-once' processing provide in a streaming system?
- Each record may be processed multiple times
- Each record is processed once with no duplicates or loss (Correct answer)
- Records can be lost during failures
- Records are processed in random order
Correct answer: Each record is processed once with no duplicates or loss
Exactly-once semantics ensure every record affects the result a single time, even across failures and retries.
Question 3: What is the role of a message broker like Apache Kafka in a data architecture?
- It stores final analytical reports
- It decouples producers and consumers by buffering streams of events (Correct answer)
- It renders dashboards
- It enforces relational constraints
Correct answer: It decouples producers and consumers by buffering streams of events
Kafka acts as a durable, scalable buffer that decouples data producers from consumers via topic-based event streams.
Question 4: Which normalization concept aims to reduce data redundancy in a relational design?
- Denormalization
- Normalization to higher normal forms (Correct answer)
- Sharding
- Indexing
Correct answer: Normalization to higher normal forms
Normalization organizes tables to minimize redundancy and dependency by progressing through normal forms.
Question 5: What does 'backfilling' a pipeline mean?
- Deleting historical partitions
- Processing past time periods that were missed or need reprocessing (Correct answer)
- Adding new columns to a table
- Compressing old log files
Correct answer: Processing past time periods that were missed or need reprocessing
Backfilling runs a pipeline over historical date ranges to populate or correct data that was missing or wrong.
Question 6: In CAP theorem, what must a distributed system trade off when a network partition occurs?
- Cost and capacity
- Consistency and availability (Correct answer)
- Latency and throughput
- Security and durability
Correct answer: Consistency and availability
CAP theorem states that during a network partition a system must choose between consistency and availability.
Question 7: What is the primary benefit of using a columnar storage engine for OLAP workloads?
- Faster single-row inserts
- Efficient reads and compression of specific columns for aggregations (Correct answer)
- Better support for transactions
- Simpler row-level locking
Correct answer: Efficient reads and compression of specific columns for aggregations
Columnar storage lets analytical queries read only needed columns and compress them well, speeding up aggregations.
Data Engineering Fundamentals 3
What is a slowly changing dimension (SCD) Type 2 designed to do?