Data Engineering Cloud Data Storage Solutions 3 — Questions and Answers
Question 1: Which Azure service provides hierarchical namespace support optimized for big-data analytics?
- Azure Blob Storage (flat)
- Azure Data Lake Storage Gen2 (Correct answer)
- Azure Files
- Azure Queue Storage
Correct answer: Azure Data Lake Storage Gen2
ADLS Gen2 adds a hierarchical namespace on top of Blob Storage for analytics workloads.
Question 2: What is the purpose of partitioning data in cloud object storage for a query engine?
- To encrypt sensitive fields
- To prune irrelevant data and reduce scan volume (Correct answer)
- To enforce schema validation
- To increase replication factor
Correct answer: To prune irrelevant data and reduce scan volume
Partitioning lets query engines skip directories that don't match filter predicates, cutting scan cost.
Question 3: Which consistency model does Amazon S3 now provide for read-after-write on new objects?
- Eventual consistency
- Strong read-after-write consistency (Correct answer)
- No consistency guarantee
- Causal consistency only
Correct answer: Strong read-after-write consistency
S3 provides strong read-after-write consistency for PUTs of new objects.
Question 4: What is a key reason to enable object versioning in a cloud bucket?
- To reduce storage cost
- To protect against accidental overwrites and deletions (Correct answer)
- To speed up queries
- To disable encryption
Correct answer: To protect against accidental overwrites and deletions
Versioning retains previous object copies, allowing recovery from unintended changes.
Question 5: Which open table format adds ACID transactions and time travel on top of a data lake?
- CSV
- Apache Iceberg (Correct answer)
- Plain Parquet
- JSON Lines
Correct answer: Apache Iceberg
Apache Iceberg (like Delta Lake and Hudi) brings ACID transactions and snapshots to lake storage.
Question 6: What does 'data egress cost' refer to in cloud storage pricing?
- Cost to store data at rest
- Cost to transfer data out of the cloud provider's network (Correct answer)
- Cost to encrypt data
- Cost to create buckets
Correct answer: Cost to transfer data out of the cloud provider's network
Egress charges apply when data leaves the provider's network, often to the internet or another region.
Question 7: Which approach minimizes small-file problems in a cloud data lake?
- Writing one file per record
- Compacting many small files into larger files (Correct answer)
- Disabling partitioning entirely
- Storing everything as JSON
Correct answer: Compacting many small files into larger files
Compaction merges many small files into fewer large files, improving read throughput and reducing overhead.
Data Engineering Cloud Data Storage Solutions 3
Which Azure service provides hierarchical namespace support optimized for big-data analytics?