LFC Architecture 3 — Questions and Answers
Question 1: Which open table format feature allows querying a Delta table as it existed at a previous point in time?
- Schema evolution
- Time travel (versioned snapshots) (Correct answer)
- Auto-optimize compaction
- Liquid clustering
Correct answer: Time travel (versioned snapshots)
Delta Lake's time travel feature uses the transaction log to reconstruct any historical version of a table using VERSION AS OF or TIMESTAMP AS OF syntax.
Question 2: What type of workload does a Databricks SQL Warehouse primarily optimize for?
- Long-running ETL batch jobs that process terabytes of raw files
- Interactive BI and SQL analytics queries requiring low latency (Correct answer)
- Distributed deep learning training on GPU clusters
- Real-time event streaming from Apache Kafka topics
Correct answer: Interactive BI and SQL analytics queries requiring low latency
Databricks SQL Warehouses use Photon-powered vectorized execution and result caching to deliver fast, cost-efficient interactive SQL for BI workloads.
Question 3: How does Delta Lake achieve atomicity for write operations on object storage?
- By locking the entire S3 bucket during each write transaction
- By writing new data files and committing a new transaction log entry atomically (Correct answer)
- By using two-phase commit with a distributed coordinator service
- By staging all writes in an in-memory buffer before flushing to disk
Correct answer: By writing new data files and committing a new transaction log entry atomically
Delta Lake writes data files optimistically then commits a single JSON log entry; readers only see committed entries, ensuring atomic all-or-nothing visibility.
Question 4: In Unity Catalog's three-level namespace, what is the correct hierarchy from outermost to innermost?
- Schema → Catalog → Table
- Catalog → Schema → Table (Correct answer)
- Table → Schema → Catalog
- Workspace → Database → Table
Correct answer: Catalog → Schema → Table
Unity Catalog uses catalog.schema.table notation, where catalogs group schemas (databases) which in turn group tables and views.
Question 5: What is the role of the Delta Lake transaction log (_delta_log directory)?
- It stores compressed copies of all Parquet data files for backup
- It records every change to the table as ordered JSON commit files for ACID compliance (Correct answer)
- It caches the results of frequently executed SQL queries
- It maintains the cluster autoscaling history for cost reporting
Correct answer: It records every change to the table as ordered JSON commit files for ACID compliance
The _delta_log directory contains sequential JSON files that record file additions, removals, and metadata changes, forming the source of truth for table state.
Question 6: Which Lakehouse architecture pattern is best suited for continuously arriving IoT sensor data that must be available for queries within seconds?
- Batch ETL with nightly Delta table refreshes
- Lambda architecture with a separate speed and batch layer
- Streaming ingestion directly into Delta tables using Structured Streaming (Correct answer)
- Manual CSV uploads triggered by a daily cron job
Correct answer: Streaming ingestion directly into Delta tables using Structured Streaming
Spark Structured Streaming writes micro-batches directly to Delta tables, making new data queryable within seconds while maintaining ACID guarantees.
Question 7: What problem does 'small file compaction' solve in a Lakehouse storage layer?
- It reduces query latency caused by too many small Parquet files that slow metadata scans (Correct answer)
- It merges multiple Delta Lake catalogs into a single Unity Catalog namespace
- It compresses ML model artifacts before storing them in MLflow
- It batches multiple streaming checkpoints into a single write operation
Correct answer: It reduces query latency caused by too many small Parquet files that slow metadata scans
Streaming and frequent small writes create many tiny Parquet files; compaction (via OPTIMIZE) merges them into larger files, reducing file-open overhead during queries.
LFC Architecture 3
Which open table format feature allows querying a Delta table as it existed at a previous point in time?