ADE ADE Cloud & Big Data Technologies 2 — Questions and Answers
Question 1: What does the term 'data lakehouse' refer to?
- A data lake used exclusively for house pricing data
- An architecture combining the flexibility of data lakes with the management features of data warehouses (Correct answer)
- A physical on-premise data storage facility
- A backup solution for data warehouses
Correct answer: An architecture combining the flexibility of data lakes with the management features of data warehouses
A data lakehouse merges the low-cost storage of data lakes with ACID transactions, schema enforcement, and BI support from data warehouses.
Question 2: Which Apache technology is used for distributed message streaming and serves as a backbone for real-time data pipelines?
- Apache Flink
- Apache Kafka (Correct answer)
- Apache Hive
- Apache Sqoop
Correct answer: Apache Kafka
Apache Kafka is a distributed event streaming platform used to build real-time data pipelines and streaming applications at scale.
Question 3: In cloud-based big data architectures, what is 'schema-on-read'?
- Applying schema validation before data is loaded into storage
- Defining the schema when data is read rather than when it is stored (Correct answer)
- Using a fixed schema for all data lake files
- Reading schema definitions from a central metadata catalog
Correct answer: Defining the schema when data is read rather than when it is stored
Schema-on-read means data is stored in raw form and the schema is applied at query time, providing flexibility for diverse data types.
Question 4: Which Azure service is used to orchestrate large-scale data movement and transformation pipelines?
- Azure Databricks
- Azure Data Factory (Correct answer)
- Azure Event Hubs
- Azure HDInsight
Correct answer: Azure Data Factory
Azure Data Factory is a cloud-based ETL and data integration service for creating data-driven workflows to orchestrate data movement and transformation.
Question 5: What is the primary advantage of using columnar storage over row-based storage for analytical workloads?
- Faster inserts of individual records
- Improved read performance for aggregate queries on specific columns (Correct answer)
- Better support for transactional updates
- Lower memory usage during data loading
Correct answer: Improved read performance for aggregate queries on specific columns
Columnar storage allows analytical queries to read only the columns needed, drastically reducing I/O and improving aggregate query performance.
Question 6: Which AWS service provides a fully managed ETL service that automatically generates ETL code?
- AWS Lambda
- AWS Glue (Correct answer)
- Amazon EMR
- Amazon Athena
Correct answer: AWS Glue
AWS Glue is a serverless ETL service that discovers data schemas via crawlers and auto-generates PySpark or Scala ETL scripts.
ADE ADE Cloud & Big Data Technologies 2
What does the term 'data lakehouse' refer to?