Data Warehousing on AWS Training Data Lake Integration 2 — Questions and Answers
Question 1: What AWS service acts as the centralized metadata catalog for organizing and discovering data lake assets?
- AWS CloudTrail
- AWS Glue Data Catalog (Correct answer)
- Amazon CloudWatch
- AWS Config
Correct answer: AWS Glue Data Catalog
The AWS Glue Data Catalog is a centralized metadata repository that stores table definitions, schema information, and data location metadata used by Athena, Redshift Spectrum, and EMR.
Question 2: What is the primary function of AWS Glue crawlers in a data lake integration workflow?
- Monitor network traffic into S3 buckets
- Automatically discover data sources, infer schemas, and populate the Glue Data Catalog (Correct answer)
- Transfer data from on-premises databases to AWS
- Encrypt objects stored in Amazon S3
Correct answer: Automatically discover data sources, infer schemas, and populate the Glue Data Catalog
Glue crawlers scan configured data stores, infer schemas from the data, and automatically create or update table definitions in the Glue Data Catalog.
Question 3: AWS Lake Formation adds which key governance capability on top of S3 and the Glue Data Catalog?
- Automatic cross-region data replication
- Fine-grained access control at the table, column, and row level for data lake resources (Correct answer)
- Real-time change data capture from operational databases
- Automatic SQL query optimization across all analytics engines
Correct answer: Fine-grained access control at the table, column, and row level for data lake resources
Lake Formation provides centralized, fine-grained permissions (table, column, row, and cell level) enabling consistent data governance across analytics services that query the data lake.
Question 4: Which connection type does AWS Glue use when writing ETL job output directly into Amazon Redshift?
- HTTP REST API calls to the Redshift endpoint
- JDBC connection using a Redshift-compatible driver (Correct answer)
- SSH tunnel through a bastion host
- S3 Transfer Acceleration upload
Correct answer: JDBC connection using a Redshift-compatible driver
AWS Glue connects to Amazon Redshift using a JDBC connection, which allows ETL jobs to read from and write to Redshift tables.
Question 5: What is the recommended method for loading large datasets from Amazon S3 into Amazon Redshift?
- Use single-row INSERT INTO statements for each record
- Use the COPY command to load data in parallel across all compute nodes (Correct answer)
- Use AWS DMS for every S3-to-Redshift data load
- Use the AWS Management Console's manual file upload feature
Correct answer: Use the COPY command to load data in parallel across all compute nodes
The COPY command leverages Redshift's massively parallel processing architecture to load data from S3 concurrently across all nodes, making it significantly faster than row-level inserts.
Question 6: In a data lake context, what does 'schema-on-read' mean?
- Schemas are enforced at the moment data is written to S3
- Data is validated against a fixed schema during ETL transformation
- The schema is applied and interpreted at query time rather than when data is stored (Correct answer)
- Only structured data conforming to a predefined schema can be stored in a data lake
Correct answer: The schema is applied and interpreted at query time rather than when data is stored
Schema-on-read stores data in its raw format and applies the schema at query time, providing flexibility to analyze the same data with different schemas for different use cases.
Question 7: Which S3 data organization strategy most effectively improves Redshift Spectrum query performance and reduces cost?
- Store all data in a single flat S3 prefix with no partitioning
- Partition data by frequently filtered columns such as date, region, or category (Correct answer)
- Distribute files randomly across multiple S3 buckets
- Consolidate all data into one large uncompressed file per table
Correct answer: Partition data by frequently filtered columns such as date, region, or category
Partitioning S3 data by commonly filtered columns enables Spectrum to prune irrelevant partitions, reducing the amount of data scanned and lowering per-query cost.
Data Warehousing on AWS Training Data Lake Integration 2
What AWS service acts as the centralized metadata catalog for organizing and discovering data lake assets?