Data Engineering Introduction To Data Engineering 3 — Questions and Answers
Question 1: In ELT, where does the transformation step occur compared to ETL?
- Inside the destination system after loading raw data (Correct answer)
- Before any data is extracted
- Only on the source application server
- It never transforms data
Correct answer: Inside the destination system after loading raw data
ELT loads raw data into the target first, then transforms it using the destination's compute power.
Question 2: What is data partitioning primarily used for?
- Improving query performance and manageability by splitting data into segments (Correct answer)
- Deleting old records automatically
- Changing column data types
- Compressing passwords
Correct answer: Improving query performance and manageability by splitting data into segments
Partitioning divides large datasets into segments so queries can scan less data and run faster.
Question 3: Which file format is columnar and commonly used in big data analytics?
- Parquet (Correct answer)
- TXT
- DOCX
- BMP
Correct answer: Parquet
Parquet is a columnar storage format optimized for analytical queries on large datasets.
Question 4: What does 'idempotency' mean for a data pipeline operation?
- Running it multiple times produces the same result as running it once (Correct answer)
- It can only run on weekends
- It always doubles the data
- It deletes the source after running
Correct answer: Running it multiple times produces the same result as running it once
An idempotent operation yields the same outcome no matter how many times it is executed.
Question 5: Which tool is widely used to orchestrate and schedule data workflows?
- Apache Airflow (Correct answer)
- Microsoft Word
- VLC Media Player
- GIMP
Correct answer: Apache Airflow
Apache Airflow orchestrates workflows as directed acyclic graphs (DAGs) with scheduling and monitoring.
Question 6: What is a primary key in a relational table?
- A column or set of columns that uniquely identifies each row (Correct answer)
- The first column alphabetically
- A backup copy of the table
- The largest value in a column
Correct answer: A column or set of columns that uniquely identifies each row
A primary key uniquely identifies every row and cannot contain null or duplicate values.
Question 7: What is the main benefit of data compression in storage systems?
- Reduced storage costs and faster I/O for large datasets (Correct answer)
- Increased password strength
- Better screen brightness
- Faster typing speed
Correct answer: Reduced storage costs and faster I/O for large datasets
Compression shrinks data size, lowering storage costs and often speeding up read/write operations.
Data Engineering Introduction To Data Engineering 3
In ELT, where does the transformation step occur compared to ETL?