MS-DS Master of Data science Data Wrangling and Preprocessing Questions and Answers — Questions and Answers
Question 1: A data scientist is working with a dataset containing customer information, including an 'Income' feature with a significant number of missing values. The distribution of the 'Income' feature is heavily right-skewed. Which of the following methods for handling missing data is most appropriate in this situation to minimize the impact of outliers?
- Deleting the rows with missing income values.
- Imputing the missing values with the mean of the 'Income' feature.
- Imputing the missing values with the median of the 'Income' feature. (Correct answer)
- Replacing missing values with a constant like zero.
Correct answer: Imputing the missing values with the median of the 'Income' feature.
For skewed distributions, the median is a more robust measure of central tendency than the mean because it is less affected by outliers. Deleting rows could lead to significant data loss, and replacing with zero would introduce bias and distort the feature's distribution.
Question 2: You are preparing a dataset for a K-Nearest Neighbors (KNN) algorithm. The dataset has features with vastly different scales: 'Age' (ranging from 20 to 70) and 'Salary' (ranging from 30,000 to 150,000). Why is feature scaling a critical step in this scenario?
- It reduces the dimensionality of the dataset.
- It ensures that features with larger ranges do not disproportionately influence the distance calculations. (Correct answer)
- It converts categorical features into a numerical format.
- It helps in identifying and removing outliers from the dataset.
Correct answer: It ensures that features with larger ranges do not disproportionately influence the distance calculations.
K-Nearest Neighbors is a distance-based algorithm. If features are not scaled, the 'Salary' feature, with its much larger range, would dominate the distance calculation, making the 'Age' feature almost irrelevant. Feature scaling, such as standardization or normalization, ensures all features contribute more equally.
Question 3: Which of the following data wrangling tasks involves creating new variables from existing ones to better represent the underlying patterns in the data for a machine learning model?
- Data Cleaning
- Feature Engineering (Correct answer)
- Data Integration
- Data Validation
Correct answer: Feature Engineering
Feature engineering is the process of using domain knowledge to create new features (variables) from the raw data that make machine learning algorithms work better. Data cleaning deals with errors and missing values, integration combines data sources, and validation checks data quality.
Question 4: A dataset contains a categorical feature 'Education Level' with the values: 'High School', 'Bachelor's', 'Master's', 'PhD'. This is an example of what type of data, and which encoding scheme would be most appropriate to preserve its inherent order?
- Nominal data; One-Hot Encoding
- Ordinal data; Label Encoding (Correct answer)
- Nominal data; Dummy Encoding
- Ordinal data; One-Hot Encoding
Correct answer: Ordinal data; Label Encoding
The 'Education Level' feature is ordinal because its categories have a meaningful, inherent order. Label Encoding is suitable here because it assigns a unique integer to each category (e.g., 0, 1, 2, 3), which preserves this ranking. One-Hot Encoding would treat each category as independent, losing the ordinal relationship.
Question 5: In the context of data preprocessing, what is the primary difference between Standardization and Normalization (Min-Max Scaling)?
- Standardization scales data to a fixed range, typically [0, 1], while Normalization centers the data around a mean of 0.
- Standardization is only used for normally distributed data, while Normalization can be used for any distribution.
- Standardization (Z-score) transforms data to have a mean of 0 and a standard deviation of 1, while Normalization scales data to a specific range (e.g., [0, 1]). (Correct answer)
- Normalization is more robust to outliers than Standardization.
Correct answer: Standardization (Z-score) transforms data to have a mean of 0 and a standard deviation of 1, while Normalization scales data to a specific range (e.g., [0, 1]).
Standardization rescales data to have a mean of zero and a standard deviation of one (Z-score). Normalization, specifically Min-Max scaling, rescales the data to a fixed range, usually 0 to 1. Standardization does not bind values to a specific range, which can be a key difference in certain algorithms.
Question 6: A data analyst is cleaning a large dataset and discovers that 5% of the records are exact duplicates. Which of the following is the most compelling reason to remove these duplicate records?
- To reduce the computational cost of training the model.
- To prevent the model from being biased towards the patterns present in the duplicated instances. (Correct answer)
- To decrease the number of features in the dataset.
- To ensure all data types are consistent across the records.
Correct answer: To prevent the model from being biased towards the patterns present in the duplicated instances.
Duplicate records can cause a machine learning model to give more weight to the patterns found in those duplicated entries, leading to overfitting and a biased model that may not generalize well to new, unseen data. While removing duplicates can reduce computational cost, the primary motivation is to prevent model bias.
MS-DS Master of Data science Data Wrangling and Preprocessing Questions and Answers
A data scientist is working with a dataset containing customer information, including an 'Income' feature with a significant number of missing values.
The distribution of the 'Income' feature is heavily right-skewed.
Which of the following methods for handling missing data is most appropriate in this situation to minimize the impact of outliers?