Data Science with Python Certification Data Science with Python Data Cleaning and Preparation 5 — Questions and Answers
Question 1: What is the key difference between `df.fillna(method='ffill')` and `df.interpolate(method='linear')`?
- ffill repeats the last value while interpolate estimates intermediate values by linear trend (Correct answer)
- interpolate always uses the column mean
- ffill only works on numeric columns
- Both produce identical results for numeric data
Correct answer: ffill repeats the last value while interpolate estimates intermediate values by linear trend
ffill propagates the last observed value unchanged, while linear interpolation calculates a proportional value between surrounding points.
Question 2: You need to flag rows where ANY column exceeds 3 standard deviations from the mean. Which expression achieves this?
- ((df - df.mean()) / df.std()).abs().gt(3).any(axis=1) (Correct answer)
- df.gt(df.mean() + 3)
- (df > 3 * df.std()).all(axis=1)
- df.zscore().gt(3)
Correct answer: ((df - df.mean()) / df.std()).abs().gt(3).any(axis=1)
Computing Z-scores, taking absolute values, checking gt(3), and using any(axis=1) identifies rows with at least one outlier column.
Question 3: Which scikit-learn transformer handles encoding of unknown categories seen at inference but not at training time?
- OrdinalEncoder(handle_unknown='use_encoded_value')
- LabelEncoder()
- OneHotEncoder(handle_unknown='ignore')
- Both A and C are valid approaches (Correct answer)
Correct answer: Both A and C are valid approaches
Both OrdinalEncoder with handle_unknown='use_encoded_value' and OneHotEncoder with handle_unknown='ignore' gracefully handle unseen categories.
Question 4: What does the `validate` parameter in `pd.merge()` do?
- Raises an error if the merge type (one-to-one, one-to-many) does not match expectations (Correct answer)
- Validates column dtypes before merging
- Checks for NaN values in the key columns
- Confirms row counts are equal after merge
Correct answer: Raises an error if the merge type (one-to-one, one-to-many) does not match expectations
Setting validate='one_to_one', 'one_to_many', or 'many_to_one' raises a MergeError if the actual relationship differs, catching unexpected duplicates.
Question 5: When splitting data into train and test sets, why must any imputation be fit ONLY on the training set?
- To prevent data leakage where test statistics influence training preprocessing (Correct answer)
- Because test data may have different dtypes
- To reduce computation time during cross-validation
- Because sklearn Pipelines require it
Correct answer: To prevent data leakage where test statistics influence training preprocessing
Fitting on all data before splitting leaks test set statistics into training, producing overly optimistic performance estimates.
Question 6: Which pandas method checks whether two DataFrames are equal, treating NaN as equal to NaN?
- df1.equals(df2) (Correct answer)
- df1 == df2
- df1.compare(df2)
- np.array_equal(df1, df2)
Correct answer: df1.equals(df2)
DataFrame.equals() considers NaN values in the same location as equal, unlike the == operator which returns NaN for NaN comparisons.
Question 7: What is the recommended way to apply different preprocessing steps to numeric and categorical columns within a single scikit-learn pipeline?
- Use ColumnTransformer to specify transformers per column subset (Correct answer)
- Run two separate pipelines and concatenate outputs manually
- Preprocess columns with pandas before passing to Pipeline
- Use FeatureUnion with column selectors
Correct answer: Use ColumnTransformer to specify transformers per column subset
ColumnTransformer dispatches different transformers to specified column subsets and assembles the results, integrating cleanly into a Pipeline.
Data Science with Python Certification Data Science with Python Data Cleaning and Preparation 5
What is the key difference between `df.fillna(method='ffill')` and `df.interpolate(method='linear')`?