MS-DS Master of Data science Master of Data Science Research & Data Analysis 1

Question 1

A data scientist splits a dataset into 80% training and 20% test sets, trains a complex neural network, and reports excellent test accuracy. A colleague suggests the model may still be overfit. Which additional evaluation strategy would best reveal this?

Accepted Answer

Use k-fold cross-validation on the full dataset and compare variance across folds

Answer

K-fold cross-validation assesses model stability across multiple train/test splits. High variance in scores across folds indicates overfitting that a single train/test split can miss, since that split may have been favorable by chance.

Question 2

A researcher is building a predictive model on a highly imbalanced dataset where only 2% of records belong to the positive class. The model achieves 98% accuracy but fails to detect any positive cases. What is the most appropriate corrective step?

Accepted Answer

Use stratified sampling and evaluate with F1-score or AUC-ROC instead of accuracy

Answer

When classes are imbalanced, accuracy is a misleading metric since predicting the majority class always yields high accuracy. Stratified sampling preserves class distribution across splits, and metrics like F1-score or AUC-ROC properly account for performance on the minority class.

Question 3

In a longitudinal study tracking student performance over four years, a researcher uses ordinary least squares regression and finds a statistically significant effect of tutoring hours. A reviewer raises concern about autocorrelation in the residuals. Which test should the researcher run first?

Accepted Answer

Durbin-Watson test

Answer

The Durbin-Watson test specifically detects autocorrelation in regression residuals. Autocorrelation is a common problem in time-series and longitudinal data because observations close in time tend to be correlated, violating the OLS independence assumption.

Question 4

A master's student wants to understand which of 30 candidate features most strongly drive customer churn. She has limited domain knowledge and needs an interpretable ranking. Which approach is most appropriate as a first step?

Accepted Answer

Compute mutual information scores between each feature and the target variable

Answer

Mutual information measures the statistical dependency between each feature and the target without assuming a linear relationship, making it a model-agnostic and interpretable first-pass feature ranking. PCA creates new components rather than ranking original features, making it less useful for interpretability.

Question 5

A data science researcher observes a strong positive correlation (r = 0.85) between ice cream sales and drowning rates across U.S. counties. What is the most accurate interpretation of this finding?

Accepted Answer

The correlation is spurious and likely driven by a confounding variable such as temperature or season

Answer

This is a classic example of confounding: warm weather increases both ice cream consumption and swimming activity, creating a spurious correlation between the two. Correlation alone never implies causation, and researchers must consider confounders before drawing causal conclusions.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Master of Data Science Research & Data Analysis 1