MS-DS Master of Data science Master of Data Science Machine Learning 1 — Questions and Answers
Question 1: Which cross-validation technique is most appropriate when the dataset is small and every data point must be used for both training and testing?
- K-fold cross-validation
- Leave-one-out cross-validation (Correct answer)
- Stratified cross-validation
- Holdout validation
Correct answer: Leave-one-out cross-validation
Leave-one-out cross-validation (LOOCV) trains the model on all data points except one, repeating this for every data point. This maximizes training data usage, making it ideal for small datasets where a fixed holdout split would waste too much data.
Question 2: In gradient boosting, each successive tree is trained to predict what?
- The original target values
- The residual errors of the previous ensemble (Correct answer)
- A random subset of features
- The mean of the target variable
Correct answer: The residual errors of the previous ensemble
Gradient boosting builds trees sequentially where each new tree fits the residual errors (pseudo-residuals) left by the previous ensemble. This additive correction process allows the model to progressively reduce error.
Question 3: What is the primary purpose of the kernel trick in Support Vector Machines?
- To reduce the number of support vectors
- To map data into a higher-dimensional space without computing the transformation explicitly (Correct answer)
- To normalize feature scales before classification
- To select the optimal regularization parameter C
Correct answer: To map data into a higher-dimensional space without computing the transformation explicitly
The kernel trick allows SVMs to operate in a high-dimensional feature space by computing dot products in that space implicitly, using a kernel function. This makes it computationally feasible to find non-linear decision boundaries without explicitly computing the high-dimensional mapping.
Question 4: A model has high training accuracy but significantly lower test accuracy. This pattern is most indicative of which problem?
- High bias
- Underfitting
- Overfitting (Correct answer)
- Data leakage prevention
Correct answer: Overfitting
A large gap between training accuracy and test accuracy indicates overfitting — the model has learned the training data's noise and specific patterns rather than generalizable relationships, causing it to perform poorly on unseen data.
Question 5: In Principal Component Analysis (PCA), the principal components are defined as:
- The features with the highest correlation to the target variable
- The eigenvectors of the feature covariance matrix, ordered by eigenvalue magnitude (Correct answer)
- The cluster centroids found by k-means on the feature space
- The features with the lowest variance across samples
Correct answer: The eigenvectors of the feature covariance matrix, ordered by eigenvalue magnitude
PCA computes the covariance matrix of the features and finds its eigenvectors. Each eigenvector (principal component) points in a direction of variance in the data, and the corresponding eigenvalue measures the amount of variance explained. Components are ordered from highest to lowest eigenvalue.
Question 6: Which metric is most appropriate for evaluating a binary classifier on a heavily imbalanced dataset where false negatives are very costly?
- Accuracy
- Recall (Sensitivity) (Correct answer)
- Specificity
- R-squared
Correct answer: Recall (Sensitivity)
Recall measures the proportion of actual positives correctly identified (TP / (TP + FN)). When false negatives are costly (e.g., missing a disease diagnosis) and the dataset is imbalanced, accuracy is misleading because a model predicting the majority class achieves high accuracy. Recall directly captures the model's ability to catch true positives.
MS-DS Master of Data science Master of Data Science Machine Learning 1
Which cross-validation technique is most appropriate when the dataset is small and every data point must be used for both training and testing?