Machine Learning Supervised Learning 5 — Questions and Answers
Question 1: What is 'data leakage' in a supervised learning pipeline?
- Training data being stored insecurely
- Information from the test set inadvertently influencing model training (Correct answer)
- Features being removed from the dataset
- Overfitting due to too many training epochs
Correct answer: Information from the test set inadvertently influencing model training
Data leakage occurs when information outside the training set (e.g., from the test set) is used during training, causing artificially inflated performance metrics.
Question 2: Which of the following is NOT a hyperparameter of a Random Forest model?
- Number of trees in the forest
- Maximum depth of each tree
- Number of features considered at each split
- Learned feature weights of the trees (Correct answer)
Correct answer: Learned feature weights of the trees
Learned feature weights are parameters determined by the training process, not hyperparameters set before training.
Question 3: When using one-hot encoding for categorical features, what problem can arise with high-cardinality variables?
- Values become negative
- Dimensionality explosion with many sparse binary columns (Correct answer)
- The model cannot converge
- Labels become continuous
Correct answer: Dimensionality explosion with many sparse binary columns
High-cardinality categorical variables create an extremely large and sparse feature matrix when one-hot encoded, increasing memory and computation costs.
Question 4: In multi-class classification with a neural network, which output activation function is most commonly used?
- Sigmoid
- ReLU
- Softmax (Correct answer)
- Tanh
Correct answer: Softmax
Softmax outputs a probability distribution over all classes (summing to 1), making it suitable for multi-class classification output layers.
Question 5: What is the 'elbow method' used for in supervised learning?
- Determining the optimal number of clusters in k-means
- Selecting the best k value for k-NN by plotting error vs. k (Correct answer)
- Identifying the best learning rate for gradient descent
- Choosing the number of trees in a Random Forest
Correct answer: Selecting the best k value for k-NN by plotting error vs. k
The elbow method plots validation error against different k values in k-NN; the optimal k is at the 'elbow' where error stops decreasing significantly.
Question 6: Which technique addresses class imbalance by generating synthetic minority class samples?
- PCA (Principal Component Analysis)
- SMOTE (Synthetic Minority Over-sampling Technique) (Correct answer)
- L2 Regularization
- Bootstrap aggregation (Bagging)
Correct answer: SMOTE (Synthetic Minority Over-sampling Technique)
SMOTE creates synthetic samples by interpolating between existing minority class instances, helping to balance the class distribution.
Question 7: What does a high AUC-ROC score (close to 1.0) indicate about a classification model?
- The model has low training accuracy
- The model perfectly separates the positive and negative classes (Correct answer)
- The model is severely overfitting
- The model requires more training data
Correct answer: The model perfectly separates the positive and negative classes
An AUC-ROC near 1.0 means the model ranks positive instances higher than negative ones across all thresholds, indicating excellent discriminative ability.
Machine Learning Supervised Learning 5
What is 'data leakage' in a supervised learning pipeline?