Machine Learning Machine Learning 5 — Questions and Answers
Question 1: What is the difference between bagging and boosting in ensemble learning?
- Bagging uses regression; boosting uses classification
- Bagging trains models in parallel on random subsets; boosting trains sequentially, correcting prior errors (Correct answer)
- Bagging reduces variance; boosting reduces the number of features
- Bagging uses decision trees only; boosting uses any model type
Correct answer: Bagging trains models in parallel on random subsets; boosting trains sequentially, correcting prior errors
Bagging builds independent models in parallel on bootstrapped subsets to reduce variance, while boosting trains models sequentially to reduce bias by correcting previous mistakes.
Question 2: What is the purpose of a validation set as distinct from a test set?
- To provide additional training data when the training set is small
- To tune hyperparameters and select the best model without using the final held-out test set (Correct answer)
- To evaluate the final model's generalization performance
- To detect class imbalance before training
Correct answer: To tune hyperparameters and select the best model without using the final held-out test set
The validation set is used during development to compare models and tune hyperparameters, while the test set is held out and used only once for final unbiased evaluation.
Question 3: What does 'precision' measure in a classification model?
- Of all actual positives, what fraction were correctly predicted
- Of all predicted positives, what fraction were actually positive (Correct answer)
- The total fraction of correctly classified examples
- The model's ability to avoid false negatives
Correct answer: Of all predicted positives, what fraction were actually positive
Precision is the ratio of true positives to all predicted positives (TP / (TP + FP)), measuring how trustworthy positive predictions are.
Question 4: Which algorithm is used in the Expectation-Maximization (EM) framework for soft clustering?
- K-means
- Gaussian Mixture Models (GMM) (Correct answer)
- DBSCAN
- Hierarchical clustering
Correct answer: Gaussian Mixture Models (GMM)
Gaussian Mixture Models use the EM algorithm to assign soft probabilities of cluster membership based on fitting multiple Gaussian distributions to the data.
Question 5: What is the main advantage of using ROC-AUC over accuracy as an evaluation metric?
- ROC-AUC accounts for class imbalance by evaluating performance across all classification thresholds (Correct answer)
- ROC-AUC is faster to compute than accuracy
- ROC-AUC only applies to binary classification problems
- ROC-AUC penalizes false negatives more than false positives
Correct answer: ROC-AUC accounts for class imbalance by evaluating performance across all classification thresholds
ROC-AUC measures the area under the ROC curve across all decision thresholds, making it robust to class imbalance and providing a holistic view of classifier performance.
Question 6: In the context of neural networks, what is backpropagation?
- The process of feeding input data forward through the network
- The algorithm that computes gradients of the loss with respect to each weight by applying the chain rule backward through the network (Correct answer)
- A technique for initializing network weights
- The method for selecting the optimal learning rate
Correct answer: The algorithm that computes gradients of the loss with respect to each weight by applying the chain rule backward through the network
Backpropagation efficiently computes gradients for all weights by applying the chain rule of calculus backward from the output loss to each layer's parameters.
Question 7: What distinguishes a hyperparameter from a model parameter?
- Hyperparameters are learned during training; model parameters are set before training
- Model parameters are learned during training; hyperparameters are set before training and control the learning process (Correct answer)
- Hyperparameters only apply to neural networks; model parameters apply to all models
- Model parameters are always integers; hyperparameters can be continuous
Correct answer: Model parameters are learned during training; hyperparameters are set before training and control the learning process
Model parameters (like weights and biases) are learned from data during training, while hyperparameters (like learning rate and tree depth) are set prior to training and govern the learning process.
Machine Learning Machine Learning 5
What is the difference between bagging and boosting in ensemble learning?