Machine Learning Classification 3 — Questions and Answers
Question 1: Which technique is used to handle class imbalance by creating synthetic minority class samples?
- Dropout
- SMOTE (Synthetic Minority Over-sampling Technique) (Correct answer)
- Principal Component Analysis
- Batch Normalization
Correct answer: SMOTE (Synthetic Minority Over-sampling Technique)
SMOTE generates new synthetic samples by interpolating between existing minority class examples and their nearest neighbors.
Question 2: In logistic regression, what is the role of the sigmoid function?
- It maps linear combinations of features to a probability between 0 and 1 (Correct answer)
- It computes the distance between data points and the decision boundary
- It normalizes the input features before training
- It selects the most relevant features from the input
Correct answer: It maps linear combinations of features to a probability between 0 and 1
The sigmoid function squashes the linear combination of inputs into the range (0, 1), allowing logistic regression to output class probabilities.
Question 3: What does the ROC curve plot?
- Precision vs. Recall at various thresholds
- True Positive Rate vs. False Positive Rate at various classification thresholds (Correct answer)
- Loss vs. number of training epochs
- Model accuracy vs. regularization strength
Correct answer: True Positive Rate vs. False Positive Rate at various classification thresholds
The ROC curve plots TPR (sensitivity) on the y-axis versus FPR (1 - specificity) on the x-axis across all possible classification thresholds.
Question 4: Which decision tree splitting criterion measures the average uncertainty weighted by each subset's size?
- Gini Impurity (Correct answer)
- Information Gain
- Gain Ratio
- Entropy
Correct answer: Gini Impurity
Gini impurity measures how often a randomly chosen element would be misclassified; weighted average across children is minimized at each split in CART.
Question 5: What is 'one-vs-rest' (OvR) strategy in multi-class classification?
- Training one classifier that outputs probabilities for all classes simultaneously
- Training one binary classifier per class to distinguish that class from all others combined (Correct answer)
- Selecting the single best binary feature for separating all classes
- Combining predictions from all pairs of classes using voting
Correct answer: Training one binary classifier per class to distinguish that class from all others combined
OvR trains N binary classifiers (one per class), each distinguishing its class from all remaining classes, then assigns the class with the highest confidence score.
Question 6: In a Random Forest classifier, how does each tree differ from the others?
- Each tree uses a different classification algorithm
- Each tree is trained on a bootstrap sample with a random subset of features at each split (Correct answer)
- Each tree is pruned to a different maximum depth
- Each tree is trained sequentially, correcting the previous tree's errors
Correct answer: Each tree is trained on a bootstrap sample with a random subset of features at each split
Random Forest introduces randomness through bootstrap sampling of data and random feature selection at each node, creating diverse, decorrelated trees.
Question 7: Which classification algorithm is most interpretable and mimics human decision-making through a flowchart-like structure?
- Neural Network
- Support Vector Machine
- Decision Tree (Correct answer)
- Gradient Boosting
Correct answer: Decision Tree
Decision trees create an explicit flowchart of if-else rules that can be visualized and easily explained to non-technical stakeholders.
Machine Learning Classification 3
Which technique is used to handle class imbalance by creating synthetic minority class samples?