Data Science with Python Certification Data Science with Python Supervised Learning Models 4 — Questions and Answers
Question 1: In a Gradient Boosting model, what are the subsequent trees trained to predict?
- The original target labels
- The residuals (errors) of the previous ensemble (Correct answer)
- Randomly selected subsets of features
- The probabilities output by the first tree
Correct answer: The residuals (errors) of the previous ensemble
Each new tree in gradient boosting fits the negative gradient of the loss function, which for MSE equals the residuals from the current ensemble's predictions.
Question 2: What is the Gini impurity used for in decision tree algorithms?
- Measuring model accuracy on the test set
- Selecting the best feature and threshold for each split (Correct answer)
- Pruning branches after the tree is fully grown
- Calculating the probability of each leaf class
Correct answer: Selecting the best feature and threshold for each split
Gini impurity quantifies how often a randomly chosen element would be misclassified, and the feature/threshold minimizing it is chosen for the split.
Question 3: You train a Naive Bayes classifier and encounter a feature value never seen during training for a class. What problem does this cause?
- Overfitting because the model memorizes training data
- Zero probability issue that collapses the posterior to zero (Correct answer)
- The model ignores that feature entirely
- The model defaults to the prior probability
Correct answer: Zero probability issue that collapses the posterior to zero
In Naive Bayes, a zero likelihood for any feature-class combination multiplies the entire posterior to zero; Laplace smoothing is used to fix this.
Question 4: When performing feature scaling, which supervised learning algorithms are LEAST sensitive to the scale of input features?
- Logistic Regression and SVM
- K-Nearest Neighbors and Neural Networks
- Decision Trees and Random Forests (Correct answer)
- Ridge Regression and Lasso
Correct answer: Decision Trees and Random Forests
Tree-based models split on feature thresholds independently for each feature, so the relative scale between features does not affect their performance.
Question 5: What does the ROC AUC score of 0.5 indicate about a binary classifier?
- The model is 50% accurate
- The model performs no better than random guessing (Correct answer)
- The model perfectly separates classes with some errors
- The model predicts all samples as the positive class
Correct answer: The model performs no better than random guessing
AUC of 0.5 corresponds to the diagonal of the ROC curve, meaning the classifier has no discriminating ability beyond random chance.
Question 6: In sklearn's Pipeline class, what is the correct order of steps?
- Any order — Pipeline reorders steps automatically
- Transformers first, then estimator last (Correct answer)
- Estimator first, then transformers
- Transformers and estimators can be freely mixed
Correct answer: Transformers first, then estimator last
sklearn's Pipeline requires all intermediate steps to be transformers (implement fit/transform) and only the final step to be an estimator.
Question 7: Which technique can be used to handle class imbalance in a supervised learning problem using scikit-learn?
- Setting class_weight='balanced' in the classifier (Correct answer)
- Using a larger test set
- Applying PCA before training
- Increasing max_iter for convergence
Correct answer: Setting class_weight='balanced' in the classifier
class_weight='balanced' automatically adjusts sample weights inversely proportional to class frequencies, giving more importance to minority classes.
Data Science with Python Certification Data Science with Python Supervised Learning Models 4
In a Gradient Boosting model, what are the subsequent trees trained to predict?