AML Supervised & Unsupervised Learning Algorithms 3 — Questions and Answers
Question 1: Which loss function is minimized by logistic regression during training?
- Mean Squared Error
- Hinge Loss
- Binary Cross-Entropy (Log Loss) (Correct answer)
- Huber Loss
Correct answer: Binary Cross-Entropy (Log Loss)
Logistic regression minimizes binary cross-entropy (negative log-likelihood), which penalizes confident wrong predictions more heavily than uncertain ones.
Question 2: What is the key difference between agglomerative and divisive hierarchical clustering?
- Agglomerative uses Euclidean distance; divisive uses cosine similarity
- Agglomerative starts with all points in one cluster; divisive starts with each point as its own cluster
- Agglomerative is top-down; divisive is bottom-up
- Agglomerative starts with each point as its own cluster and merges; divisive starts with one cluster and splits (Correct answer)
Correct answer: Agglomerative starts with each point as its own cluster and merges; divisive starts with one cluster and splits
Agglomerative clustering is bottom-up: each point starts as its own cluster, and pairs are merged iteratively; divisive is top-down, splitting clusters recursively.
Question 3: In a decision tree, information gain is computed as the reduction in which quantity after a split?
- Gini impurity
- Entropy (Correct answer)
- Mean Squared Error
- Log Loss
Correct answer: Entropy
Information gain measures the reduction in entropy (Shannon entropy) of the target variable achieved by splitting on a particular feature.
Question 4: What is the primary advantage of using Isolation Forest over LOF for anomaly detection?
- Isolation Forest produces calibrated anomaly probabilities
- Isolation Forest scales efficiently to high-dimensional, large datasets (Correct answer)
- Isolation Forest requires no hyperparameters
- Isolation Forest is supervised and needs labeled anomalies
Correct answer: Isolation Forest scales efficiently to high-dimensional, large datasets
Isolation Forest has linear time complexity and scales well to high dimensions and large datasets by isolating anomalies with random partitioning, unlike the quadratic-complexity LOF.
Question 5: Which scenario best illustrates the concept of 'label leakage' in a supervised classification task?
- Using cross-validation to select hyperparameters
- Including a feature derived from the target variable in training features (Correct answer)
- Splitting data into train/test sets before preprocessing
- Applying SMOTE only to the training fold during cross-validation
Correct answer: Including a feature derived from the target variable in training features
Label leakage occurs when a feature directly or indirectly encodes information about the target variable, causing artificially high training performance that won't generalize.
Question 6: In AdaBoost, what happens to the weights of incorrectly classified samples after each boosting round?
- Their weights are decreased so the next classifier focuses on different samples
- Their weights are increased so the next classifier prioritizes them (Correct answer)
- Their weights are reset to uniform to prevent overfitting
- Their weights remain unchanged; only the classifier weights change
Correct answer: Their weights are increased so the next classifier prioritizes them
AdaBoost increases the sample weights of misclassified instances, forcing subsequent weak learners to focus more attention on the harder-to-classify examples.
Question 7: What does the linkage criterion 'Ward's method' minimize when merging clusters in hierarchical clustering?
- Maximum pairwise distance between clusters
- Average pairwise distance between all inter-cluster point pairs
- Increase in total within-cluster variance after the merge (Correct answer)
- Difference between cluster centroid distances
Correct answer: Increase in total within-cluster variance after the merge
Ward's method merges the two clusters whose fusion causes the smallest increase in total within-cluster sum of squares, producing compact, roughly equal-sized clusters.
AML Supervised & Unsupervised Learning Algorithms 3
Which loss function is minimized by logistic regression during training?