AI AI Engineer: Machine Learning Fundamentals and Algorithms 2 — Questions and Answers
Question 1: What is the kernel trick in Support Vector Machines (SVM)?
- A method to reduce training time by approximating the decision boundary
- A technique that implicitly maps data to a higher-dimensional space to find a linear separator (Correct answer)
- A regularization method that penalizes support vectors far from the margin
- A pruning strategy that removes irrelevant features before training
Correct answer: A technique that implicitly maps data to a higher-dimensional space to find a linear separator
The kernel trick computes dot products in a higher-dimensional feature space without explicitly transforming the data, enabling SVMs to find linear decision boundaries for data that is nonlinearly separable in the original space.
Question 2: What is the primary difference between gradient boosting and bagging (e.g., Random Forest)?
- Bagging builds models sequentially, while gradient boosting builds them in parallel
- Gradient boosting builds models sequentially where each model corrects the errors of the previous one (Correct answer)
- Bagging is only applicable to regression tasks, while gradient boosting handles classification
- Gradient boosting averages predictions, while bagging uses voting to select the best model
Correct answer: Gradient boosting builds models sequentially where each model corrects the errors of the previous one
Gradient boosting builds an ensemble sequentially, where each new model focuses on correcting the residual errors of the combined previous models, reducing bias — unlike bagging which builds models independently in parallel to reduce variance.
Question 3: Why is feature standardization (zero mean, unit variance) important before applying algorithms like SVM or k-nearest neighbors?
- It reduces the number of training iterations required for convergence
- It prevents features with larger numerical ranges from dominating distance-based calculations (Correct answer)
- It ensures the model produces probabilistic outputs between 0 and 1
- It automatically removes irrelevant features from the dataset
Correct answer: It prevents features with larger numerical ranges from dominating distance-based calculations
Distance-based algorithms are sensitive to feature scale — a feature ranging 0–10,000 will dominate a feature ranging 0–1 in distance calculations, so standardization puts all features on equal footing.
Question 4: What does Principal Component Analysis (PCA) accomplish?
- It selects the most important features based on their correlation with the target variable
- It reduces dimensionality by projecting data onto orthogonal axes of maximum variance (Correct answer)
- It clusters data points into groups based on their proximity in feature space
- It generates synthetic training samples to balance imbalanced datasets
Correct answer: It reduces dimensionality by projecting data onto orthogonal axes of maximum variance
PCA finds orthogonal principal components (linear combinations of original features) ordered by the amount of variance they explain, allowing dimensionality reduction by keeping only the top components.
Question 5: In k-means clustering, how is the optimal number of clusters (k) typically determined?
- By using the default value of k=3 as it works for most datasets
- By setting k equal to the square root of the number of data points
- By using the elbow method, which plots inertia vs. k and identifies the point of diminishing returns (Correct answer)
- By running the algorithm with k=1 and incrementally adding clusters until accuracy stops improving
Correct answer: By using the elbow method, which plots inertia vs. k and identifies the point of diminishing returns
The elbow method plots the within-cluster sum of squares (inertia) against different values of k — the 'elbow' point where adding more clusters yields diminishing reductions in inertia suggests the optimal k.
Question 6: What is the purpose of a validation set, distinct from both the training set and test set?
- To provide additional data for training when the dataset is small
- To tune hyperparameters and select the best model without contaminating the final test evaluation (Correct answer)
- To evaluate the final model performance and report results to stakeholders
- To detect data drift after a model is deployed to production
Correct answer: To tune hyperparameters and select the best model without contaminating the final test evaluation
The validation set is used during development to tune hyperparameters and compare models — using the test set for this purpose would cause leakage, making the test set an unreliable measure of real-world performance.
Question 7: Which of the following statements best describes a naive Bayes classifier?
- It builds a decision boundary by maximizing the margin between classes
- It applies Bayes' theorem assuming conditional independence between features given the class label (Correct answer)
- It iteratively adjusts weights to minimize the cross-entropy loss between predictions and labels
- It partitions the feature space into rectangular regions using a series of threshold decisions
Correct answer: It applies Bayes' theorem assuming conditional independence between features given the class label
Naive Bayes applies Bayes' theorem to compute the posterior probability of each class and classifies based on the highest probability, making the 'naive' assumption that features are conditionally independent given the class.
AI AI Engineer: Machine Learning Fundamentals and Algorithms 2
What is the kernel trick in Support Vector Machines (SVM)?