AI AI Engineer: Machine Learning Fundamentals and Algorithms 1 — Questions and Answers
Question 1: What does the bias-variance tradeoff describe in machine learning?
- The balance between model complexity and generalization to unseen data (Correct answer)
- The relationship between training speed and model accuracy
- The ratio of labeled to unlabeled data in a dataset
- The tradeoff between precision and recall in classification
Correct answer: The balance between model complexity and generalization to unseen data
The bias-variance tradeoff describes how increasing model complexity reduces bias but increases variance, while simpler models have high bias but low variance — the goal is to minimize total error on unseen data.
Question 2: Which regularization technique adds the sum of the absolute values of model weights as a penalty term to the loss function?
- L2 (Ridge) regularization
- L1 (Lasso) regularization (Correct answer)
- Dropout regularization
- Elastic Net regularization
Correct answer: L1 (Lasso) regularization
L1 (Lasso) regularization penalizes the sum of absolute values of weights, which encourages sparsity by driving some weights to exactly zero, effectively performing feature selection.
Question 3: What is the primary purpose of k-fold cross-validation?
- To increase the size of the training dataset through data augmentation
- To obtain a more reliable estimate of model performance by using all data for both training and validation (Correct answer)
- To reduce training time by splitting data into smaller batches
- To select the optimal number of features for a model
Correct answer: To obtain a more reliable estimate of model performance by using all data for both training and validation
K-fold cross-validation splits data into k subsets, trains on k-1 folds, and validates on the remaining fold, rotating until all folds are used — giving a robust performance estimate without wasting data.
Question 4: Which evaluation metric is most appropriate when classes are severely imbalanced and the cost of false negatives is high?
- Accuracy
- Mean Squared Error
- F1 Score (Correct answer)
- R-squared
Correct answer: F1 Score
The F1 Score is the harmonic mean of precision and recall, making it suitable for imbalanced datasets where accuracy is misleading — it balances the cost of false positives and false negatives.
Question 5: What is the primary advantage of ensemble methods like Random Forest over a single decision tree?
- They require less computational power to train
- They are easier to interpret and visualize
- They reduce variance by aggregating predictions from multiple models (Correct answer)
- They eliminate the need for feature engineering
Correct answer: They reduce variance by aggregating predictions from multiple models
Random Forest reduces variance by averaging predictions from many decision trees trained on random subsets of data and features, resulting in better generalization than a single tree which tends to overfit.
Question 6: In gradient descent, what does the learning rate control?
- The number of training epochs before the model converges
- The size of the steps taken toward the minimum of the loss function (Correct answer)
- The proportion of data used in each training batch
- The threshold for classifying a prediction as positive
Correct answer: The size of the steps taken toward the minimum of the loss function
The learning rate determines how large each update step is during gradient descent — too large causes oscillation or divergence, while too small results in slow convergence or getting stuck in local minima.
Question 7: Which of the following best describes overfitting in a machine learning model?
- The model performs poorly on both training and test data
- The model performs well on training data but poorly on unseen test data (Correct answer)
- The model converges before reaching the global minimum of the loss function
- The model requires too many features to make accurate predictions
Correct answer: The model performs well on training data but poorly on unseen test data
Overfitting occurs when a model learns the noise and patterns specific to training data so well that it fails to generalize — resulting in high training accuracy but poor test accuracy.
AI AI Engineer: Machine Learning Fundamentals and Algorithms 1
What does the bias-variance tradeoff describe in machine learning?