AI Engineer: Machine Learning Fundamentals and Algorithms Flashcards

What does the bias-variance tradeoff describe in machine learning?

Answer: The balance between model complexity and generalization to unseen data

The bias-variance tradeoff describes how increasing model complexity reduces bias but increases variance, while simpler models have high bias but low variance — the goal is to minimize total error on unseen data.

Which regularization technique adds the sum of the absolute values of model weights as a penalty term to the loss function?

Answer: L1 (Lasso) regularization

L1 (Lasso) regularization penalizes the sum of absolute values of weights, which encourages sparsity by driving some weights to exactly zero, effectively performing feature selection.

What is the primary purpose of k-fold cross-validation?

Answer: To obtain a more reliable estimate of model performance by using all data for both training and validation

K-fold cross-validation splits data into k subsets, trains on k-1 folds, and validates on the remaining fold, rotating until all folds are used — giving a robust performance estimate without wasting data.

Which evaluation metric is most appropriate when classes are severely imbalanced and the cost of false negatives is high?

Answer: F1 Score

The F1 Score is the harmonic mean of precision and recall, making it suitable for imbalanced datasets where accuracy is misleading — it balances the cost of false positives and false negatives.

What is the primary advantage of ensemble methods like Random Forest over a single decision tree?

Answer: They reduce variance by aggregating predictions from multiple models

Random Forest reduces variance by averaging predictions from many decision trees trained on random subsets of data and features, resulting in better generalization than a single tree which tends to overfit.

In gradient descent, what does the learning rate control?

Answer: The size of the steps taken toward the minimum of the loss function

The learning rate determines how large each update step is during gradient descent — too large causes oscillation or divergence, while too small results in slow convergence or getting stuck in local minima.

Which of the following best describes overfitting in a machine learning model?

Answer: The model performs well on training data but poorly on unseen test data

Overfitting occurs when a model learns the noise and patterns specific to training data so well that it fails to generalize — resulting in high training accuracy but poor test accuracy.