Master of Data Science 1 Flashcards

Which ensemble learning method trains multiple models sequentially, where each model attempts to correct the errors made by the previous one?

Answer: Boosting

Boosting trains models sequentially, with each subsequent model focusing on the misclassified examples from the previous model. Bagging trains models independently in parallel, Random Forest is a specific bagging method, and stacking combines predictions using a meta-learner.

In Principal Component Analysis (PCA), what do the eigenvalues of the covariance matrix represent?

Answer: The amount of variance explained by each principal component

Eigenvalues represent the variance explained by each principal component. The eigenvectors (not eigenvalues) represent the directions of maximum variance. This is why eigenvalues are used to decide how many components to retain.

What distinguishes stochastic gradient descent (SGD) from batch gradient descent?

Answer: SGD updates model parameters using one (or a few) training examples per iteration rather than the entire dataset

SGD updates weights using a single sample (or mini-batch) at a time, making each update faster and introducing noise that can help escape local minima. Batch gradient descent uses the full dataset for each update, computing the true gradient but at higher computational cost.

In the context of a Master of Data Science program, what does 'data leakage' during model training refer to?

Answer: When information from the test set inadvertently influences the training process

Data leakage occurs when information outside the training data (typically from the validation or test set) is used to build the model, resulting in overly optimistic performance estimates that do not generalize. This is a critical pitfall in real-world data science workflows.

Which probability theorem is foundational to generative classifiers like Naive Bayes, allowing posterior class probabilities to be computed from likelihoods and priors?

Answer: Bayes' Theorem

Bayes' Theorem relates the posterior probability P(class|data) to the likelihood P(data|class) and the prior P(class). Naive Bayes classifiers apply this with the 'naive' assumption of conditional feature independence to make computation tractable.

What is the vanishing gradient problem in deep neural networks, and which architecture feature was specifically introduced to mitigate it?

Answer: Gradients shrink exponentially as they propagate to earlier layers; mitigated by residual (skip) connections

In deep networks, gradients computed during backpropagation are multiplied across many layers, causing them to shrink exponentially and making early-layer weights learn very slowly. Residual connections (introduced in ResNets) allow gradients to flow directly through skip connections, bypassing this multiplicative decay.