Supervised Learning Algorithms Flashcards

A data scientist is building a model to classify emails as 'spam' or 'not spam'. The dataset is large and has many features (words). The scientist wants a model that is fast to train and makes a strong assumption that the presence of a particular word is unrelated to the presence of any other word. Which algorithm is most suitable for this task?

Answer: Naive Bayes

Naive Bayes is a probabilistic classifier that is well-suited for text classification tasks like spam filtering. Its core strength lies in the 'naive' assumption of conditional independence between features, which means it assumes that the presence of one word in an email is independent of the presence of others, given the class (spam or not spam). This assumption, while often not perfectly true in reality, allows the model to be trained very efficiently on high-dimensional data.

A financial institution wants to build a model to predict loan defaults. They have a dataset with many features, some of which are highly correlated. To prevent overfitting and improve model interpretability, they want to use a linear model that can also perform feature selection by shrinking some of the coefficients to exactly zero. Which of the following algorithms would be the best choice?

Answer: Lasso Regression (L1 Regularization)

Lasso Regression, which uses L1 regularization, is the ideal choice for this scenario. The L1 penalty adds the sum of the absolute values of the coefficients to the loss function. This has the effect of forcing the coefficients of less important or redundant features to become exactly zero, effectively performing feature selection and creating a more interpretable, sparse model. Ridge Regression (L2) shrinks coefficients towards zero but does not set them to exactly zero.

When dealing with a complex, non-linearly separable dataset, which technique allows a Support Vector Machine (SVM) to find a separating hyperplane?

Answer: The Kernel Trick

The Kernel Trick is a core concept in SVMs that allows them to handle non-linearly separable data. It works by implicitly mapping the input data into a higher-dimensional space where a linear separator (hyperplane) can be found. This is done efficiently without ever having to compute the coordinates of the data in that higher-dimensional space, which would be computationally expensive.

A developer is building a recommendation system for an e-commerce website. The goal is to recommend products to a user based on the products purchased by the 'K' most similar users. This is a classic application for which supervised learning algorithm?

Answer: K-Nearest Neighbors (KNN)

The K-Nearest Neighbors (KNN) algorithm is well-suited for recommendation systems. It is an instance-based learning algorithm that classifies a new data point based on the majority class of its 'K' nearest neighbors in the feature space. In this scenario, 'users' are the data points, and their purchase history defines their features. The algorithm finds the K most similar users and recommends products based on their behavior.

Which of the following best describes the primary advantage of using a Random Forest algorithm over a single Decision Tree?

Answer: Reduced risk of overfitting and lower variance.

The primary advantage of a Random Forest is its ability to reduce overfitting. A single Decision Tree is prone to overfitting because it can create a complex structure that memorizes the training data, including its noise. A Random Forest, which is an ensemble of many decision trees, mitigates this by training each tree on a random subset of data and features, and then averaging their predictions. This process reduces the model's variance and improves its generalization to new, unseen data.

In the context of ensemble learning, what is the key difference in how Bagging and Boosting train their base learners?

Answer: Bagging trains learners in parallel on different subsets of data, while Boosting trains them sequentially, with each learner focusing on the errors of the previous one.

The fundamental difference lies in the training process. Bagging (like Random Forest) trains multiple base learners independently and in parallel, each on a different bootstrap sample of the dataset. Boosting, on the other hand, trains learners sequentially. Each new learner is trained to correct the mistakes made by the previous learners by giving more weight to the misclassified instances. This sequential process is designed to reduce bias.