Supervised Learning Models Flashcards

A data science team is building a model to predict the probability of customer churn (a binary outcome). The team decides to use a Logistic Regression model. Which of the following best describes the fundamental nature of Logistic Regression?

Answer: It is a classification algorithm that models the probability of a discrete outcome by applying a sigmoid function to a linear combination of features.

Logistic Regression is fundamentally a classification algorithm, not a regression algorithm, despite its name. It is used to predict a binary or categorical outcome. It calculates the probability of an instance belonging to a certain class by passing a linear combination of the input features through a logistic (sigmoid) function, which outputs a value between 0 and 1.

A team is developing a model to classify handwritten digits from 0 to 9. They are concerned about the model being too complex and memorizing the noise in the training data, which could lead to poor performance on new, unseen digits. Which supervised learning technique directly addresses this problem by adding a penalty term to the loss function to discourage overly complex models?

Answer: Regularization

Regularization is a set of techniques used to prevent overfitting in machine learning models. It works by adding a penalty term to the model's loss function, which discourages the model from assigning excessive weight to its parameters, thereby reducing complexity. Common types include L1 (Lasso) and L2 (Ridge) regularization.

Which of the following is a primary advantage of using a Random Forest model over a single Decision Tree?

Answer: Reduced risk of overfitting and improved generalization.

A Random Forest is an ensemble method that builds multiple decision trees and merges their predictions. This process of averaging predictions from many trees, each trained on a different subset of the data, significantly reduces the model's variance and makes it less prone to overfitting compared to a single, deep decision tree.

A data scientist is working with a dataset that has a non-linear decision boundary between two classes. Which algorithm is particularly effective at handling such data by implicitly mapping the input features into a higher-dimensional space?

Answer: Support Vector Machine (SVM) with a non-linear kernel

Support Vector Machines (SVMs) can efficiently perform a non-linear classification using what is called the kernel trick. By applying a non-linear kernel function (like the Radial Basis Function or Polynomial kernel), the algorithm can find a separating hyperplane in a higher-dimensional space without explicitly computing the coordinates of the data in that space, making it effective for non-linear problems.

A developer is building a simple recommendation system for an e-commerce site. The goal is to suggest products to a user based on the purchases of the 'k' most similar users. This approach is a direct application of which supervised learning algorithm?

Answer: K-Nearest Neighbors (KNN)

The K-Nearest Neighbors (KNN) algorithm is a non-parametric method used for classification and regression. In a classification context, it classifies a new data point based on the features of its 'k' nearest neighbors in the training set. This principle is directly applicable to recommendation systems, where 'neighbors' are similar users and the 'class' is the product to recommend.

The Naive Bayes classifier is a popular algorithm for text classification tasks like spam filtering. Its effectiveness relies on a simplifying assumption that is often not true in the real world. What is this 'naive' assumption?

Answer: That all features are conditionally independent of each other, given the class.

The 'naive' in Naive Bayes comes from its core assumption: that all input features are conditionally independent of each other given the class label. For example, in spam filtering, it assumes the presence of the word 'money' is independent of the presence of the word 'free', given that the email is spam. While this is rarely true, the algorithm often performs surprisingly well in practice.