Supervised Learning: Classification Flashcards

A data science team is developing a model to predict customer churn (whether a customer will leave or stay). The dataset is imbalanced, with a significantly smaller number of 'churn' instances. The business priority is to identify as many customers who are likely to churn as possible, even if it means some non-churning customers are incorrectly flagged. Which evaluation metric should the team primarily focus on optimizing?

Answer: Recall (Sensitivity)

Recall (or Sensitivity) is the most appropriate metric in this scenario because it measures the model's ability to correctly identify all actual positive cases (customers who churned). The formula is True Positives / (True Positives + False Negatives). Prioritizing recall is crucial when the cost of false negatives (failing to identify a churner) is high. Accuracy can be misleading in imbalanced datasets, and precision (which focuses on the correctness of positive predictions) is less important than capturing all potential churners.

Which of the following is a key characteristic of classification problems in supervised learning?

Answer: The target variable is categorical.

In supervised learning, classification problems are defined by having a target variable that consists of a finite number of discrete categories or classes. The goal of the model is to learn a mapping from input features to predict the correct category for new, unseen data. Predicting a continuous value is a regression task, using unlabeled data is characteristic of unsupervised learning, and grouping similar data is clustering.

A team has built a binary classification model and visualized its performance using a confusion matrix. The matrix shows the following: True Positives (TP) = 80, False Positives (FP) = 20, True Negatives (TN) = 450, False Negatives (FN) = 50. What is the precision of this model?

Answer: 0.800

Precision is calculated as the ratio of correctly predicted positive observations to the total predicted positive observations. The formula is TP / (TP + FP). In this case, Precision = 80 / (80 + 20) = 80 / 100 = 0.80. This means that when the model predicts the positive class, it is correct 80% of the time.

Which of the following algorithms is NOT typically used for classification tasks?

Answer: K-Means Clustering

K-Means Clustering is an unsupervised learning algorithm used for partitioning data into a set number of clusters based on feature similarity. It does not use labeled data to predict a categorical outcome. Logistic Regression, Support Vector Machines (SVM), and Decision Trees are all popular and powerful supervised learning algorithms used for classification.

In the context of building a classification model, what is the primary purpose of a train/test split?

Answer: To evaluate the model's performance on unseen data and check for overfitting.

The dataset is split into a training set and a testing set. The model learns from the patterns in the training set. The test set, which the model has not been exposed to during training, is then used to provide an unbiased evaluation of how well the model generalizes to new, unseen data. This process is crucial for identifying overfitting, where a model performs well on training data but poorly on new data.

A financial institution is building a model to detect fraudulent credit card transactions. A 'False Positive' in this context would mean:

Answer: A legitimate transaction is incorrectly flagged as fraudulent.

In this scenario, the 'positive' class is a fraudulent transaction. A False Positive occurs when the model incorrectly predicts the positive class. Therefore, a legitimate (negative class) transaction that is incorrectly flagged as fraudulent (positive class) is a False Positive.