Business Analysis Statistics Test #3 — Questions and Answers
Question 1: Regression models that don't include duplicated input variables can:
- Improve parameter estimates' stability and raise overfitting's danger.
- Make parameter estimations less stable and make overfitting more likely.
- Enhance parameter estimates' stability and cut down on overfitting danger. (Correct answer)
- Destabilize parameter estimations to reduce overfitting danger.
Correct answer: Enhance parameter estimates' stability and cut down on overfitting danger.
Regression models that do not include duplicated or highly correlated (redundant) input variables generally benefit from enhanced parameter estimates' stability. Redundancy can lead to multicollinearity, making it difficult to determine the individual effect of each predictor and inflating standard errors. Removing such variables simplifies the model, improves interpretability, and reduces the risk of overfitting by focusing on truly independent contributions.
Question 2: An analyst is aware that storeId, a categorical predictor, is a key indicator of the objective. To be a workable predictor in the model, store Id has too many levels. In order to treat them as members of the same class level, the analyst wants to consolidate stores. What are the two most efficient solutions to the issue? (Select two.)
- Eliminate store_id as a predictor in the model because it has too many levels to be feasible.
- To maintain the stochastic variety in the observations, randomly group the stores into five groups.
- Combine stores that are related by using subject-matter expertise. (Correct answer)
- By combining similar stores using Greenacre's approach, you can cluster. (Correct answer)
Correct answer: Combine stores that are related by using subject-matter expertise.
When a categorical predictor like `storeId` has too many levels, it can lead to sparse data, overfitting, and computational issues. Two effective solutions involve grouping similar stores. One approach is to use subject-matter expertise to combine stores based on known characteristics like location, size, or customer demographics. Another sophisticated method is to employ clustering techniques, such as Greenacre's approach (often associated with correspondence analysis), to identify and group stores with similar response patterns, thereby reducing the number of levels while retaining predictive power.
Question 3: Which interpretation of the estimate is accurate?
- For every $1,000 increase in salary, the likelihood of the incident increases by 1.128.
- For every dollar of pay rise, the likelihood of the incident increases by 1.142.
- For every $1,000 increase in wage, the likelihood of the incident increases by 1.142. (Correct answer)
- For every dollar of pay rise, the likelihood of the incident increases by 1.142.
Correct answer: For every $1,000 increase in wage, the likelihood of the incident increases by 1.142.
Assuming 1.142 represents an odds ratio for a $1,000 increase in wage, the correct interpretation is that for every $1,000 increase in wage, the *odds* of the incident occurring multiply by 1.142. This means the likelihood (or odds) of the incident increases by 14.2% for each $1,000 wage increment. Option C best reflects this interpretation of a multiplicative change in likelihood per unit increase.
Question 4: Regression models with redundant input variables may:
- Improve parameter estimates' stability and raise overfitting's danger.
- Destabilize parameter estimations to reduce overfitting danger.
- Enhance parameter estimates' stability and cut down on overfitting danger.
- Make parameter estimations less stable and make overfitting more likely. (Correct answer)
Correct answer: Make parameter estimations less stable and make overfitting more likely.
Regression models containing redundant input variables, often due to multicollinearity, can lead to several problems. These include making parameter estimates less stable, meaning their values can fluctuate significantly with small changes in the data. Furthermore, redundant variables can increase the risk of overfitting, as the model might learn noise or specific patterns from the training data that do not generalize well to new data, leading to poorer performance on unseen observations.
Question 5: Which sampling techniques are suitable for data partitioning for model evaluation?
- Simple random sampling with replacement
- Sequential random sampling with replacement
- Simple random sampling without replacement (Correct answer)
- Stratified random sampling without replacement (Correct answer)
Correct answer: Simple random sampling without replacement
Simple random sampling without replacement is suitable because it ensures each observation has an equal chance of being selected for a partition and prevents duplicates across subsets, creating independent datasets for training, validation, and testing. Stratified random sampling without replacement is also suitable, especially for imbalanced datasets, as it ensures that each partition maintains the same proportion of key characteristics as the original dataset, leading to more representative and robust model evaluation. Both methods help in creating unbiased and independent data splits for model evaluation.
Question 6: What is a reasonable separation between training, validation, and testing data to undertake an honest evaluation of a predictive model?
- Training: 50% Validation: 50% Testing: 0% (Correct answer)
- Training: 100% Validation: 0% Testing: 0%
- Training: 50% Validation: 0% Testing: 50%
- Training: 0% Validation: 100% Testing: 0%
Correct answer: Training: 50% Validation: 50% Testing: 0%
A 50% training and 50% validation split is considered a reasonable separation for model evaluation, particularly during the development phase. This division allows the model to learn from a substantial portion of the data while reserving an equally large, independent portion for assessing its performance and generalization ability. While a dedicated test set is often used for final, unbiased evaluation, this split provides a robust way to tune hyperparameters and prevent overfitting during initial model development.
Question 7: Building a model that disproportionately over-represents those cases with an event occuring (e.g., a 50-50 event/non-event split) is a typical strategy for forecasting uncommon events in the LOGISTIprocedure. What issue does this bring up?
- Only the intercept estimate is biased (Correct answer)
- Only the non-intercept parameter estimates are biased.
- Sensitivity estimates are biased
- All parameter estimates are biased.
Correct answer: Only the intercept estimate is biased
When uncommon events are oversampled in logistic regression, it artificially inflates the prevalence of the event in the training data. This directly biases the intercept estimate, as the intercept reflects the log-odds of the event when all predictors are zero, which is now skewed by the altered prevalence. However, the non-intercept parameter estimates, which represent the relationship between predictors and the log-odds of the event, remain unbiased because the relative effects of the predictors on the odds ratio are preserved despite the oversampling.
Question 8: Data that were oversampled as a result of an uncommon target are turned into a confusion matrix. Which variables are unaffected by this oversampling?
- Sensitivity and Specificity (Correct answer)
- PV+ and PV-
- Specificity and PV-
- Sensitivity and PV+
Correct answer: Sensitivity and Specificity
Sensitivity (True Positive Rate) and Specificity (True Negative Rate) are unaffected by oversampling because they are calculated based on the conditional probabilities of correct classification within each true class. Sensitivity measures the proportion of actual positive cases correctly identified, while Specificity measures the proportion of actual negative cases correctly identified. Oversampling changes the overall prevalence of classes but does not alter the model's ability to correctly classify instances within their respective true classes, thus preserving these metrics.
Question 9: A model is created by an analyst utilizing the LOGISTIC process. The sensitivity and specificity statistics on a validation data set for various cutoff values are now of interest to them. What combination of options and statements will provide these statistics?
- Score data=valid1 out=roc;
- Score data=valid1 outroc=roc; (Correct answer)
- Mode1resp(event= '1') = gender region/outroc=roc;
- Mode1resp(event"1") = gender region/ out=roc;
Correct answer: Score data=valid1 outroc=roc;
In SAS's LOGISTIC procedure, the `SCORE` statement is used to apply a previously fitted model to a new dataset, such as a validation set. To obtain sensitivity and specificity statistics across various cutoff values for ROC curve analysis, the `OUTROC=` option must be specified within the `SCORE` statement. This option directs the procedure to output these performance metrics into a new dataset, allowing for a comprehensive evaluation of the model's discriminative power on the validation data.
Question 10: Which approach DOES NOT work well for scoring fresh data against a predetermined target in a logistic regression model?
- Use the SCORE statement in the LOGISTIC procedure.
- Augment the training data set with new observations and rerun the LOGISTIC procedure. (Correct answer)
- Augment the training data set with new observations and set their responses to missing.
- Use the saved parameter estimates from the LOGISTIC procedure and score new observations in the SCORE procedure.
Correct answer: Augment the training data set with new observations and rerun the LOGISTIC procedure.
Augmenting the training data with new observations and rerunning the LOGISTIC procedure is an inefficient and inappropriate method for scoring fresh data against a predetermined target. This approach would retrain the model, potentially altering its parameters based on the new data, rather than applying the existing, predetermined model. The standard and correct methods involve using the `SCORE` statement or applying saved parameter estimates to the new data, which ensures the original model is used for prediction without modification.
Question 11: Assume that it costs $10 to solicit a non-responder and earns $200 to do so. On an SAS data set called VALID, the logistic regression model produces a probability score called P R. The responder variable Pinch, a 1/0 variable coded as 1 for the responder, can be found in the VALID data set. When a customer's probability score is higher than 0.05, they will be contacted. Which SAS software in the data set calculates the profit for each customer?
- Profit=(P_R>0.05)*Purch*200-(P_R>.05)*(1-Purch)*10; (Correct answer)
- Profit=(P_R.05)*(1-Purch)*10;
- If P_R> 0.05; profit=(P_R>0.05)*Purch*200-(P_R>.05)*(1-Purch)*10;
- If P_R> 0.05;profit=(P_R>0.05)*Purch*200+(P_R
Correct answer: Profit=(P_R>0.05)*Purch*200-(P_R>.05)*(1-Purch)*10;
The SAS code in option A correctly calculates the profit for each customer based on the given business rules. The expression `(P_R>0.05)` acts as a binary indicator (1 if true, 0 if false) for whether a customer is contacted. If contacted, the first term `(P_R>0.05)*Purch*200` adds $200 for responders (`Purch=1`) and $0 for non-responders (`Purch=0`). The second term `-(P_R>.05)*(1-Purch)*10` subtracts $10 for non-responders (`Purch=0`) who were contacted, and $0 otherwise, accurately capturing earnings and costs.
Business Analysis Statistics Test #3
Regression models that don't include duplicated input variables can: