Data Science Statistical Concepts and Analysis Questions and Answers 1

Question 1

A data scientist runs an A/B test on a new website feature and calculates a p-value of 0.03. The significance level (alpha) was set at 0.05. Which of the following is the most accurate interpretation of this result?

Accepted Answer

The result is statistically significant, providing evidence to reject the null hypothesis.

Answer

The p-value (0.03) is less than the pre-determined significance level, alpha (0.05). This indicates a statistically significant result. Therefore, we reject the null hypothesis, which typically states there is no difference or no effect. The p-value itself is the probability of observing the data (or more extreme data) if the null hypothesis were true, not the probability that the null hypothesis is true.

Question 2

In the context of the bias-variance tradeoff in machine learning, a model with high variance and low bias is most likely to exhibit which of the following?

Accepted Answer

Overfitting, where it performs very well on training data but poorly on test data.

Answer

High variance indicates that the model is highly sensitive to small fluctuations in the training data, essentially memorizing the noise along with the signal. This leads to excellent performance on the training set but poor generalization to new, unseen data (test set), a classic sign of overfitting. Low bias means the model is complex enough to capture the underlying patterns.

Question 3

A pharmaceutical company is testing a new drug. The null hypothesis is that the drug has no effect. A Type I error in this scenario would be:

Accepted Answer

Concluding the drug is effective when it actually is not.

Answer

A Type I error occurs when we incorrectly reject a true null hypothesis. In this case, the null hypothesis is that the drug has no effect. Rejecting this true null hypothesis means concluding the drug is effective when, in reality, it isn't. This is also known as a 'false positive'.

Question 4

A pollster wants to estimate the average income of adults in a large city. They take a random sample of 500 adults. According to the Central Limit Theorem, what can be said about the distribution of the sample mean income?

Accepted Answer

It will be approximately a normal distribution, regardless of the city's actual income distribution.

Answer

The Central Limit Theorem (CLT) states that the distribution of sample means will be approximately normal for a sufficiently large sample size, even if the original population distribution is not normal. This principle is fundamental for making inferences about a population mean from a sample mean.

Question 5

A data analyst calculates a 95% confidence interval for the average user engagement time on a new app feature to be [15.2 minutes, 18.6 minutes]. What is the correct interpretation of this interval?

Accepted Answer

If we were to repeat this sampling process many times, 95% of the calculated confidence intervals would contain the true average engagement time for all users.

Answer

A 95% confidence interval has a frequentist interpretation. It means that the method used to construct the interval will, in the long run, capture the true population parameter (in this case, the true average engagement time) in 95% of the samples taken. It does not give the probability that a single, specific interval contains the true mean.

Data Science Practice Test

Data Science Practice Test

Data Science Statistical Concepts and Analysis Questions and Answers 1