MS-DS Master of Data science Statistical Inference Concepts Questions and Answers

Question 1

A data scientist at an e-commerce company runs an A/B test on a new checkout button design. The null hypothesis is that the new button has no effect on the conversion rate. After analyzing the results, they obtain a p-value of 0.03. Assuming a significance level (alpha) of 0.05, what is the correct interpretation of this p-value?

Accepted Answer

If the null hypothesis were true, there would be a 3% probability of observing a difference in conversion rates at least as extreme as the one detected.

Answer

The p-value is the probability of observing the collected data, or something more extreme, assuming the null hypothesis is true. A p-value of 0.03 means that if there were truly no difference between the buttons (H0 is true), there would only be a 3% chance of seeing a result this strong or stronger just due to random sampling variation. Since 0.03 is less than the alpha of 0.05, the result is statistically significant, and we reject the null hypothesis.

Question 2

A data scientist calculates a 95% confidence interval for the average user session duration to be [150, 170] seconds. Which of the following is the correct interpretation of this interval?

Accepted Answer

If we were to repeat this sampling process many times, we would expect 95% of the calculated confidence intervals to contain the true population mean session duration.

Answer

A 95% confidence interval relates to the reliability of the estimation procedure, not a specific interval. It means that if the same sampling method were used to generate many different samples and a confidence interval were calculated for each, about 95% of these intervals would capture the true, unknown population parameter. Common misinterpretations include stating there's a 95% probability the true mean is in the specific interval or that 95% of the data points lie within the interval.

Question 3

A pharmaceutical company conducts a clinical trial for a new drug. The null hypothesis (H0) is that the drug has no effect. The alternative hypothesis (H1) is that the drug has a positive effect. They conclude the drug is effective (reject H0), but in reality, it is not. In the context of hypothesis testing, what type of error has been made?

Accepted Answer

Type I Error

Answer

A Type I error occurs when the null hypothesis is incorrectly rejected when it is actually true. In this scenario, the null hypothesis (the drug has no effect) was true, but the company concluded it was effective, thereby rejecting a true null hypothesis. This is also known as a 'false positive'.

Question 4

Which of the following statements best describes the primary utility of the Central Limit Theorem (CLT) for a data scientist conducting statistical inference?

Accepted Answer

It allows for the use of statistical tests assuming a normal distribution for the sample mean's sampling distribution, even if the population distribution is non-normal, given a sufficiently large sample size.

Answer

The Central Limit Theorem states that the sampling distribution of the sample mean will be approximately normally distributed, regardless of the shape of the original population distribution, as long as the sample size is sufficiently large (often cited as n > 30). This is crucial because many common inferential tests (like t-tests and the construction of confidence intervals for means) rely on the assumption of normality. The CLT provides the justification for using these tests on data from non-normal populations.

Question 5

In the context of designing an experiment, what does the 'statistical power' of a hypothesis test represent?

Accepted Answer

The probability of correctly rejecting the null hypothesis when it is in fact false.

Answer

Statistical power is the probability of detecting an effect if there is a real effect to be detected. It is defined as 1 - β (the probability of a Type II error). A test with high power has a high probability of correctly finding a true effect and rejecting a false null hypothesis.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Statistical Inference Concepts Questions and Answers