Free MS-DS Master of Data science Statistical and Probabilistic Analysis Questions and Answers — Questions and Answers
Question 1: A data science team is conducting an A/B test on a new website feature. After collecting data, they calculate a p-value of 0.03. If their significance level (alpha) is set at 0.05, which of the following is the most accurate interpretation of the result?
- There is a 3% probability that the new feature has no effect on the outcome.
- The result is statistically significant, meaning there is sufficient evidence to reject the null hypothesis. (Correct answer)
- There is a 97% probability that the new feature is effective.
- The result is not statistically significant, and the team should accept the null hypothesis.
Correct answer: The result is statistically significant, meaning there is sufficient evidence to reject the null hypothesis.
The p-value represents the probability of observing the collected data, or more extreme data, assuming the null hypothesis is true. [6, 21] Since the p-value (0.03) is less than the significance level (alpha = 0.05), the result is considered statistically significant. This provides enough evidence to reject the null hypothesis, which typically states there is no effect or difference. [12]
Question 2: A data scientist is working with a dataset representing the daily number of users visiting a specific webpage. The distribution of this data is highly skewed to the right. To perform hypothesis testing on the average number of daily users, which statistical principle allows the use of a Z-test or t-test, provided the sample size is large enough?
- The Law of Large Numbers
- Bayes' Theorem
- The Central Limit Theorem (Correct answer)
- The Empirical Rule
Correct answer: The Central Limit Theorem
The Central Limit Theorem (CLT) states that the distribution of sample means will be approximately normal, regardless of the population's original distribution, as long as the sample size is sufficiently large (often cited as n > 30). [2, 3, 23] This allows for the use of parametric tests like Z-tests or t-tests, which assume normality of the sampling distribution. [25]
Question 3: Which of the following scenarios is best modeled by a Poisson distribution?
- The number of heads in 10 consecutive coin flips.
- The time until the next customer arrives at a coffee shop.
- The number of defective items in a batch of 100, where the probability of a defect is constant.
- The number of customer service calls received by a call center in one hour. (Correct answer)
Correct answer: The number of customer service calls received by a call center in one hour.
A Poisson distribution is used to model the number of events occurring within a fixed interval of time or space, given a known constant mean rate and independence of events. The number of calls per hour fits this description. [1, 15] A Binomial distribution is better for the coin flips and defective items (fixed number of trials). [14, 21] An Exponential distribution would model the time *between* events, not the count of events. [4]
Question 4: In the context of statistical analysis, which statement best describes the fundamental difference between the Frequentist and Bayesian approaches?
- Frequentists use probability to describe long-run frequencies of events, while Bayesians use it to quantify the degree of belief in a statement. [22] (Correct answer)
- Frequentist methods are only applicable to discrete data, whereas Bayesian methods are only for continuous data.
- Bayesian statistics always yields a single point estimate for a parameter, while Frequentist statistics provides a probability distribution.
- Frequentist statistics requires a prior distribution for parameters, while Bayesian statistics does not.
Correct answer: Frequentists use probability to describe long-run frequencies of events, while Bayesians use it to quantify the degree of belief in a statement. [22]
The core philosophical difference lies in the interpretation of probability. Frequentists view probability as the long-run frequency of an outcome in repeated experiments, treating population parameters as fixed, unknown constants. [9, 22] Bayesians, in contrast, use probability to represent a degree of belief or confidence about a parameter, which can be updated as more data becomes available. They treat parameters as random variables. [13, 16]
Question 5: A machine learning model is built to classify emails as 'spam' or 'not spam'. The model has a 99% accuracy rate. However, only 0.5% of all emails are actually spam. Why might accuracy be a misleading metric in this scenario?
- The model is likely overfitting to the training data.
- A high accuracy can be achieved by a naive model that classifies every email as 'not spam'. (Correct answer)
- The dataset is too small to draw any meaningful conclusions.
- Accuracy is not a valid metric for classification problems.
Correct answer: A high accuracy can be achieved by a naive model that classifies every email as 'not spam'.
This is a classic example of a class imbalance problem. If a model simply classifies all emails as 'not spam', it would be correct 99.5% of the time, achieving high accuracy but completely failing at its goal of identifying spam. In such cases, other metrics like Precision, Recall, or the F1-score provide a much better assessment of the model's performance on the minority class.
Question 6: A data scientist calculates a 95% confidence interval for the average user engagement time on a new app feature to be [15.2 minutes, 18.6 minutes]. What is the correct interpretation of this interval?
- There is a 95% probability that the true average engagement time is between 15.2 and 18.6 minutes.
- 95% of all users have an engagement time between 15.2 and 18.6 minutes.
- If we were to repeat this sampling process many times, 95% of the calculated confidence intervals would contain the true population mean. (Correct answer)
- The sample mean engagement time has a 95% chance of being the true population mean.
Correct answer: If we were to repeat this sampling process many times, 95% of the calculated confidence intervals would contain the true population mean.
A confidence interval's interpretation is based on the long-run frequency of the procedure. The 95% confidence level means that if the same sampling method were used to generate many different samples, we would expect 95% of the resulting confidence intervals to capture the true, unknown population mean. [2] It does not assign a probability to the true parameter being within a specific, calculated interval. [6]
Free MS-DS Master of Data science Statistical and Probabilistic Analysis Questions and Answers
A data science team is conducting an A/B test on a new website feature.
After collecting data, they calculate a p-value of 0.03.
If their significance level (alpha) is set at 0.05, which of the following is the most accurate interpretation of the result?