Statistics Professional Standards & Competencies 3 — Questions and Answers
Question 1: According to professional standards, what must a statistician do when they realize a previously published result contained an error?
- Quietly correct it in future work without alerting anyone
- Issue a correction or retraction in the original publication venue (Correct answer)
- Inform only the immediate client or employer
- Wait to see if others detect the error before acting
Correct answer: Issue a correction or retraction in the original publication venue
Professional ethics require prompt disclosure and correction of errors in published work through appropriate channels.
Question 2: The principle of 'reproducibility' in statistical practice primarily means:
- Obtaining the same p-value across different software packages
- Providing sufficient documentation so that another analyst can obtain the same results from the same data (Correct answer)
- Running the same analysis multiple times to confirm results
- Using random seed values consistently
Correct answer: Providing sufficient documentation so that another analyst can obtain the same results from the same data
Reproducibility requires transparent documentation of methods, code, and data so independent analysts can verify results.
Question 3: A statistician is asked to consult on a study outside their area of expertise. The most professionally responsible action is to:
- Accept and learn the domain as the project progresses
- Decline and refer the client to a more qualified statistician
- Accept but clearly disclose the limitations of their expertise (Correct answer)
- Request a higher fee to compensate for additional learning time
Correct answer: Accept but clearly disclose the limitations of their expertise
While declining is ideal, if accepting, the statistician must disclose expertise limitations so the client can make an informed decision.
Question 4: Which practice violates the principle of data integrity in statistical work?
- Using multiple imputation for missing data
- Removing outliers from a dataset without disclosure (Correct answer)
- Transforming skewed variables before analysis
- Weighting survey responses for nonresponse
Correct answer: Removing outliers from a dataset without disclosure
Removing outliers without documenting and disclosing the decision distorts the dataset and violates transparency standards.
Question 5: The 'replication crisis' in science primarily implicates which statistical practice?
- Using Bayesian instead of frequentist methods
- Overfitting models to small training datasets
- Routine underpowered studies combined with p-hacking and selective reporting (Correct answer)
- Excessive use of nonparametric tests
Correct answer: Routine underpowered studies combined with p-hacking and selective reporting
The replication crisis is largely attributed to underpowered studies, flexible analysis choices, and suppression of null results inflating the false positive rate.
Question 6: In the context of professional statistical practice, 'pre-registration' of a study primarily addresses which concern?
- Ensuring IRB approval is obtained before data collection
- Preventing outcome switching and p-hacking by committing to analysis plans in advance (Correct answer)
- Registering intellectual property rights to statistical methods
- Complying with federal funding agency reporting requirements
Correct answer: Preventing outcome switching and p-hacking by committing to analysis plans in advance
Pre-registration locks in hypotheses and analysis plans before data collection, preventing post-hoc flexibility that inflates false positives.
Question 7: A statistician is hired by a law firm as an expert witness. Their primary obligation should be to:
- Advocate for the position of the hiring law firm
- Provide an objective and accurate statistical analysis to the court (Correct answer)
- Minimize statistical uncertainty to simplify jury comprehension
- Match the conclusions of the opposing expert when possible
Correct answer: Provide an objective and accurate statistical analysis to the court
Expert witnesses have a duty to the court and truth, not to the hiring party; statistical testimony must be objective and accurate.
Statistics Professional Standards & Competencies 3
According to professional standards, what must a statistician do when they realize a previously published result contained an error?