TIPS Research Methods & Evidence-Based Practice 3 — Questions and Answers
Question 1: A TIPS program evaluation finds a statistically significant reduction in over-service incidents after training. What does 'statistically significant' mean in this context?
- The reduction is large enough to be practically important
- The result is unlikely to be due to chance alone (Correct answer)
- The study was conducted by a reputable institution
- All participants agreed the training was useful
Correct answer: The result is unlikely to be due to chance alone
Statistical significance means the observed result would rarely occur by chance under the null hypothesis, typically indicated by a p-value below a predetermined threshold such as 0.05.
Question 2: Which type of bias is most likely to affect a study where TIPS-trained servers self-report their own compliance with responsible beverage service guidelines?
- Selection bias
- Recall bias
- Social desirability bias (Correct answer)
- Attrition bias
Correct answer: Social desirability bias
Social desirability bias occurs when respondents answer in ways they believe are viewed favorably rather than truthfully, a common issue in self-reported compliance studies.
Question 3: TIPS relies on evidence that alcohol affects judgment before motor coordination. Which area of the brain is primarily responsible for judgment and impulse control?
- Cerebellum
- Brain stem
- Prefrontal cortex (Correct answer)
- Occipital lobe
Correct answer: Prefrontal cortex
The prefrontal cortex governs executive functions including judgment, decision-making, and impulse control, and is highly sensitive to alcohol even at low BAC levels.
Question 4: A meta-analysis of server intervention studies shows an overall effect size of d=0.35. How should a TIPS trainer interpret this finding?
- The interventions had no measurable effect
- Server training produced a small but meaningful positive effect across studies (Correct answer)
- The interventions were highly effective and should be universally mandated
- Effect size is irrelevant without knowing the sample sizes
Correct answer: Server training produced a small but meaningful positive effect across studies
Cohen's d of 0.35 is generally considered a small-to-medium effect size, indicating a modest but real benefit of server training programs aggregated across multiple studies.
Question 5: When TIPS training materials cite research, they prioritize peer-reviewed studies over news articles primarily because peer review:
- Guarantees the study findings are correct
- Subjects the research to independent expert scrutiny before publication (Correct answer)
- Ensures the study was funded by a government agency
- Makes the study results legally defensible
Correct answer: Subjects the research to independent expert scrutiny before publication
Peer review requires independent experts to evaluate a study's methodology and conclusions before publication, increasing the credibility and rigor of the research.
Question 6: In a longitudinal study tracking DUI incidents at TIPS-certified bars over five years, which threat to internal validity is most concerning?
- Instrumentation changes
- History effects from concurrent DUI law changes (Correct answer)
- Experimenter expectancy bias
- Floor effects in measurement
Correct answer: History effects from concurrent DUI law changes
History effects occur when external events such as new DUI laws or enforcement campaigns happen during a study and could confound the measured impact of TIPS certification.
Question 7: Evidence-based TIPS guidelines recommend that servers document refusal of service incidents. The primary research rationale for this practice is to:
- Satisfy federal alcohol reporting requirements
- Create a record that can be used to identify patterns and support legal defense (Correct answer)
- Allow managers to discipline servers who refuse too often
- Provide data for marketing purposes
Correct answer: Create a record that can be used to identify patterns and support legal defense
Documentation of refusals creates an evidence trail that helps identify recurring problem patrons and provides a contemporaneous record supporting the establishment's due diligence if legal action occurs.
TIPS Research Methods & Evidence-Based Practice 3
A TIPS program evaluation finds a statistically significant reduction in over-service incidents after training.
What does 'statistically significant' mean in this context?