SAC Data Analysis & Reporting 3 â Questions and Answers
Question 1: A student assistance counselor compares students' pre-program and post-program scores on a substance use attitude scale. This is an example of which evaluation design?
- Cross-sectional comparison group design
- Pre-experimental one-group pretest-posttest design (Correct answer)
- Randomized controlled trial
- Retrospective case study
Correct answer: Pre-experimental one-group pretest-posttest design
A one-group pretest-posttest design measures change within a single group from before to after an intervention, without a comparison group.
Question 2: Which statistic BEST describes the typical score in a data set when there are extreme outliers present?
- Mean
- Mode
- Median (Correct answer)
- Standard deviation
Correct answer: Median
The median is resistant to extreme values (outliers), making it the best measure of central tendency when a data set contains outliers.
Question 3: A program shows a statistically significant reduction in risk behavior. This finding PRIMARILY means that:
- The reduction is large enough to be clinically meaningful for students
- The result is unlikely to have occurred by chance alone (Correct answer)
- The program should be expanded immediately district-wide
- Students self-reported fewer risk behaviors due to social desirability
Correct answer: The result is unlikely to have occurred by chance alone
Statistical significance indicates that the probability of the observed result occurring by chance is below a defined threshold (typically p < .05), not that the effect is practically large.
Question 4: An 'outcome measure' in a student assistance program BEST refers to:
- The number of sessions delivered per student
- A change in student knowledge, attitude, or behavior following an intervention (Correct answer)
- The cost-per-student of a prevention program
- Staff credentials required to deliver services
Correct answer: A change in student knowledge, attitude, or behavior following an intervention
Outcome measures capture changes in the target populationâsuch as attitudes, knowledge, or behaviorsâthat result from program participation.
Question 5: Which of the following is a PROCESS (fidelity) measure rather than an outcome measure?
- Reduction in reported drug use at 6-month follow-up
- Improvement in students' refusal skills scores
- Percentage of curriculum lessons delivered as designed (Correct answer)
- Decrease in office discipline referrals after counseling
Correct answer: Percentage of curriculum lessons delivered as designed
Fidelity (process) measures document whether a program was implemented as intended, such as the percentage of curriculum sessions delivered correctly.
Question 6: When evaluating a substance use prevention program's effectiveness, a counselor should compare results to:
- The program developer's published success stories
- Baseline data collected from the same school before implementation (Correct answer)
- National arrest statistics for youth substance offenses
- The highest-performing school in the country
Correct answer: Baseline data collected from the same school before implementation
Comparing post-program data to the school's own baseline is the most internally valid method for detecting program-related change in the local context.
Question 7: A counselor notices that older students complete prevention surveys with significantly lower risk scores than younger students. Before concluding prevention is working, the counselor should consider:
- Whether the instrument was designed for younger students only
- The possibility that older students underreport due to social desirability bias
- Whether the program dosage was higher for older students
- All of the above as alternative explanations (Correct answer)
Correct answer: All of the above as alternative explanations
Multiple alternative explanationsâinstrument appropriateness, response bias, and dosage differencesâmust all be considered before attributing the finding to program effectiveness.
SAC Data Analysis & Reporting 3
A student assistance counselor compares students' pre-program and post-program scores on a substance use attitude scale.
This is an example of which evaluation design?