CPP Program Evaluation & Outcomes 2 — Questions and Answers
Question 1: A prevention coalition wants to determine if their school-based program is being delivered as intended. Which evaluation type best addresses this question?
- Outcome evaluation
- Impact evaluation
- Fidelity/process evaluation (Correct answer)
- Cost-benefit analysis
Correct answer: Fidelity/process evaluation
Fidelity or process evaluation assesses whether a program is implemented as designed, checking dosage, reach, and adherence to the model.
Question 2: Which statistical measure is most appropriate when a prevention evaluator wants to compare pre/post scores on a continuous variable for the same group of participants?
- Independent samples t-test
- Chi-square test
- Paired samples t-test (Correct answer)
- ANOVA
Correct answer: Paired samples t-test
A paired samples t-test compares means from the same participants at two time points, making it ideal for pre/post single-group designs.
Question 3: In program evaluation, 'dose received' refers to:
- The number of sessions a facilitator delivers
- The amount of program content participants actually engage with (Correct answer)
- The cost per participant in a program
- The frequency of data collection intervals
Correct answer: The amount of program content participants actually engage with
Dose received measures the extent to which participants actually engaged with and absorbed program content, distinct from dose delivered by staff.
Question 4: A CPP evaluates a mentoring program and finds a statistically significant result with p=0.03 but an effect size of d=0.08. The most accurate interpretation is:
- The program has a large practical impact
- The result is statistically significant but the practical effect is negligible (Correct answer)
- The program should be immediately scaled up
- Effect size and p-value are redundant measures
Correct answer: The result is statistically significant but the practical effect is negligible
A small effect size (d=0.08) indicates minimal practical significance even when statistical significance is achieved, especially with large samples.
Question 5: Which design feature most strengthens causal inference in a prevention program evaluation?
- Larger sample size
- Random assignment to conditions (Correct answer)
- Longer follow-up periods
- Multiple data collection instruments
Correct answer: Random assignment to conditions
Random assignment (randomized controlled trial) is the gold standard for establishing causality by equating groups on known and unknown confounders.
Question 6: A logic model's 'outputs' are best described as:
- Long-term changes in community health indicators
- Direct products of program activities such as sessions held and materials distributed (Correct answer)
- Funding sources and in-kind contributions
- Changes in participant knowledge and attitudes
Correct answer: Direct products of program activities such as sessions held and materials distributed
Outputs are the direct, countable products of program activities (e.g., number of workshops conducted, participants served), not outcomes or impacts.
Question 7: Regression to the mean is a significant threat to internal validity when:
- A program selects participants based on extremely high or low scores (Correct answer)
- The evaluator uses a randomized control group
- Follow-up periods extend beyond 12 months
- Multiple comparison groups are used
Correct answer: A program selects participants based on extremely high or low scores
When participants are selected because of extreme scores, subsequent measures naturally tend toward the average regardless of the intervention, mimicking program effects.
CPP Program Evaluation & Outcomes 2
A prevention coalition wants to determine if their school-based program is being delivered as intended.
Which evaluation type best addresses this question?