CPP Program Evaluation & Outcomes 3 — Questions and Answers
Question 1: Which evaluation framework is specifically designed to build evaluation capacity within organizations and involves stakeholders throughout the entire evaluation process?
- RAND Program Theory
- Participatory evaluation (Correct answer)
- Quasi-experimental design
- Return on investment analysis
Correct answer: Participatory evaluation
Participatory evaluation actively involves stakeholders—including program staff, clients, and community members—in designing and conducting the evaluation to build local capacity.
Question 2: The SAMHSA Strategic Prevention Framework Step 3 is 'Planning.' Which evaluation component is most directly aligned with this step?
- Developing the logic model and selecting evidence-based programs (Correct answer)
- Collecting baseline data on risk and protective factors
- Assessing community readiness for change
- Reporting outcomes to federal funders
Correct answer: Developing the logic model and selecting evidence-based programs
Step 3 Planning involves developing the logic model, selecting evidence-based interventions, and outlining the evaluation plan tied to identified needs.
Question 3: An evaluator discovers that participants who completed all program sessions had significantly better outcomes than those who only attended some sessions. This finding most likely reflects:
- A true program dose-response effect
- Selection bias—completers may differ systematically from non-completers
- Random measurement error
- Both A and B are plausible interpretations (Correct answer)
Correct answer: Both A and B are plausible interpretations
Dose-response patterns can reflect true program effects, but completers may also be more motivated or have fewer barriers, making causal attribution ambiguous without random assignment.
Question 4: In substance abuse prevention evaluation, 'protective factor' outcomes are important because they:
- Directly measure substance use rates
- Assess factors that buffer against risk and promote resilience (Correct answer)
- Replace the need for control groups
- Are required by all federal grant programs
Correct answer: Assess factors that buffer against risk and promote resilience
Protective factors such as family bonding, school connectedness, and positive peer norms buffer against risk factors and are key intermediate outcome targets in prevention.
Question 5: A prevention evaluator uses a comparison group from a neighboring community rather than random assignment. This design is called a:
- Randomized controlled trial
- Single-subject design
- Quasi-experimental design (Correct answer)
- Pre-experimental design
Correct answer: Quasi-experimental design
Quasi-experimental designs use nonrandomly assigned comparison groups to approximate experimental conditions and improve causal inference over pre-experimental designs.
Question 6: Which data source is most appropriate for measuring community-level changes in underage drinking rates for a population-level prevention initiative?
- Pre/post surveys of program participants
- Youth Risk Behavior Survey (YRBS) trend data (Correct answer)
- Staff activity logs
- Focus groups with community leaders
Correct answer: Youth Risk Behavior Survey (YRBS) trend data
The YRBS provides surveillance data on youth health behaviors at the population level, making it appropriate for tracking community-wide changes in underage drinking.
Question 7: When writing evaluation findings for a general community audience, a CPP should prioritize:
- Including all statistical tables and regression coefficients
- Translating findings into plain language with practical implications (Correct answer)
- Using technical jargon to establish credibility
- Reporting only statistically significant findings
Correct answer: Translating findings into plain language with practical implications
Communicating evaluation findings to lay audiences requires plain language, visual aids, and emphasis on practical meaning rather than statistical complexity.
CPP Program Evaluation & Outcomes 3
Which evaluation framework is specifically designed to build evaluation capacity within organizations and involves stakeholders throughout the entire evaluation process?