CPE Evaluation Methods & Reporting 2 — Questions and Answers
Question 1: A program evaluator wants to understand participants' lived experiences with a job-training program. Which data collection method is MOST appropriate?
- Randomized controlled trial
- In-depth phenomenological interviews (Correct answer)
- Pre/post standardized testing
- Administrative records review
Correct answer: In-depth phenomenological interviews
Phenomenological interviews capture subjective lived experiences, making them ideal for understanding participant perspectives in qualitative evaluation.
Question 2: In a utilization-focused evaluation, the primary driver of evaluation design decisions is:
- Theoretical rigor and academic publication standards
- Intended use by intended users (Correct answer)
- Cost-effectiveness of data collection
- Evaluator's methodological expertise
Correct answer: Intended use by intended users
Utilization-focused evaluation, developed by Patton, centers all design decisions on how primary intended users will actually use the findings.
Question 3: Which threat to internal validity occurs when participants in a control group seek out alternative treatments because they know they are not receiving the intervention?
- Selection bias
- Compensatory rivalry (Correct answer)
- Compensatory equalization
- Diffusion of treatment
Correct answer: Compensatory rivalry
Compensatory rivalry (the 'John Henry effect') occurs when control group members work harder or seek alternatives because they know they are in the comparison group.
Question 4: A mixed-methods evaluation uses qualitative data to explain unexpected quantitative findings. This integration strategy is called:
- Triangulation design
- Explanatory sequential design (Correct answer)
- Exploratory sequential design
- Embedded design
Correct answer: Explanatory sequential design
Explanatory sequential design collects quantitative data first, then uses qualitative data to explain or elaborate on the quantitative results.
Question 5: When reporting evaluation findings to a community of color that has historically been exploited by researchers, an evaluator should FIRST:
- Present only statistically significant findings
- Engage community members in reviewing and interpreting the findings (Correct answer)
- Translate all materials into the community's primary language
- Secure IRB approval before sharing results
Correct answer: Engage community members in reviewing and interpreting the findings
Culturally responsive evaluation requires community engagement in interpreting findings to ensure accuracy and build trust with historically marginalized groups.
Question 6: A Gantt chart is used in evaluation planning primarily to:
- Display budget allocation across evaluation activities
- Illustrate the logic model's causal pathway
- Schedule and track evaluation tasks over time (Correct answer)
- Map stakeholder power and interest levels
Correct answer: Schedule and track evaluation tasks over time
A Gantt chart is a project management tool that visually displays evaluation tasks on a timeline, showing start/end dates and dependencies.
Question 7: Which reporting format is MOST appropriate when the primary audience for evaluation findings is frontline program staff who need actionable guidance?
- Peer-reviewed journal article
- Technical report with full methodology appendix
- Brief executive summary with specific recommendations (Correct answer)
- Oral presentation to the board of directors
Correct answer: Brief executive summary with specific recommendations
Frontline staff need concise, actionable information, making a brief summary with clear recommendations the most effective reporting format for this audience.
CPE Evaluation Methods & Reporting 2
A program evaluator wants to understand participants' lived experiences with a job-training program.
Which data collection method is MOST appropriate?