CSP Evaluation Research & Impact Assessment 2 — Questions and Answers
Question 1: In a logic model, which component represents the direct products of program activities, such as number of workshops delivered?
- Outcomes
- Outputs (Correct answer)
- Inputs
- Impacts
Correct answer: Outputs
Outputs are the direct, measurable products of program activities, distinct from longer-term outcomes.
Question 2: A program evaluator finds that participants improved post-test scores but a comparison group improved equally without the program. This suggests:
- Strong program effectiveness
- A maturation threat to internal validity
- A history threat to internal validity
- No program effect attributable to the intervention (Correct answer)
Correct answer: No program effect attributable to the intervention
When a comparison group shows identical gains, the improvement cannot be attributed to the program itself.
Question 3: Which cost analysis method compares program costs against outcomes measured in natural units rather than monetary values?
- Cost-benefit analysis
- Cost-effectiveness analysis (Correct answer)
- Return on investment analysis
- Social return on investment
Correct answer: Cost-effectiveness analysis
Cost-effectiveness analysis expresses outcomes in natural units (e.g., crimes prevented) rather than converting them to dollars.
Question 4: Developmental evaluation is best suited for:
- Stable, mature programs seeking summative judgments
- Innovative programs in dynamic, complex environments (Correct answer)
- Randomized controlled trials requiring control groups
- Programs with clearly defined logic models and fixed outcomes
Correct answer: Innovative programs in dynamic, complex environments
Developmental evaluation supports innovation and adaptation in complex, emergent program contexts.
Question 5: A key limitation of using administrative data in program evaluation is:
- It is always too expensive to collect
- It was collected for purposes other than evaluation, potentially limiting construct validity (Correct answer)
- It cannot be analyzed quantitatively
- It requires informed consent from all participants
Correct answer: It was collected for purposes other than evaluation, potentially limiting construct validity
Administrative data is collected for operational purposes, so it may not measure the constructs evaluators actually need.
Question 6: The counterfactual in impact evaluation refers to:
- The actual outcomes observed in program participants
- What would have happened to participants had they not received the program (Correct answer)
- The unintended negative consequences of a program
- The baseline measurement taken before program implementation
Correct answer: What would have happened to participants had they not received the program
The counterfactual represents the hypothetical scenario of what outcomes would have occurred without the intervention.
Question 7: Which evaluation approach emphasizes negotiating with multiple stakeholders who hold different, potentially conflicting values about a program?
- Goal-free evaluation
- Fourth-generation evaluation (Correct answer)
- Utilization-focused evaluation
- Theory-driven evaluation
Correct answer: Fourth-generation evaluation
Fourth-generation evaluation (Guba & Lincoln) centers on stakeholder constructions and negotiation of meaning.
CSP Evaluation Research & Impact Assessment 2
In a logic model, which component represents the direct products of program activities, such as number of workshops delivered?