CAP Policy Analysis & Evaluation 2 — Questions and Answers
Question 1: Which evaluation framework assesses whether a nonprofit's activities actually caused the observed outcomes, as opposed to other factors?
- Needs assessment
- Counterfactual analysis (Correct answer)
- Stakeholder mapping
- Logic model review
Correct answer: Counterfactual analysis
Counterfactual analysis compares what happened with what would have occurred absent the intervention, establishing causation.
Question 2: A philanthropic advisor reviewing a housing policy finds that participants self-selected into the program. This introduces which threat to evaluation validity?
- Attrition bias
- Selection bias (Correct answer)
- Instrumentation bias
- Hawthorne effect
Correct answer: Selection bias
Selection bias occurs when participants differ systematically from non-participants, skewing outcome comparisons.
Question 3: In policy analysis, 'cost-effectiveness analysis' differs from 'cost-benefit analysis' primarily because it:
- Ignores administrative overhead
- Expresses outcomes in monetary terms
- Measures outcomes in natural units rather than dollars (Correct answer)
- Requires randomized control trials
Correct answer: Measures outcomes in natural units rather than dollars
Cost-effectiveness analysis compares costs per unit of outcome (e.g., cost per life saved) without monetizing the outcome itself.
Question 4: Which stage of the policy cycle is most directly informed by program evaluation findings?
- Agenda setting
- Policy formulation
- Policy revision and termination (Correct answer)
- Legislative drafting
Correct answer: Policy revision and termination
Evaluation findings feed into policy revision or termination decisions by demonstrating whether a program is achieving its goals.
Question 5: A CAP advisor wants to understand stakeholders' subjective experiences with a charitable housing program. The most appropriate method is:
- Randomized control trial
- In-depth qualitative interviews (Correct answer)
- Administrative data mining
- Quasi-experimental design
Correct answer: In-depth qualitative interviews
In-depth qualitative interviews capture nuanced stakeholder perspectives and lived experiences that quantitative methods miss.
Question 6: When evaluating a multi-year philanthropic initiative, 'formative evaluation' is best described as:
- Assessment conducted after the program ends
- Ongoing evaluation used to improve the program during implementation (Correct answer)
- Measurement of long-term community impact
- Review of financial statements by external auditors
Correct answer: Ongoing evaluation used to improve the program during implementation
Formative evaluation gathers data during implementation to guide program improvements before the final assessment.
Question 7: Which indicator would be classified as an 'outcome' rather than an 'output' in a program logic model?
- Number of workshops delivered
- Pounds of food distributed
- Percentage of participants who gained employment (Correct answer)
- Number of staff trained
Correct answer: Percentage of participants who gained employment
Outcomes reflect changes in knowledge, skills, attitudes, or conditions for participants, whereas outputs are the direct products of activities.
CAP Policy Analysis & Evaluation 2
Which evaluation framework assesses whether a nonprofit's activities actually caused the observed outcomes, as opposed to other factors?