CRP Assessment & Evaluation Techniques 3 — Questions and Answers
Question 1: What does 'construct validity' mean in the context of a research instrument?
- The instrument consistently produces the same results over time
- The instrument measures the theoretical construct it is intended to measure (Correct answer)
- The instrument covers all dimensions of the domain being assessed
- The instrument predicts a future outcome accurately
Correct answer: The instrument measures the theoretical construct it is intended to measure
Construct validity refers to how well a test or measure actually captures the underlying theoretical concept it is supposed to assess.
Question 2: In cost-benefit analysis, the 'net present value' (NPV) is used to:
- Standardize qualitative outcomes across programs
- Account for the time value of money when comparing future costs and benefits (Correct answer)
- Rank programs by participant satisfaction scores
- Estimate sample size requirements for impact evaluations
Correct answer: Account for the time value of money when comparing future costs and benefits
NPV discounts future costs and benefits to their present value, enabling a meaningful comparison of programs with different time horizons.
Question 3: Which sampling method would be most efficient when a complete list of the target population does not exist?
- Simple random sampling
- Systematic sampling
- Respondent-driven sampling (RDS) (Correct answer)
- Stratified random sampling
Correct answer: Respondent-driven sampling (RDS)
Respondent-driven sampling is designed for hard-to-reach populations where no sampling frame exists, using peer referral chains.
Question 4: An evaluation that compares multiple programs or sites to identify best practices is best described as:
- Single-case experimental design
- Cross-site evaluation (Correct answer)
- Developmental evaluation
- Utilization-focused evaluation
Correct answer: Cross-site evaluation
Cross-site evaluations systematically compare findings across different programs or locations to identify effective models.
Question 5: Which type of error occurs when a researcher concludes there IS a significant effect when in reality there is none?
- Type II error (false negative)
- Type I error (false positive) (Correct answer)
- Measurement error
- Sampling error
Correct answer: Type I error (false positive)
A Type I error, or false positive, is the incorrect rejection of a true null hypothesis, concluding an effect exists when it does not.
Question 6: In qualitative research, 'member checking' is used primarily to:
- Verify that all participants signed consent forms
- Enhance credibility by having participants review and confirm findings (Correct answer)
- Ensure the sampling frame is representative of the population
- Assess inter-rater reliability among research team members
Correct answer: Enhance credibility by having participants review and confirm findings
Member checking strengthens credibility by returning findings or interpretations to participants to confirm accuracy and resonance.
Question 7: The counterfactual in an impact evaluation refers to:
- The cost of delivering the program per participant
- What would have happened to participants had they not received the intervention (Correct answer)
- The set of outcomes that contradict the program's theory of change
- Baseline data collected before program implementation
Correct answer: What would have happened to participants had they not received the intervention
The counterfactual represents the unobserved state — what outcomes would have looked like without the intervention — which is estimated through comparison groups.
CRP Assessment & Evaluation Techniques 3
What does 'construct validity' mean in the context of a research instrument?