CGA Assessment & Evaluation Methods 2 — Questions and Answers
Question 1: Which evaluation design is most appropriate when random assignment to treatment and control groups is not feasible?
- Randomized controlled trial
- Quasi-experimental design (Correct answer)
- Pre-experimental design
- Cross-sectional survey
Correct answer: Quasi-experimental design
Quasi-experimental designs allow for causal inference when random assignment is not possible by using comparison groups or statistical controls.
Question 2: A grant evaluator uses a Difference-in-Differences (DiD) approach. What does this method compare?
- Pre- and post-scores only for the treatment group
- Post-scores of treatment vs. control groups
- Changes over time in treatment vs. control groups (Correct answer)
- Baseline characteristics of participants
Correct answer: Changes over time in treatment vs. control groups
DiD compares the change in outcomes over time between a treatment group and a control group, removing shared time trends.
Question 3: Which term describes the unintended positive outcomes of a grant-funded program discovered during evaluation?
- Spillover effects (Correct answer)
- Displacement effects
- Deadweight loss
- Confounding variables
Correct answer: Spillover effects
Spillover effects (also called positive externalities) are unintended benefits that extend beyond the program's direct participants.
Question 4: An evaluator tracking whether activities were completed on schedule is conducting which type of evaluation?
- Impact evaluation
- Process evaluation (Correct answer)
- Summative evaluation
- Cost-benefit analysis
Correct answer: Process evaluation
Process evaluation monitors program implementation fidelity, including whether activities were carried out as planned and on schedule.
Question 5: Which statistical concept indicates the minimum detectable effect size that a study can reliably identify?
- Statistical power (Correct answer)
- Confidence interval
- Standard deviation
- Regression coefficient
Correct answer: Statistical power
Statistical power is the probability that a study will detect an effect of a given size if it truly exists, informing sample size decisions.
Question 6: A funder requires grantees to report outputs, outcomes, and impact. Which of these represents an OUTPUT?
- Reduced recidivism rates among participants
- Number of training sessions delivered (Correct answer)
- Improved employment rates in the community
- Increased program participant self-efficacy
Correct answer: Number of training sessions delivered
Outputs are the direct, countable products of program activities, such as the number of training sessions held.
Question 7: When an evaluator controls for pre-existing differences between groups using statistical regression, this technique is called:
- Propensity score matching
- Analysis of covariance (ANCOVA) (Correct answer)
- Instrumental variable analysis
- Stratified sampling
Correct answer: Analysis of covariance (ANCOVA)
ANCOVA adjusts for pre-existing group differences on covariates, making post-test comparisons more accurate.
CGA Assessment & Evaluation Methods 2
Which evaluation design is most appropriate when random assignment to treatment and control groups is not feasible?