CPE Data Collection & Analysis 5 — Questions and Answers
Question 1: An evaluator uses a comparison group that was not randomly assigned. This design is classified as:
- Experimental
- Quasi-experimental (Correct answer)
- Descriptive
- Meta-analytic
Correct answer: Quasi-experimental
Quasi-experimental designs use non-randomly assigned comparison groups, providing causal evidence weaker than true experiments.
Question 2: Missing data in a program evaluation dataset can best be handled by:
- Always deleting cases with missing values
- Documenting the extent, patterns, and applying appropriate imputation methods (Correct answer)
- Reporting only complete cases without disclosure
- Replacing all missing values with the maximum score
Correct answer: Documenting the extent, patterns, and applying appropriate imputation methods
Proper handling of missing data requires understanding its pattern and using appropriate methods like imputation while maintaining transparency.
Question 3: A program evaluator is reviewing data and finds several extreme outliers. The best first step is to:
- Delete all outliers immediately
- Investigate whether outliers are data errors or legitimate extreme cases (Correct answer)
- Ignore them and proceed with analysis
- Replace them with the mean value
Correct answer: Investigate whether outliers are data errors or legitimate extreme cases
Outliers should be investigated first to determine if they represent data entry errors or valid extreme observations before deciding how to handle them.
Question 4: Which data collection approach allows participants to respond at their own pace and provides anonymity, making it suitable for sensitive topics?
- In-person structured interview
- Telephone focus group
- Self-administered anonymous survey (Correct answer)
- Direct behavioral observation
Correct answer: Self-administered anonymous survey
Self-administered anonymous surveys reduce social desirability bias and allow honest responses on sensitive topics at the respondent's own pace.
Question 5: In logic model-based evaluation, which data type is most appropriate for measuring 'inputs'?
- Participant satisfaction ratings
- Resources such as staff hours, funding, and materials (Correct answer)
- Long-term community outcomes
- Behavioral change indicators
Correct answer: Resources such as staff hours, funding, and materials
Inputs in a logic model refer to the resources invested in a program, including staff, funding, equipment, and materials.
Question 6: A program evaluation report shows statistically significant results but the effect size is very small. What does this mean for practical use?
- The findings are definitely important for practice
- Statistical significance guarantees practical importance
- The program change may not be meaningful in real-world terms (Correct answer)
- Effect size is irrelevant to evaluation decisions
Correct answer: The program change may not be meaningful in real-world terms
Small effect sizes suggest minimal practical impact even when results are statistically significant due to large sample sizes.
Question 7: Which technique involves systematically reviewing existing research findings to summarize what is known about a program's effectiveness?
- Needs assessment
- Formative evaluation
- Systematic literature review (Correct answer)
- Process monitoring
Correct answer: Systematic literature review
A systematic literature review uses rigorous methods to identify, appraise, and synthesize existing research on a topic or intervention.
CPE Data Collection & Analysis 5
An evaluator uses a comparison group that was not randomly assigned.
This design is classified as: