CPE Data Collection & Analysis 2 — Questions and Answers
Question 1: A program evaluator wants to capture participant experiences in depth without constraining responses. Which data collection method is most appropriate?
- Structured survey
- Open-ended interview (Correct answer)
- Checklist observation
- Standardized test
Correct answer: Open-ended interview
Open-ended interviews allow participants to express experiences in their own words without predefined constraints.
Question 2: When analyzing Likert-scale data from a program evaluation survey, which statistical measure is most appropriate for the central tendency?
- Mean only
- Mode or median (Correct answer)
- Standard deviation
- Correlation coefficient
Correct answer: Mode or median
Likert-scale data is ordinal, so mode or median is more appropriate than mean for central tendency.
Question 3: A CPE evaluator conducts focus groups but is concerned about dominant participants skewing results. What technique best mitigates this?
- Increasing group size
- Using a skilled facilitator with structured turn-taking (Correct answer)
- Recording sessions only
- Eliminating follow-up probes
Correct answer: Using a skilled facilitator with structured turn-taking
A skilled facilitator using structured turn-taking ensures all voices are heard and prevents dominance bias.
Question 4: In program evaluation, 'triangulation' of data primarily serves to:
- Reduce the cost of data collection
- Increase validity by using multiple data sources (Correct answer)
- Simplify the analysis process
- Limit the number of stakeholders involved
Correct answer: Increase validity by using multiple data sources
Triangulation uses multiple sources, methods, or analysts to cross-verify findings and strengthen validity.
Question 5: A program shows statistically significant improvement in test scores. What additional consideration is critical before concluding the program caused the improvement?
- Sample size calculation
- Ruling out alternative explanations through design (Correct answer)
- Publishing the results immediately
- Switching to qualitative methods
Correct answer: Ruling out alternative explanations through design
Statistical significance alone does not establish causation; rival explanations must be ruled out through evaluation design.
Question 6: Which type of validity refers to whether a data collection instrument measures what it is intended to measure?
- External validity
- Predictive validity
- Construct validity (Correct answer)
- Ecological validity
Correct answer: Construct validity
Construct validity assesses whether an instrument actually measures the theoretical construct it is designed to measure.
Question 7: An evaluator uses existing program records rather than collecting new data. This approach is called:
- Primary data collection
- Secondary data analysis (Correct answer)
- Participatory research
- Naturalistic inquiry
Correct answer: Secondary data analysis
Secondary data analysis involves examining pre-existing data collected for another purpose rather than gathering new data.
CPE Data Collection & Analysis 2
A program evaluator wants to capture participant experiences in depth without constraining responses.
Which data collection method is most appropriate?