CPM Outcome Evaluation & Surveys 3 — Questions and Answers
Question 1: A CPM wants to measure the long-term impact of mentoring on a mentee's career advancement. Which evaluation timeframe is most appropriate?
- Immediate post-session feedback
- 30-day follow-up survey
- 6- to 12-month longitudinal follow-up (Correct answer)
- Weekly check-in logs during the program
Correct answer: 6- to 12-month longitudinal follow-up
Career advancement outcomes emerge over months to years, so a longitudinal follow-up after program completion is needed to capture lasting impact.
Question 2: Which element of a well-designed survey question helps prevent acquiescence bias?
- Using leading language to guide answers
- Including reverse-worded items (Correct answer)
- Keeping all questions positively framed
- Limiting questions to yes/no format
Correct answer: Including reverse-worded items
Reverse-worded items detect respondents who agree with everything regardless of content, helping to identify and correct for acquiescence bias.
Question 3: When evaluating mentoring outcomes for a diverse cohort, why is disaggregating data by demographics important?
- To reduce the total sample size needed
- To identify differential program effectiveness across subgroups (Correct answer)
- To simplify data reporting for stakeholders
- To comply with anonymization requirements
Correct answer: To identify differential program effectiveness across subgroups
Disaggregating data reveals whether the program benefits all participants equally or whether certain groups experience different outcomes, supporting equity analysis.
Question 4: A program director notices survey results vary significantly depending on whether surveys are administered online or on paper. This illustrates:
- Sampling error
- Mode effect (Correct answer)
- Ceiling effect
- Regression to the mean
Correct answer: Mode effect
Mode effect occurs when different survey delivery methods (online vs. paper) produce systematically different responses, threatening data validity.
Question 5: Which outcome best represents a Level 4 (Results) finding in a mentoring program evaluation?
- Mentees rated sessions as highly satisfying
- Mentees reported increased self-confidence post-program
- Mentees applied new negotiation techniques at work
- The organization saw a 15% reduction in voluntary turnover after the program (Correct answer)
Correct answer: The organization saw a 15% reduction in voluntary turnover after the program
Level 4 Results focuses on organizational or systemic outcomes such as retention rates, productivity, or revenue attributable to the mentoring program.
Question 6: Triangulation in outcome evaluation refers to:
- Using three Likert-scale anchors instead of five
- Gathering data from multiple sources or methods to increase credibility (Correct answer)
- Interviewing three mentors per mentee
- Calculating a composite score from three sub-scales
Correct answer: Gathering data from multiple sources or methods to increase credibility
Triangulation strengthens evaluation validity by cross-checking findings from multiple data sources, methods, or evaluators.
Question 7: A mentor notices that mentees consistently give maximum satisfaction scores regardless of session quality. This is known as:
- Social desirability bias
- Ceiling effect (Correct answer)
- Halo effect
- Recency bias
Correct answer: Ceiling effect
A ceiling effect occurs when scores cluster at the top of the scale, limiting the ability to detect meaningful differences in satisfaction levels.
CPM Outcome Evaluation & Surveys 3
A CPM wants to measure the long-term impact of mentoring on a mentee's career advancement.
Which evaluation timeframe is most appropriate?