SPP Performance Measurement & Evaluation 5 — Questions and Answers
Question 1: Which evaluation design provides the strongest evidence of causation when assessing a strategic initiative's impact?
- Pre-test/post-test design with no comparison group
- Randomized controlled trial with a true control group (Correct answer)
- Cross-sectional survey at a single point in time
- Expert panel consensus review
Correct answer: Randomized controlled trial with a true control group
Randomized controlled trials provide the strongest causal evidence by randomly assigning participants to treatment and control groups, eliminating selection bias.
Question 2: A strategic planner identifies that a division's KPI scores improved dramatically after the measurement methodology was changed mid-year. This situation MOST likely represents:
- Genuine strategic progress driven by improved execution
- A measurement artifact rather than real performance improvement (Correct answer)
- Evidence that the new methodology is superior
- Confirmation that the original KPI was invalid
Correct answer: A measurement artifact rather than real performance improvement
Mid-year methodology changes break time-series comparability and can produce apparent performance improvements that reflect measurement differences rather than real organizational gains.
Question 3: In performance management, 'management by exception' means:
- Evaluating all KPIs equally regardless of variance size
- Focusing leadership attention on metrics that deviate significantly from targets (Correct answer)
- Excluding financial metrics from the performance review process
- Delegating all performance reporting to frontline managers
Correct answer: Focusing leadership attention on metrics that deviate significantly from targets
Management by exception directs limited leadership bandwidth toward metrics that show significant variance from targets, preventing information overload from reviewing all indicators equally.
Question 4: An organization's strategic performance report shows high scores on 'customer acquisition' but declining scores on 'customer retention.' What strategic insight does this pattern suggest?
- The sales team is outperforming the marketing team
- The value proposition may be failing to meet post-purchase expectations (Correct answer)
- Customer satisfaction surveys are unreliable and should be replaced
- The organization should immediately reduce its customer acquisition budget
Correct answer: The value proposition may be failing to meet post-purchase expectations
High acquisition paired with low retention suggests the value proposition, product quality, or customer experience is not delivering on promises made during the sales process.
Question 5: The SMART criteria for setting performance targets require that targets be Specific, Measurable, Achievable, Relevant, and:
- Repeatable
- Resourced
- Time-bound (Correct answer)
- Ranked
Correct answer: Time-bound
The 'T' in SMART stands for Time-bound, meaning every target must have a defined deadline or time horizon to create accountability and enable meaningful progress tracking.
Question 6: Which stakeholder group is MOST important to engage when selecting KPIs that will be used to evaluate a public sector strategic plan?
- Bond rating agencies evaluating fiscal health
- Citizens and service recipients who experience program outcomes (Correct answer)
- Peer municipalities using similar measurement frameworks
- Consulting firms with performance management expertise
Correct answer: Citizens and service recipients who experience program outcomes
Public sector strategic plans ultimately exist to serve citizens; engaging service recipients ensures KPIs measure what matters to those experiencing program outcomes, not just what is administratively convenient.
Question 7: A 'theory of change' document in strategic planning primarily serves to:
- Satisfy grant reporting requirements by documenting expenditures
- Make explicit the causal assumptions linking activities to long-term outcomes (Correct answer)
- Replace the strategic plan when organizational conditions change
- Provide a legal framework for program accountability
Correct answer: Make explicit the causal assumptions linking activities to long-term outcomes
A theory of change articulates and tests the causal logic connecting an organization's activities and outputs to intended intermediate and long-term outcomes, making assumptions visible and testable.
SPP Performance Measurement & Evaluation 5
Which evaluation design provides the strongest evidence of causation when assessing a strategic initiative's impact?