AAS Program Evaluation & Outcomes 3 — Questions and Answers
Question 1: An evaluator uses the Columbia Suicide Severity Rating Scale (C-SSRS) as an outcome measure pre- and post-intervention. Which psychometric property is MOST critical to verify before using it in this way?
- Face validity
- Test-retest reliability and sensitivity to change (Correct answer)
- Content validity only
- Inter-rater reliability among researchers
Correct answer: Test-retest reliability and sensitivity to change
To detect change over time, an instrument must be reliable and sensitive to genuine change (responsiveness), not just stable.
Question 2: Cost-effectiveness analysis in suicide prevention compares program costs against:
- The number of staff hours invested
- Outcomes achieved per unit of cost (Correct answer)
- Total grant funding received
- The cost of similar programs nationally
Correct answer: Outcomes achieved per unit of cost
Cost-effectiveness analysis expresses program costs relative to a defined unit of outcome, such as cost per suicide attempt prevented.
Question 3: Which evaluation approach is MOST aligned with community-based participatory research principles in suicide prevention?
- Engaging community members as co-investigators in study design and interpretation (Correct answer)
- Hiring external expert evaluators to maintain objectivity
- Using standardized national instruments without adaptation
- Conducting evaluation only after program completion
Correct answer: Engaging community members as co-investigators in study design and interpretation
CBPR involves community members as equal partners throughout the evaluation process, increasing cultural relevance and buy-in.
Question 4: Dosage in program evaluation refers to:
- The severity of suicidal ideation measured at intake
- The amount of the intervention participants actually receive (Correct answer)
- The number of staff delivering the program
- The frequency of supervisor oversight sessions
Correct answer: The amount of the intervention participants actually receive
Dosage measures how much of the intended intervention each participant received, which affects interpretation of outcomes.
Question 5: A zero-suicide initiative hospital reports a 40% reduction in inpatient suicides. The most significant threat to the internal validity of this finding is:
- Regression to the mean, especially if the initiative followed an unusually high-rate period (Correct answer)
- Insufficient sample size for statistical testing
- Lack of fidelity monitoring for clinical staff
- Inadequate training duration for the initiative
Correct answer: Regression to the mean, especially if the initiative followed an unusually high-rate period
If the program was implemented after an abnormally high-rate period, regression to the mean—not the intervention—may explain the decrease.
Question 6: Summative evaluation differs from formative evaluation in that summative evaluation is primarily used to:
- Improve program delivery while it is ongoing
- Judge overall program worth or effectiveness after implementation (Correct answer)
- Train new program facilitators
- Identify barriers to participant enrollment
Correct answer: Judge overall program worth or effectiveness after implementation
Summative evaluation assesses final outcomes and program merit, while formative evaluation supports ongoing program improvement.
Question 7: When a suicide prevention program serves multiple cultural communities, which evaluation adaptation is MOST important?
- Translating all materials using machine translation tools
- Using culturally validated instruments and community-specific norms (Correct answer)
- Applying national average benchmarks uniformly across all groups
- Limiting data collection to avoid burdening minority participants
Correct answer: Using culturally validated instruments and community-specific norms
Instruments must be validated for each cultural group and interpreted against culturally relevant norms to avoid biased conclusions.
AAS Program Evaluation & Outcomes 3
An evaluator uses the Columbia Suicide Severity Rating Scale (C-SSRS) as an outcome measure pre- and post-intervention.
Which psychometric property is MOST critical to verify before using it in this way?