CTRS Outcome Measurement & Program Evaluation 3 — Questions and Answers
Question 1: Which evaluation design provides the strongest evidence that a TR program caused observed client improvements?
- Descriptive program report
- Pre-test only design
- Randomized controlled trial (Correct answer)
- Post-test only design with anecdotal notes
Correct answer: Randomized controlled trial
A randomized controlled trial (RCT) with random assignment to treatment and control groups provides the highest level of causal evidence.
Question 2: In program evaluation, formative evaluation differs from summative evaluation in that formative evaluation:
- Occurs at program completion to judge overall effectiveness
- Is conducted during program implementation to improve ongoing delivery (Correct answer)
- Compares the program to national benchmarks
- Determines whether to continue or discontinue a program
Correct answer: Is conducted during program implementation to improve ongoing delivery
Formative evaluation gathers data during program delivery so adjustments can be made in real time, while summative evaluation judges final outcomes.
Question 3: A CTRS is selecting an outcome measure for clients with traumatic brain injury. She should prioritize an instrument that has been validated with:
- Healthy community adults
- Pediatric populations
- Individuals with TBI or neurological conditions (Correct answer)
- Geriatric nursing home residents
Correct answer: Individuals with TBI or neurological conditions
Validity of an instrument must be established within the specific population for which it is being used, in this case individuals with TBI.
Question 4: The concept of 'minimal clinically important difference' (MCID) in outcome measurement refers to:
- The largest possible score change on a given instrument
- The smallest change in score that clients perceive as meaningful (Correct answer)
- The standard error of measurement for a scale
- The average score of a normative comparison group
Correct answer: The smallest change in score that clients perceive as meaningful
MCID identifies the threshold of score change that represents a real, meaningful improvement to the client, not just statistical noise.
Question 5: A CTRS conducting a needs assessment for a new wellness program should first:
- Purchase intervention materials and equipment
- Design the evaluation plan and outcome measures
- Identify the target population's needs, resources, and gaps (Correct answer)
- Write program goals and select activities
Correct answer: Identify the target population's needs, resources, and gaps
A needs assessment begins by understanding the target population's current status, unmet needs, and available resources before designing any program.
Question 6: Content validity of an outcome measure is best established by:
- Correlating instrument scores with established gold-standard measures
- Having subject matter experts review items for relevance and comprehensiveness (Correct answer)
- Testing the instrument with a large normative sample
- Calculating internal consistency using Cronbach's alpha
Correct answer: Having subject matter experts review items for relevance and comprehensiveness
Content validity is established through systematic expert review to confirm that instrument items adequately represent the construct of interest.
Question 7: A CTRS reports that 82% of clients in her stress management program achieved their individualized leisure goals. This is an example of:
- Efficiency data
- Process evaluation data
- Outcome data (Correct answer)
- Input data
Correct answer: Outcome data
Goal achievement rates reflect changes in client status attributable to the program, which is the definition of outcome data.
CTRS Outcome Measurement & Program Evaluation 3
Which evaluation design provides the strongest evidence that a TR program caused observed client improvements?