CPCE Research & Program Evaluation 2 — Questions and Answers
Question 1: A counselor wants to determine whether a new career counseling intervention caused a reduction in client anxiety. Which research design best establishes causality?
- Correlational study with a large sample
- Randomized controlled trial with a control group (Correct answer)
- Single-subject case study with repeated measures
- Cross-sectional survey administered post-intervention
Correct answer: Randomized controlled trial with a control group
A randomized controlled trial with random assignment to treatment and control groups is the gold standard for establishing causal relationships.
Question 2: In program evaluation, 'formative evaluation' is best described as:
- Assessing overall program outcomes after completion
- Ongoing assessment used to improve a program while it is running (Correct answer)
- Comparing two programs to determine which is more cost-effective
- A needs assessment conducted before program implementation
Correct answer: Ongoing assessment used to improve a program while it is running
Formative evaluation collects data during program implementation to guide ongoing improvements.
Question 3: Which of the following is an example of a Type II error in counseling research?
- Concluding an intervention is effective when it is not
- Failing to detect a real treatment effect that exists (Correct answer)
- Using an inappropriate statistical test for the data
- Measuring the wrong construct with a validated instrument
Correct answer: Failing to detect a real treatment effect that exists
A Type II error (false negative) occurs when a researcher fails to reject a null hypothesis that is actually false.
Question 4: A school counselor collects data on student grades, attendance, and disciplinary referrals before and after implementing a social-emotional learning program. This is an example of:
- Summative evaluation using archival data (Correct answer)
- Needs assessment using primary data sources
- Process evaluation focusing on program fidelity
- Cost-benefit analysis of counseling services
Correct answer: Summative evaluation using archival data
Using existing school records to assess outcomes after a program constitutes summative evaluation with archival (secondary) data.
Question 5: When a researcher reports a statistically significant result with p < .05, this means:
- There is a 95% probability the intervention will work for all clients
- The probability of obtaining results this extreme by chance alone is less than 5% (Correct answer)
- The effect size is large enough to be clinically meaningful
- The null hypothesis has been proven false with certainty
Correct answer: The probability of obtaining results this extreme by chance alone is less than 5%
A p-value below .05 indicates there is less than a 5% probability that the observed results are due to random chance.
Question 6: Which sampling method would be MOST appropriate for studying a rare population, such as counselors who have experienced vicarious trauma?
- Simple random sampling
- Stratified random sampling
- Snowball sampling (Correct answer)
- Systematic sampling
Correct answer: Snowball sampling
Snowball sampling uses referrals from identified participants to recruit others, making it ideal for hard-to-reach or rare populations.
Question 7: A counselor evaluating a substance abuse prevention program finds it reduced use among participants but the effect did not generalize to the broader school population. This illustrates a threat to:
- Internal validity
- Statistical conclusion validity
- External validity (generalizability) (Correct answer)
- Construct validity of the outcome measure
Correct answer: External validity (generalizability)
External validity refers to the degree to which findings can be generalized beyond the study sample to other populations or settings.
CPCE Research & Program Evaluation 2
A counselor wants to determine whether a new career counseling intervention caused a reduction in client anxiety.
Which research design best establishes causality?