CCC Research & Evaluation in Career Counseling 2 — Questions and Answers
Question 1: A career counselor evaluates a college career center's services by measuring student employment rates before and after program participation. This is an example of:
- Action research
- A pre-post program evaluation design (Correct answer)
- A randomized controlled trial
- Phenomenological inquiry
Correct answer: A pre-post program evaluation design
Measuring outcomes before and after an intervention describes a pre-post program evaluation design, commonly used in career services accountability.
Question 2: Which statistical concept indicates the probability that a study's result occurred by chance alone?
- Effect size
- Standard deviation
- p-value (Correct answer)
- Confidence interval
Correct answer: p-value
The p-value indicates the probability that the observed result is due to chance; typically, p < .05 is considered statistically significant.
Question 3: Construct validity of a career interest inventory is best demonstrated by:
- High consistency of scores over repeated administrations
- The instrument's ability to measure the theoretical construct it purports to assess (Correct answer)
- Agreement between the test developer and a subject matter expert on item wording
- Low average scores across a heterogeneous sample
Correct answer: The instrument's ability to measure the theoretical construct it purports to assess
Construct validity confirms that an instrument actually measures the theoretical construct (e.g., career interest) it is designed to assess.
Question 4: In a career counseling outcome study, an effect size of d = 0.80 is generally interpreted as:
- A small, practically insignificant effect
- A medium effect
- A large, practically meaningful effect (Correct answer)
- An invalid result requiring replication
Correct answer: A large, practically meaningful effect
Cohen's d = 0.80 is conventionally classified as a large effect size, indicating a strong practical difference between groups.
Question 5: Which program evaluation model emphasizes stakeholder input and the practical utility of evaluation findings for decision-making?
- Goal-free evaluation
- The CIPP model (Context, Input, Process, Product)
- Utilization-focused evaluation (Correct answer)
- Experimental control group design
Correct answer: Utilization-focused evaluation
Utilization-focused evaluation, developed by Patton, centers on ensuring evaluation findings are actively used by stakeholders for decisions.
Question 6: When a career program's evaluation data show statistically significant improvement but a very small effect size, the counselor should conclude:
- The program is highly effective and should be expanded immediately
- The improvement is real but may have limited practical importance (Correct answer)
- The data are unreliable and the study should be repeated
- The assessment instruments used were invalid
Correct answer: The improvement is real but may have limited practical importance
Statistical significance indicates a real effect exists, but a small effect size means the practical magnitude of that improvement may be minimal.
Question 7: Accountability in career services programs most directly involves:
- Ensuring all staff hold licensed professional counselor credentials
- Documenting and demonstrating program outcomes and the value of services provided (Correct answer)
- Limiting client caseloads to maintain counselor quality
- Using only fee-based career assessments to ensure rigor
Correct answer: Documenting and demonstrating program outcomes and the value of services provided
Accountability requires systematically documenting program outcomes and communicating the value and impact of career services to stakeholders.
CCC Research & Evaluation in Career Counseling 2
A career counselor evaluates a college career center's services by measuring student employment rates before and after program participation.
This is an example of: