CFD Assessment & Evaluation Methods 5 — Questions and Answers
Question 1: What is the PRIMARY risk of using only objective (selected-response) tests to assess complex professional competencies?
- They are too time-consuming to administer
- They may fail to capture application, judgment, and performance in authentic contexts (Correct answer)
- They are inherently unreliable due to guessing
- They cannot be aligned with learning objectives
Correct answer: They may fail to capture application, judgment, and performance in authentic contexts
Selected-response items efficiently test recall and comprehension but are poorly suited to assess complex competencies requiring authentic performance.
Question 2: In the context of program evaluation, 'logic models' are used to:
- Determine statistical significance of outcome data
- Visually map the connections between program inputs, activities, outputs, and intended outcomes (Correct answer)
- Replace rubrics for grading complex projects
- Identify which students are at risk of failing
Correct answer: Visually map the connections between program inputs, activities, outputs, and intended outcomes
Logic models graphically represent the causal chain from resources and activities to short- and long-term outcomes, clarifying program theory.
Question 3: Which approach to grading is MOST consistent with growth mindset principles?
- Averaging all scores, including early failed attempts
- Allowing reassessment and recording the most recent or highest score (Correct answer)
- Grading on a strict curve to maintain class rank
- Weighting final exams at 90% of the total grade
Correct answer: Allowing reassessment and recording the most recent or highest score
Allowing reassessment and using the most recent or highest score rewards improvement and signals that learning over time is what matters.
Question 4: A faculty developer recommends 'calibration sessions' before graders score student work. The main goal is to:
- Speed up the grading process by dividing labor equally
- Align graders' interpretations of rubric criteria to increase inter-rater reliability (Correct answer)
- Identify which students need academic support
- Satisfy accreditation documentation requirements
Correct answer: Align graders' interpretations of rubric criteria to increase inter-rater reliability
Calibration sessions have graders score anchor papers together and discuss differences, aligning interpretations and reducing subjective variability.
Question 5: Which scenario BEST illustrates construct-irrelevant variance reducing assessment validity?
- A chemistry exam includes questions on all major topics covered in class
- A writing assessment penalizes non-native speakers for grammar when measuring scientific reasoning (Correct answer)
- A performance task mirrors real-world job conditions accurately
- A rubric uses clear and observable behavioral descriptors
Correct answer: A writing assessment penalizes non-native speakers for grammar when measuring scientific reasoning
Construct-irrelevant variance occurs when assessment scores are influenced by factors unrelated to the target construct, such as language proficiency when measuring science skills.
Question 6: A CFD candidate is designing a workshop evaluation. Which combination of data sources provides the MOST complete picture of workshop effectiveness?
- Only end-of-session satisfaction surveys
- Pre/post knowledge tests combined with follow-up observation of classroom practice (Correct answer)
- Attendance records and facilitator self-ratings
- Peer nominations and social media reactions
Correct answer: Pre/post knowledge tests combined with follow-up observation of classroom practice
Combining knowledge tests (learning) with follow-up classroom observation (behavior transfer) addresses multiple Kirkpatrick levels and yields richer evidence.
Question 7: When aligning assessments with learning objectives, which question should a faculty developer ask FIRST?
- How many points should each assessment be worth?
- What evidence would demonstrate that students have achieved the objective? (Correct answer)
- How long should students spend completing the assessment?
- Which assessment format is easiest to grade?
Correct answer: What evidence would demonstrate that students have achieved the objective?
Backward design starts by defining what evidence of learning would look like before choosing assessment formats or grading schemes.
CFD Assessment & Evaluation Methods 5
What is the PRIMARY risk of using only objective (selected-response) tests to assess complex professional competencies?