Assessment & Evaluation Techniques Flashcards

A consultant wants to measure inter-rater reliability when two assessors independently score the same competency interviews. Which statistic is most appropriate?

Answer: Cohen's kappa coefficient

Cohen's kappa measures agreement between two raters beyond what would be expected by chance alone, making it ideal for categorical scoring.

During a process evaluation, a consultant identifies that a documented process is rarely followed by staff. This gap between prescribed and actual practice is best described as which type of gap?

Answer: Compliance gap

A compliance gap exists when employees deviate from documented standards or procedures, regardless of whether they have the capability to follow them.

A senior consultant designs a pre/post assessment to measure knowledge gained from a consulting intervention. Which threat to internal validity is most controlled by using a control group?

Answer: History effects (external events)

A control group that experiences the same time period but not the intervention isolates whether observed changes are due to the intervention or concurrent external events.

Which assessment framework explicitly links strategic objectives to operational metrics through a cascade of financial, customer, internal process, and learning perspectives?

Answer: Balanced Scorecard

The Balanced Scorecard by Kaplan and Norton translates strategy into four interdependent measurement perspectives to provide a holistic performance view.

A client engages a senior consultant to evaluate whether a new sales process increased revenue. The consultant finds a strong correlation between process adoption and revenue growth. What critical analytical step must follow before concluding causation?

Answer: Rule out confounding variables that could explain the relationship

Correlation does not establish causation; confounding variables (e.g., market conditions, seasonality) must be identified and controlled before attributing revenue growth to the process.

In a 360-degree feedback assessment, which characteristic most distinguishes it from traditional top-down performance reviews?

Answer: It collects performance input from peers, subordinates, and self in addition to managers

360-degree feedback gathers multi-source input to provide a comprehensive view of an individual's behaviors and competencies from all relevant perspectives.

A senior consultant recommends using a Delphi method for a complex organizational assessment. Which scenario makes this technique most appropriate?

Answer: When expert consensus is needed on uncertain topics without face-to-face group dynamics influencing responses

The Delphi method uses iterative anonymous expert surveys to build consensus on ambiguous topics while minimizing groupthink and dominant-personality bias.