Evaluation Research & Impact Assessment Flashcards

In interrupted time series analysis, the program's impact is identified by examining:

Answer: Changes in level and/or slope of outcome trends at the intervention point

Interrupted time series detects program effects by analyzing whether the outcome trend shifts at the point of intervention.

Which validity type is most threatened when an evaluator uses a test that was normed on a different population than the one being evaluated?

Answer: Construct validity

Using measures normed on a different population threatens construct validity because the instrument may not measure the intended construct for that group.

A program serving unhoused individuals shows high dropout before post-test. The evaluator should be most concerned about:

Answer: Attrition bias potentially distorting outcome estimates

High differential attrition means those who completed the program may differ systematically from dropouts, biasing effect estimates.

Contribution analysis is used in evaluation when:

Answer: Multiple interacting factors make attribution to a single cause impossible

Contribution analysis builds a plausible causal story when complexity prevents clean attribution to a single program.

When conducting a needs assessment, the difference between 'normative need' and 'felt need' is:

Answer: Normative need is defined by experts; felt need is perceived by the population itself

Normative need is defined by professional or expert standards, while felt need reflects what people say they need themselves.

An evaluator using 'triangulation' combines multiple data sources primarily to:

Answer: Increase confidence in findings by checking consistency across methods

Triangulation strengthens credibility by examining whether findings converge across different methods or data sources.

The American Evaluation Association's guiding principle of 'systematic inquiry' means evaluators must:

Answer: Conduct technically sound data collection and analysis appropriate to the evaluation question

Systematic inquiry requires technically adequate methods, not necessarily any one specific design.