Data Collection & Analysis Flashcards

A program evaluator needs to assess behavioral changes in youth after an intervention. Which data source provides the most direct evidence?

Answer: Direct observation of participant behavior

Direct observation provides firsthand evidence of actual behavioral changes rather than self-reported or proxy data.

What does 'unit of analysis' refer to in a program evaluation?

Answer: The entity (individual, group, organization) about which data are collected

The unit of analysis is the entity whose characteristics are being examined, such as individuals, groups, or programs.

An evaluator notices that participants score higher on a measure simply because they've taken the same test before. This threat to validity is called:

Answer: Testing effect

The testing effect occurs when prior exposure to a measure artificially inflates subsequent scores.

Which sampling strategy ensures every member of the population has an equal chance of being selected?

Answer: Simple random sampling

Simple random sampling gives every population member an equal probability of selection, reducing selection bias.

In content analysis of qualitative data, 'coding' refers to:

Answer: Assigning labels to segments of data to identify themes

Coding in qualitative analysis involves systematically labeling data segments to identify patterns and themes.

A CPE evaluator discovers that data collectors recorded information inconsistently across sites. This is an example of which threat to data quality?

Answer: Instrumentation

Instrumentation threat occurs when changes in measurement procedures or observers affect data consistency and validity.

Which approach to data analysis is most appropriate when the evaluator wants to generate hypotheses rather than test them?

Answer: Exploratory data analysis

Exploratory data analysis examines data patterns without prespecified hypotheses, making it ideal for hypothesis generation.