Mixed Deck — All CPE Topics Flashcards

A CPE evaluator discovers that data collectors recorded information inconsistently across sites. This is an example of which threat to data quality?

Answer: Instrumentation

Instrumentation threat occurs when changes in measurement procedures or observers affect data consistency and validity.

Which reporting format is MOST appropriate when the primary audience for evaluation findings is frontline program staff who need actionable guidance?

Answer: Brief executive summary with specific recommendations

Frontline staff need concise, actionable information, making a brief summary with clear recommendations the most effective reporting format for this audience.

Why set measurable objectives?

Answer: Track progress and effectiveness

Setting measurable objectives allows program managers to quantify the desired changes or achievements expected from the program. These objectives provide clear benchmarks against which progress can be tracked, enabling effective monitoring, evaluation of effectiveness, and accountability for results.

A funder requires an evaluator to guarantee positive findings before contracting. The evaluator should:

Answer: Decline and explain this violates evaluation ethics

Guaranteeing findings in advance violates the AEA Guiding Principles for Evaluators, specifically integrity, honesty, and independence.

In which situation is a 'communication plan' MOST critical to develop at the start of a program evaluation?

Answer: When multiple stakeholder groups have different information needs, languages, or literacy levels

A communication plan is most critical when diverse stakeholder groups require tailored messaging, formats, and timelines to ensure all parties receive usable information.

Why is stakeholder involvement important?

Answer: Ensures relevance and support

Stakeholder involvement is crucial because it brings diverse perspectives, expertise, and resources to the planning process. Engaging stakeholders ensures the program is relevant to their needs, gains their support and buy-in, and increases the likelihood of successful implementation and sustainability.

How does cultural awareness improve communication?

Answer: Respect diversity and clarity

Cultural awareness in communication involves understanding and respecting the diverse values, beliefs, and communication styles of different cultural groups. This sensitivity helps to tailor messages appropriately, avoid misunderstandings, and build rapport across cultural boundaries. By acknowledging and adapting to cultural differences, communication becomes clearer, more inclusive, and more effective for all parties involved.

Which of the following is an example of 'dissemination' as distinct from 'communication' in program evaluation?

Answer: Publishing an evaluation brief for broad public access on an organization's website

Dissemination refers to proactively spreading findings to broad audiences through accessible products like briefs, toolkits, or publications.

Why should reports be tailored to audiences?

Answer: Effective communication

Tailoring reports to specific audiences involves adjusting the language, level of detail, and format to suit their needs and understanding. This ensures that the message is received clearly and effectively, preventing confusion or disengagement. Effective communication maximizes the impact and utility of the evaluation findings for each stakeholder group, leading to better informed decisions.

Which of the following best describes 'stretch targets' in performance measurement?

Answer: Ambitious goals set beyond expected performance to encourage innovation and improvement

Stretch targets are ambitious performance goals set beyond what is routinely expected, intended to motivate programs to reach higher levels of performance.

In a utilization-focused evaluation, the primary driver of evaluation design decisions is:

Answer: Intended use by intended users

Utilization-focused evaluation, developed by Patton, centers all design decisions on how primary intended users will actually use the findings.

Which of the following BEST describes the role of a stakeholder advisory group in program evaluation?

Answer: To provide contextual input, review findings, and improve evaluation relevance

Advisory groups provide contextual expertise and stakeholder perspectives that strengthen evaluation relevance without overriding evaluator independence.

When reporting evaluation findings to a community of color that has historically been exploited by researchers, an evaluator should FIRST:

Answer: Engage community members in reviewing and interpreting the findings

Culturally responsive evaluation requires community engagement in interpreting findings to ensure accuracy and build trust with historically marginalized groups.

Which tool is MOST useful for identifying gaps between a program's Theory of Change and its actual implementation?

Answer: An implementation fidelity assessment

An implementation fidelity assessment examines whether the program is being delivered as designed, revealing gaps between theory and practice.

A program evaluation report shows statistically significant results but the effect size is very small. What does this mean for practical use?

Answer: The program change may not be meaningful in real-world terms

Small effect sizes suggest minimal practical impact even when results are statistically significant due to large sample sizes.

In performance measurement, what does 'data disaggregation' refer to?

Answer: Breaking down aggregate data by subgroups such as race, gender, or age to examine disparities

Data disaggregation involves breaking down overall program data into subgroups to identify whether outcomes differ across demographic or other characteristics, revealing potential disparities.

What does 'unit of analysis' refer to in a program evaluation?

Answer: The entity (individual, group, organization) about which data are collected

The unit of analysis is the entity whose characteristics are being examined, such as individuals, groups, or programs.

Which technique BEST supports ongoing stakeholder communication throughout a multi-year program evaluation?

Answer: Providing regular progress briefs, interim findings, and feedback loops

Regular progress briefs and feedback loops keep stakeholders informed and allow course corrections before the evaluation concludes.

What distinguishes an outcome indicator from an output indicator?

Answer: Outcome indicators measure changes in knowledge, behavior, or conditions, while output indicators measure direct products of activities

Outcome indicators measure changes in knowledge, skills, behaviors, or conditions resulting from a program, while output indicators measure the direct products of program activities.

How does sustainability planning benefit programs?

Answer: Ensure long-term impact

Sustainability planning focuses on developing strategies to ensure a program's continued operation and impact beyond its initial funding or implementation phase. By considering long-term viability, it helps secure ongoing resources, build capacity, and embed the program within communities or organizations, ensuring lasting benefits.