Government Procurement Certification Program Evaluation 2 — Questions and Answers
Question 1: Which evaluation type is conducted DURING program execution to allow real-time improvements?
- Summative evaluation
- Formative evaluation (Correct answer)
- Impact evaluation
- Meta-evaluation
Correct answer: Formative evaluation
Formative evaluation occurs during program implementation to provide feedback that can improve the program while it is still running.
Question 2: In procurement program evaluation, what does a 'logic model' primarily depict?
- The financial audit trail of a contract
- The causal chain from inputs to outcomes (Correct answer)
- The organizational chart of the contracting office
- The schedule of procurement milestones
Correct answer: The causal chain from inputs to outcomes
A logic model visually maps the assumed causal relationships between program inputs, activities, outputs, and short- and long-term outcomes.
Question 3: A contracting officer wants to determine whether a vendor training program caused improved contract compliance rates. Which method best establishes causation?
- Pre-post comparison without a control group
- Randomized controlled trial with a comparison group (Correct answer)
- Stakeholder satisfaction survey
- Cost accounting review
Correct answer: Randomized controlled trial with a comparison group
A randomized controlled trial with a comparison group controls for confounding variables and provides the strongest evidence of causation.
Question 4: Which of the following best describes 'outcome evaluation' in a government procurement context?
- Reviewing whether procurement activities followed regulations
- Assessing whether the program achieved its intended results (Correct answer)
- Auditing vendor invoices for accuracy
- Tracking the number of contracts awarded per quarter
Correct answer: Assessing whether the program achieved its intended results
Outcome evaluation measures the extent to which a program achieved its intended goals and objectives, focusing on results rather than processes.
Question 5: What is the primary purpose of an evaluation stakeholder analysis?
- To assign blame for program failures
- To identify who has an interest in evaluation findings and how to engage them (Correct answer)
- To determine the size of the evaluation budget
- To select the statistical methods for data analysis
Correct answer: To identify who has an interest in evaluation findings and how to engage them
Stakeholder analysis identifies individuals and groups who have a stake in the program and its evaluation, guiding appropriate engagement strategies.
Question 6: When evaluating a multi-year procurement program, what distinguishes 'intermediate outcomes' from 'long-term outcomes'?
- Intermediate outcomes are financial; long-term outcomes are operational
- Intermediate outcomes occur within 1-3 years; long-term outcomes require 5+ years to materialize (Correct answer)
- Intermediate outcomes are measured by auditors; long-term outcomes by program managers
- There is no meaningful distinction between them
Correct answer: Intermediate outcomes occur within 1-3 years; long-term outcomes require 5+ years to materialize
Intermediate outcomes are shorter-term changes expected within a few years, while long-term outcomes represent the ultimate goals that take longer to achieve.
Question 7: Which data collection method is MOST appropriate when an evaluator needs to understand the context and nuances behind procurement outcomes?
- Large-scale quantitative survey
- In-depth qualitative interviews (Correct answer)
- Automated transaction log analysis
- Random sampling of contract files
Correct answer: In-depth qualitative interviews
In-depth qualitative interviews allow evaluators to explore context, perceptions, and reasons behind outcomes that quantitative data cannot fully capture.
Government Procurement Certification Program Evaluation 2
Which evaluation type is conducted DURING program execution to allow real-time improvements?