CCBA Solution Evaluation and Validation 1 — Questions and Answers
Question 1: What is the PRIMARY goal of solution evaluation in business analysis?
- To identify which software vendor provides the best product
- To assess whether the implemented solution delivers the expected business value (Correct answer)
- To test all software functionality for defects
- To document technical specifications for maintenance teams
Correct answer: To assess whether the implemented solution delivers the expected business value
Solution evaluation assesses whether the delivered solution actually achieves the intended business outcomes and value — it goes beyond technical testing to examine business impact.
Solution evaluation in BABOK examines whether the solution has delivered the expected business value and outcomes. While technical testing verifies that the solution works as specified, evaluation assesses whether the specifications were right in the first place — did the solution actually improve the business situation? It involves comparing realized benefits against the business case and identifying any remaining gaps.
Question 2: What is 'user acceptance testing' (UAT) designed to verify?
- That the system performs within acceptable response time limits
- That the solution meets business requirements and is acceptable to users for production use (Correct answer)
- That the solution is free of all software defects
- That the solution complies with regulatory requirements
Correct answer: That the solution meets business requirements and is acceptable to users for production use
UAT verifies that the solution meets the agreed business requirements and that end users find it acceptable for production use — it is the business validation gate before go-live.
User acceptance testing is performed by actual business users testing the solution against real business scenarios and requirements. Its purpose is to confirm that the solution meets business needs, works in the actual operating environment, and is fit for purpose. UAT is typically the final approval step before production deployment and represents the business community's formal acceptance of the solution.
Question 3: Which of the following BEST describes 'solution limitations' as assessed in solution evaluation?
- Technical defects in the software code
- Constraints or gaps in the solution that prevent it from fully meeting business needs (Correct answer)
- Legal restrictions on the use of the technology
- Budget overruns on the implementation project
Correct answer: Constraints or gaps in the solution that prevent it from fully meeting business needs
Solution limitations are constraints or gaps that prevent the solution from fully satisfying business requirements — these may be acceptable trade-offs or may require remediation.
Solution limitations are characteristics of the delivered solution that constrain its ability to fully meet business requirements or deliver expected value. They may arise from technical constraints, budget limitations, time constraints, or evolving business needs. Documenting limitations helps stakeholders understand the residual gap between what was delivered and what was ideally needed, and informs decisions about future enhancements.
Question 4: What is the purpose of a 'benefit realization assessment' after solution deployment?
- To calculate developer bonuses for delivering the project on time
- To measure whether the solution has actually delivered the benefits projected in the business case (Correct answer)
- To identify opportunities for system upgrades
- To document lessons learned for the project archive
Correct answer: To measure whether the solution has actually delivered the benefits projected in the business case
A benefit realization assessment measures whether the projected benefits from the business case have been achieved after the solution has been in operation, enabling accountability and learning.
Benefit realization assessment compares the benefits actually achieved (measured from operational data) against the projections in the original business case. This accountability mechanism helps organizations learn whether their business cases are realistic, identify unrealized benefits that may require additional action, and build a track record for future investment decisions. It is typically conducted 3-12 months after go-live.
Question 5: In solution evaluation, what does 'validation' mean as distinct from 'verification'?
- Validation tests the code; verification tests the requirements
- Verification ensures the solution was built correctly; validation ensures the correct solution was built (Correct answer)
- Validation is done by testers; verification is done by business analysts
- There is no meaningful distinction between the two terms
Correct answer: Verification ensures the solution was built correctly; validation ensures the correct solution was built
Verification asks 'Did we build it right?' (conformance to specifications), while validation asks 'Did we build the right thing?' (fitness for business purpose).
Verification confirms that the solution conforms to its specifications — it was built according to the requirements. Validation confirms that the specifications were correct and that the built solution actually meets the stakeholders' real-world needs. A system can be verified (correctly built to spec) but fail validation (the specs didn't reflect actual needs). Both are essential to solution evaluation.
Question 6: Which of the following indicates that a solution is performing below expected value?
- Minor software bugs reported by users after deployment
- Significant variance between projected business benefits and actual measured outcomes (Correct answer)
- User interface feedback requesting aesthetic changes
- A deployment that was completed two weeks late
Correct answer: Significant variance between projected business benefits and actual measured outcomes
A significant gap between projected benefits (from the business case) and actually measured outcomes indicates that the solution is underperforming in terms of business value delivery.
Solution performance below expected value is primarily indicated by business outcome metrics falling short of the projections in the business case — for example, expected cost savings not materializing, customer satisfaction not improving as projected, or process efficiency gains not being realized. This triggers further analysis to determine whether the issue lies with the solution itself, its implementation, adoption, or the original benefit estimates.
CCBA Solution Evaluation and Validation 1
What is the PRIMARY goal of solution evaluation in business analysis?