CAP Quality Assurance & Compliance 3 — Questions and Answers
Question 1: An analytics vendor delivers a model trained on data the client suspects includes records obtained without consent. What should the CAP professional recommend?
- Deploy the model and address consent issues if regulators ask
- Accept the model since the vendor bears all legal responsibility
- Retrain the model on the same data with different parameters
- Verify data provenance and consent status before accepting the model (Correct answer)
Correct answer: Verify data provenance and consent status before accepting the model
Professionals must confirm data was lawfully and ethically sourced before using models built on it.
Question 2: What distinguishes model validation from model verification in an analytics quality framework?
- Validation checks the model solves the right business problem; verification checks it was built correctly to specification (Correct answer)
- Validation is done by machines; verification is done by humans
- Validation applies only to regression models; verification applies to classification
- They are identical terms used interchangeably in all standards
Correct answer: Validation checks the model solves the right business problem; verification checks it was built correctly to specification
Validation asks 'did we build the right model?' while verification asks 'did we build the model right?'
Question 3: A healthcare analytics project involves protected health information. Which safeguard is most directly required to maintain compliance during analysis?
- Publishing patient-level results to demonstrate transparency
- De-identifying or limiting access to patient data per applicable privacy regulations (Correct answer)
- Storing all data on the analyst's personal laptop for convenience
- Sharing raw records with the entire project team by default
Correct answer: De-identifying or limiting access to patient data per applicable privacy regulations
Health privacy regulations require de-identification or strict access controls when handling protected health information.
Question 4: A stakeholder pressures an analyst to remove outliers until the model supports a predetermined conclusion. What quality and ethics principle does complying violate?
- The preference for simple models over complex ones
- The rule that outliers must always be retained
- Objectivity and integrity of the analytical process (Correct answer)
- The requirement to use the largest possible dataset
Correct answer: Objectivity and integrity of the analytical process
Manipulating data to reach a predetermined conclusion violates analytical objectivity and professional integrity.
Question 5: Which element should a formal model risk assessment include before an analytics model is approved for production?
- The names of every employee in the analytics department
- A guarantee that the model will never make an incorrect prediction
- The marketing plan for announcing the model
- Potential impacts of model errors and the severity of wrong decisions (Correct answer)
Correct answer: Potential impacts of model errors and the severity of wrong decisions
Risk assessments evaluate the consequences and likelihood of model errors to determine appropriate controls.
Question 6: During a peer review of an analytics deliverable, the reviewer's primary responsibility is to:
- Independently assess the soundness of methods, assumptions, and conclusions (Correct answer)
- Approve the work quickly to avoid delaying the project
- Rewrite the analysis using their own preferred techniques
- Focus only on formatting and grammar of the report
Correct answer: Independently assess the soundness of methods, assumptions, and conclusions
Peer review provides an independent check on methodology, assumptions, and whether conclusions follow from the evidence.
Question 7: A company plans to transfer customer data from the EU to servers in another country for analysis. What compliance mechanism is most relevant?
- Using a stronger machine learning algorithm
- Ensuring an approved cross-border data transfer mechanism is in place under GDPR (Correct answer)
- Compressing the data so it transfers faster
- Renaming the data files to remove references to the EU
Correct answer: Ensuring an approved cross-border data transfer mechanism is in place under GDPR
GDPR restricts international transfers of personal data unless an approved mechanism such as adequacy or standard contractual clauses applies.
CAP Quality Assurance & Compliance 3
An analytics vendor delivers a model trained on data the client suspects includes records obtained without consent.
What should the CAP professional recommend?