CAIC Cybersecurity & Risk 3 — Questions and Answers
Question 1: What does 'model inversion attack' allow an adversary to do?
- Flip the model's predictions to the opposite class
- Reconstruct approximate training data from model outputs (Correct answer)
- Inject backdoors during fine-tuning
- Replace a production model with a malicious replica
Correct answer: Reconstruct approximate training data from model outputs
Model inversion attacks exploit query access to infer or reconstruct sensitive training data from model responses.
Question 2: Which of the following is a key principle of 'privacy by design' as applied to AI systems?
- Collecting maximum data to improve model accuracy
- Embedding privacy controls proactively into system architecture from the start (Correct answer)
- Disclosing data usage only after a breach occurs
- Relying solely on post-deployment audits for compliance
Correct answer: Embedding privacy controls proactively into system architecture from the start
Privacy by design requires integrating data protection mechanisms during the design phase, not as an afterthought.
Question 3: A financial institution's AI fraud detection model begins performing poorly six months after deployment due to changing fraud patterns. This is an example of:
- Adversarial attack
- Model drift (Correct answer)
- Underfitting
- Supply chain compromise
Correct answer: Model drift
Model drift occurs when real-world data distributions shift from the training distribution, degrading performance over time.
Question 4: Which control specifically addresses the risk that a third-party pre-trained model contains a hidden backdoor triggered by a specific input pattern?
- Network segmentation
- Model provenance verification and red-team testing (Correct answer)
- Role-based access control
- TLS encryption of API endpoints
Correct answer: Model provenance verification and red-team testing
Verifying model provenance and conducting adversarial red-team testing help detect backdoors embedded in third-party models.
Question 5: In the context of AI risk, 'explainability' is important for cybersecurity because:
- It makes models run faster during inference
- It allows security teams to understand and audit why a model flagged or missed a threat (Correct answer)
- It reduces the size of model weights for easier storage
- It eliminates the need for human review of AI decisions
Correct answer: It allows security teams to understand and audit why a model flagged or missed a threat
Explainability enables security analysts to audit AI-driven decisions, identify errors, and detect manipulation or bias.
Question 6: Which approach best protects an AI model's intellectual property and parameters from extraction via repeated querying?
- Disabling all logging
- Rate limiting queries and adding output perturbation (Correct answer)
- Using open-source model weights
- Storing the model in a public cloud bucket
Correct answer: Rate limiting queries and adding output perturbation
Rate limiting restricts query volume while output perturbation makes it harder to reconstruct model behavior through API queries.
Question 7: What is the main cybersecurity concern when AI systems are integrated with legacy enterprise software?
- The AI model becomes too accurate
- Legacy systems may introduce unpatched vulnerabilities that attackers can exploit via the AI integration layer (Correct answer)
- AI components slow down legacy database queries
- AI models cannot process structured data from legacy systems
Correct answer: Legacy systems may introduce unpatched vulnerabilities that attackers can exploit via the AI integration layer
Legacy systems often carry unpatched vulnerabilities that attackers can leverage through the new AI integration layer as an attack path.
CAIC Cybersecurity & Risk 3
What does 'model inversion attack' allow an adversary to do?