DSE Ethical Considerations in AI 4 — Questions and Answers
Question 1: Which of the following is an example of 'automation bias' in an AI-assisted medical diagnostic system?
- The model flags incorrect lab values due to data entry errors
- Clinicians over-rely on the AI recommendation and skip their own assessment (Correct answer)
- The system automates billing codes incorrectly
- The model learns from unbalanced class distributions
Correct answer: Clinicians over-rely on the AI recommendation and skip their own assessment
Automation bias is the tendency for humans to defer to automated systems even when their own judgment would yield a better result.
Question 2: The principle of 'data minimization' in AI ethics states that:
- Models should use as few parameters as possible
- Only data strictly necessary for the stated purpose should be collected (Correct answer)
- Datasets must be compressed before storage
- Feature counts should not exceed 100 per model
Correct answer: Only data strictly necessary for the stated purpose should be collected
Data minimization reduces privacy risks by ensuring organizations collect only the data they genuinely need, limiting exposure in case of a breach.
Question 3: A loan approval model achieves equal accuracy across racial groups but still denies loans to minority applicants at a higher rate. Which fairness metric captures this disparity?
- Accuracy parity
- Demographic parity (statistical parity) (Correct answer)
- Calibration
- Individual fairness
Correct answer: Demographic parity (statistical parity)
Demographic parity measures whether the positive outcome rate is equal across groups, regardless of model accuracy within groups.
Question 4: When deploying an AI system in a criminal sentencing context, which concern is MOST ethically critical?
- Inference latency of the model
- Potential for the model to perpetuate systemic racial bias in sentencing (Correct answer)
- The licensing cost of the ML framework used
- Model interpretability for software engineers
Correct answer: Potential for the model to perpetuate systemic racial bias in sentencing
Criminal sentencing has irreversible consequences on individuals' lives, making fairness and bias the paramount ethical concern.
Question 5: Which practice helps ensure accountability when an AI system causes harm?
- Keeping model architecture proprietary
- Maintaining detailed audit logs of model decisions and data lineage (Correct answer)
- Training models only on public datasets
- Reducing the number of stakeholders who can access the model
Correct answer: Maintaining detailed audit logs of model decisions and data lineage
Audit logs create a traceable record of decisions and data flows, enabling post-hoc investigation and accountability when AI causes harm.
Question 6: The IEEE Ethically Aligned Design framework prioritizes which of the following as its first principle?
- Efficiency and scalability
- Human well-being as the primary goal of AI development (Correct answer)
- Intellectual property protection
- Open-source accessibility
Correct answer: Human well-being as the primary goal of AI development
IEEE's Ethically Aligned Design places human well-being at the center, asserting that AI must ultimately serve and enhance human flourishing.
Question 7: An AI model's training data was collected without proper consent from social media users. Publishing a paper using this model raises which primary ethical concern?
- Overfitting due to social media noise
- Violating users' privacy and autonomy through non-consensual data use (Correct answer)
- Insufficient model regularization
- Lack of hyperparameter tuning
Correct answer: Violating users' privacy and autonomy through non-consensual data use
Using data collected without consent violates individuals' autonomy and privacy rights, regardless of how the data was technically obtained.
DSE Ethical Considerations in AI 4
Which of the following is an example of 'automation bias' in an AI-assisted medical diagnostic system?