FIDO Research Methods & Evidence-Based Practice 2 — Questions and Answers
Question 1: When designing a usability study for a FIDO2 authenticator, which control technique best isolates the effect of the authenticator design from user familiarity?
- Within-subjects counterbalanced design (Correct answer)
- Single-group pre-test only design
- Convenience sampling with no control group
- Expert review without user participants
Correct answer: Within-subjects counterbalanced design
A within-subjects counterbalanced design controls for individual differences and order effects, isolating the authenticator variable.
Question 2: A researcher compares phishing resistance of FIDO keys versus OTP tokens across 300 participants. The p-value is 0.03. What does this indicate?
- There is a statistically significant difference at the 0.05 level (Correct answer)
- The effect size is large
- The study has high ecological validity
- The null hypothesis is confirmed
Correct answer: There is a statistically significant difference at the 0.05 level
A p-value of 0.03 falls below the conventional 0.05 threshold, indicating the result is statistically significant.
Question 3: Which type of evidence is considered highest on the evidence hierarchy when evaluating FIDO authenticator security claims?
- Systematic review and meta-analysis of controlled trials (Correct answer)
- Expert opinion from a single manufacturer
- Case study of one enterprise deployment
- Anecdotal field reports from help desk staff
Correct answer: Systematic review and meta-analysis of controlled trials
Systematic reviews and meta-analyses synthesize multiple controlled studies and sit at the top of the evidence hierarchy.
Question 4: In a FIDO authenticator threat modeling study, what does 'attack surface reduction' as an evidence-based goal primarily involve?
- Minimizing the number of exploitable interfaces and entry points (Correct answer)
- Increasing the number of authentication factors
- Expanding API endpoint coverage
- Adding biometric fallback options
Correct answer: Minimizing the number of exploitable interfaces and entry points
Attack surface reduction focuses on eliminating or hardening exploitable interfaces to lower the probability of successful attacks.
Question 5: A study measures false acceptance rate (FAR) and false rejection rate (FRR) for a biometric FIDO authenticator. Which metric best represents the tradeoff between security and usability?
- Equal Error Rate (EER) (Correct answer)
- Total Error Rate (TER)
- Genuine Acceptance Rate (GAR)
- Impostor Acceptance Rate (IAR)
Correct answer: Equal Error Rate (EER)
The Equal Error Rate (EER) is the point where FAR equals FRR, representing the optimal tradeoff between security and usability.
Question 6: Researchers conducting a longitudinal study on FIDO key retention find 40% of participants lose their key within 6 months. What evidence-based intervention does literature most support?
- Account recovery via registered backup authenticators (Correct answer)
- Mandatory daily PIN changes
- Single-device enforcement with no backup
- Biometric-only authentication without backup
Correct answer: Account recovery via registered backup authenticators
Evidence supports registering backup authenticators as the most practical recovery mechanism to maintain access continuity.
Question 7: Which research approach is most appropriate for generating hypotheses about real-world FIDO deployment failures in enterprises?
- Grounded theory analysis of incident reports (Correct answer)
- Double-blind randomized controlled trial
- Laboratory experiment with synthetic tasks
- Cross-sectional survey with Likert scales
Correct answer: Grounded theory analysis of incident reports
Grounded theory is used to build theory from naturalistic data such as incident reports, making it ideal for exploratory hypothesis generation.
FIDO Research Methods & Evidence-Based Practice 2
When designing a usability study for a FIDO2 authenticator, which control technique best isolates the effect of the authenticator design from user familiarity?