FIDO Research Methods & Evidence-Based Practice 3 — Questions and Answers
Question 1: A FIDO certification lab conducts conformance testing. Which principle ensures test results are reproducible by an independent lab?
- Test transparency and documented methodology (Correct answer)
- Proprietary black-box testing protocols
- Single evaluator assessment
- Vendor self-attestation
Correct answer: Test transparency and documented methodology
Reproducibility requires transparent, documented methodologies so independent labs can replicate results under the same conditions.
Question 2: When evaluating a FIDO authenticator's resistance to side-channel attacks, which research methodology provides the strongest empirical evidence?
- Controlled laboratory experiments measuring power consumption and timing (Correct answer)
- Literature review of manufacturer whitepapers
- Structured interviews with security engineers
- Regulatory compliance checklist review
Correct answer: Controlled laboratory experiments measuring power consumption and timing
Controlled experiments measuring physical signals like power and timing provide direct empirical evidence of side-channel vulnerabilities.
Question 3: In FIDO security research, what is the primary purpose of a threat model developed before empirical testing begins?
- To identify and scope the adversarial scenarios the study should evaluate (Correct answer)
- To certify the device meets FIDO2 Level 2 requirements
- To replace penetration testing entirely
- To determine user acceptance rates
Correct answer: To identify and scope the adversarial scenarios the study should evaluate
A threat model scopes the adversary capabilities and attack scenarios the research will empirically validate.
Question 4: A meta-analysis of FIDO phishing-resistance studies reports a pooled odds ratio of 0.02 compared to passwords. What does this finding mean?
- FIDO users are 98% less likely to be successfully phished than password users (Correct answer)
- FIDO authentication is 0.02 seconds faster than passwords
- 2% of FIDO users report phishing incidents
- The studies included had a 2% overlap in samples
Correct answer: FIDO users are 98% less likely to be successfully phished than password users
An odds ratio of 0.02 means FIDO users have dramatically reduced odds of phishing success—approximately 98% lower than password users.
Question 5: Which sampling method is most appropriate when researching FIDO adoption across heterogeneous enterprise sectors?
- Stratified random sampling by industry sector (Correct answer)
- Snowball sampling from a single vendor
- Convenience sampling of volunteers
- Purposive sampling from one geographic region
Correct answer: Stratified random sampling by industry sector
Stratified random sampling ensures proportional representation across industry sectors, supporting generalizable findings.
Question 6: A researcher reports that FIDO-based login reduces support ticket volume by 60% but does not report confidence intervals. What is the key limitation of this evidence?
- The precision and reliability of the estimate cannot be assessed without confidence intervals (Correct answer)
- The study is automatically invalid without statistical testing
- A 60% reduction is too large to be credible
- Ticket volume is not a valid outcome measure
Correct answer: The precision and reliability of the estimate cannot be assessed without confidence intervals
Without confidence intervals, readers cannot assess the precision of the estimate or whether the true effect could be negligible.
Question 7: In evidence-based evaluation of FIDO Authenticator Level requirements, what does 'normative' versus 'informative' language in the specification signify?
- Normative text states mandatory requirements; informative text provides non-binding guidance (Correct answer)
- Normative text is optional; informative text is required
- Both terms mean the same thing in FIDO specifications
- Informative text overrides normative requirements in edge cases
Correct answer: Normative text states mandatory requirements; informative text provides non-binding guidance
Normative text (using 'SHALL/MUST') defines mandatory requirements for conformance, while informative text provides guidance without obligation.
FIDO Research Methods & Evidence-Based Practice 3
A FIDO certification lab conducts conformance testing.
Which principle ensures test results are reproducible by an independent lab?