Full-Stack Development Research & Evidence-Based Practice 4 — Questions and Answers
Question 1: A team measures API response time using arithmetic mean. A load test reveals occasional 10-second spikes hidden by the average. Which metric better captures this?
- Median
- 95th or 99th percentile latency (Correct answer)
- Standard deviation alone
- Mode
Correct answer: 95th or 99th percentile latency
High percentile latency (p95, p99) surfaces tail latencies that hurt user experience but are masked by averaging.
Question 2: When reviewing a peer's pull request, you notice a new algorithm lacks any cited source for its correctness. What is the best evidence-based response?
- Approve it if the tests pass
- Request either a proof sketch, a reference to a well-known algorithm, or test coverage for edge cases (Correct answer)
- Reject it because all algorithms need academic citations
- Rewrite the algorithm yourself
Correct answer: Request either a proof sketch, a reference to a well-known algorithm, or test coverage for edge cases
Correctness evidence—whether a proof, authoritative reference, or comprehensive edge-case tests—reduces the risk of subtle bugs in algorithmic code.
Question 3: Which of the following is an example of survivorship bias in evaluating development frameworks?
- Choosing a framework because it has the most documentation
- Concluding a framework is reliable because successful companies use it, ignoring the many that failed with it (Correct answer)
- Preferring open-source frameworks over proprietary ones
- Using a framework that matches team expertise
Correct answer: Concluding a framework is reliable because successful companies use it, ignoring the many that failed with it
Survivorship bias occurs when we only see successful outcomes; companies that failed with the same framework are rarely visible in public discourse.
Question 4: A startup claims their new database achieves '10x faster writes than Postgres.' What detail is MOST critical to evaluate this claim?
- The company's funding amount
- The workload type, dataset size, hardware configuration, and Postgres version used in the benchmark (Correct answer)
- The number of engineers at the startup
- Whether the claim appeared in a peer-reviewed journal
Correct answer: The workload type, dataset size, hardware configuration, and Postgres version used in the benchmark
Benchmark validity depends entirely on whether the test conditions match your use case; without methodology details, the number is meaningless.
Question 5: What does 'dogfooding' mean in the context of evidence-based development?
- Testing software on animals before human users
- A team using their own product internally to discover issues before external release (Correct answer)
- Feeding raw data into ML models without preprocessing
- A canary deployment to 100% of traffic
Correct answer: A team using their own product internally to discover issues before external release
Dogfooding generates internal evidence about real usability and reliability issues before customers encounter them.
Question 6: A developer proposes replacing a working library with a newer alternative because the new one 'feels more modern.' What evidence should drive this decision?
- The newer library's logo and website design
- Measured improvements in bundle size, performance, developer ergonomics, or maintenance burden for their specific use case (Correct answer)
- The number of GitHub stars the new library has
- Whether the new library was mentioned at a recent conference
Correct answer: Measured improvements in bundle size, performance, developer ergonomics, or maintenance burden for their specific use case
Technology decisions should be driven by measurable outcomes relevant to your project, not aesthetics or hype.
Question 7: In software research, what is the difference between internal validity and external validity?
- Internal validity concerns sample size; external validity concerns p-values
- Internal validity means the study measures what it claims; external validity means findings generalize to other contexts (Correct answer)
- Internal validity applies to unit tests; external validity applies to integration tests
- They are synonyms for reliability and validity respectively
Correct answer: Internal validity means the study measures what it claims; external validity means findings generalize to other contexts
Internal validity asks 'did we measure the right thing correctly?' while external validity asks 'do these results apply beyond this study's conditions?'
Full-Stack Development Research & Evidence-Based Practice 4
A team measures API response time using arithmetic mean.
A load test reveals occasional 10-second spikes hidden by the average.
Which metric better captures this?