Full-Stack Development Research & Evidence-Based Practice 2 â Questions and Answers
Question 1: When evaluating a third-party library for production use, which metric is MOST important as a primary signal of long-term viability?
- Number of GitHub stars
- Frequency of recent commits and maintained changelog (Correct answer)
- Total number of npm downloads ever
- Age of the repository
Correct answer: Frequency of recent commits and maintained changelog
Active maintenanceâevidenced by recent commits and a maintained changelogâbest predicts a library's long-term viability and security patch cadence.
Question 2: A developer benchmarks two sorting algorithms and finds Algorithm A is faster on their test machine. What threat to internal validity should they address first?
- Publication bias
- Confounding variables such as dataset size and hardware warm-up (Correct answer)
- Lack of a control group
- Insufficient sample diversity
Correct answer: Confounding variables such as dataset size and hardware warm-up
Confounding variables like dataset characteristics and CPU cache warm-up can invalidate benchmark results without proper controls.
Question 3: Which practice best supports reproducible research in software experiments?
- Sharing only the final results in a blog post
- Publishing code, datasets, and environment specifications together (Correct answer)
- Running experiments on the latest hardware available
- Using proprietary tools so results are verifiable by vendors
Correct answer: Publishing code, datasets, and environment specifications together
Reproducibility requires sharing all artifactsâcode, data, and environmentâso others can independently verify results.
Question 4: A team wants to determine if a new caching strategy reduces API latency. What is the correct null hypothesis?
- The new caching strategy reduces latency by at least 20%
- The new caching strategy has no effect on API latency (Correct answer)
- The old strategy is better than the new one
- Latency cannot be measured reliably
Correct answer: The new caching strategy has no effect on API latency
The null hypothesis always states there is no effect; the experiment attempts to reject it with sufficient evidence.
Question 5: When reading a case study claiming a microservices migration reduced deployment time by 60%, what is the strongest caveat to consider?
- Case studies are always fabricated
- Results may not generalize because case studies reflect a single organization's context (Correct answer)
- The percentage improvement is too large to be credible
- Microservices always improve deployment time
Correct answer: Results may not generalize because case studies reflect a single organization's context
Case studies provide rich detail but low external validityâorganizational context, team size, and tooling heavily influence outcomes.
Question 6: What does a p-value of 0.03 mean in the context of a performance A/B test?
- There is a 3% chance the new version is better
- There is a 3% chance of observing results this extreme if the null hypothesis is true (Correct answer)
- The result is practically significant
- The effect size is 3%
Correct answer: There is a 3% chance of observing results this extreme if the null hypothesis is true
A p-value measures the probability of obtaining the observed data (or more extreme) assuming the null hypothesis is true, not the probability the hypothesis is correct.
Question 7: A developer finds conflicting benchmarks for database connection pooling online. What is the best next step?
- Trust the more recent article regardless of methodology
- Replicate the benchmark in their specific environment and workload (Correct answer)
- Average the reported numbers across all sources
- Discard all external benchmarks and rely on intuition
Correct answer: Replicate the benchmark in their specific environment and workload
Benchmarks are highly context-dependent; replicating them in your actual environment eliminates external confounds.
Full-Stack Development Research & Evidence-Based Practice 2
When evaluating a third-party library for production use, which metric is MOST important as a primary signal of long-term viability?