Research & Evidence-Based Practice Flashcards

A cloud platform team wants to decide which observability tool to adopt. Which research approach produces the most actionable, context-specific evidence?

Answer: Run a time-boxed bake-off of finalist tools against your actual production workloads using predefined evaluation criteria

A structured bake-off against real workloads with defined criteria generates first-party evidence directly applicable to your environment.

What does 'reproducibility' mean in the context of cloud infrastructure experiments?

Answer: Another engineer can follow the documented methodology and obtain consistent results

Reproducibility means the experiment is documented thoroughly enough that independent replication yields the same findings, a core scientific validity standard.

A cloud team is deciding whether to adopt a service mesh based on a single case study from a large tech company. What critical thinking concern applies here?

Answer: A single case study from a dissimilar organization may not generalize to your context

Single case studies, especially from organizations with very different scale or architecture, have limited generalizability and require corroborating evidence.

Which approach best supports evidence-based selection of a cloud region for a new application deployment?

Answer: Gather latency data to target user populations, compliance requirements, and cost data, then score regions against weighted criteria

Multi-criteria scoring with quantitative data for each relevant factor produces a defensible, evidence-based region selection.

What is 'survivorship bias' and how can it distort cloud architecture research?

Answer: Overweighting architectures that are still running while ignoring failed designs that were abandoned

Survivorship bias leads teams to study only successful architectures, missing lessons from failures that were quietly discarded.

A cloud team wants to reduce mean time to recovery (MTTR). Which evidence-based practice is most directly supported by incident post-mortem research?

Answer: Implementing blameless post-mortems with documented action items and tracking their completion rate

Research on high-performing engineering organizations consistently shows blameless post-mortems with tracked remediation items reduce MTTR over time.

When a cloud vendor publishes a benchmark claiming their service is '3x faster,' what should a cloud engineer examine first?

Answer: The workload type, test conditions, and whether the comparison baseline is representative of your use case

Performance benchmarks are only meaningful if the workload type and conditions match your use case; otherwise the comparison is irrelevant.