Research & Evidence-Based Practice Flashcards

A cloud team wants to evaluate the security posture of their infrastructure. Which evidence source provides the most objective, standardized baseline?

Answer: CIS Benchmarks or NIST frameworks applied via automated compliance scanning tools

CIS Benchmarks and NIST frameworks are independently developed standards, and automated scanning provides objective, repeatable measurement against them.

What is the primary purpose of an A/B test in a cloud platform engineering context?

Answer: To isolate the causal effect of a single change by exposing different populations to each variant simultaneously

A/B testing isolates causality by holding all variables constant except the one being tested, enabling confident attribution of observed differences to that change.

A cloud engineer references a 2019 study on Kubernetes networking performance to justify a current architectural decision. What concern should be raised?

Answer: Kubernetes networking has evolved significantly since 2019 and the findings may no longer reflect current behavior

Cloud and container technologies evolve rapidly, so older studies may reflect deprecated behavior, outdated defaults, or features that have since changed substantially.

Which statistical concept is most important to understand when evaluating cloud cost-saving reports that claim a specific percentage reduction?

Answer: Whether the reported change is statistically significant and not within normal variability

Without statistical significance testing, a reported percentage reduction could simply reflect normal cost variance rather than a genuine improvement.

A team uses the DORA metrics (deployment frequency, lead time, MTTR, change failure rate) to benchmark their engineering performance. What is a key limitation of comparing your DORA scores to published industry averages?

Answer: Industry averages aggregate diverse contexts and may not reflect organizations with similar tech stacks, team sizes, or domains

Aggregated industry benchmarks mask significant contextual variation, making direct comparison without accounting for organizational similarity potentially misleading.

What is the main advantage of using Infrastructure as Code (IaC) for research experiments in cloud environments?

Answer: IaC enables reproducible, version-controlled environments that can be reliably replicated for consistent experiment conditions

IaC captures environment configuration in code, ensuring each experiment iteration starts from an identical, documented baseline — a prerequisite for reproducible results.

A cloud team discovers conflicting recommendations between their cloud provider's Well-Architected Framework and an independent third-party guide. How should they resolve this conflict?

Answer: Evaluate both sources for their reasoning, test the conflicting approaches in your context, and document which performs better for your workload

Empirically testing conflicting recommendations in your own environment produces context-specific evidence that supersedes general guidance from either source.