Assessing Credibility of Claims Flashcards

A health claim states: 'Nine out of ten doctors recommend our supplement.' Which credibility problem is MOST directly illustrated by this statement?

Answer: Lack of specification — which doctors, how many, what question was asked, and who funded the survey are all unknown

A credible statistical claim requires specification of the sample, methodology, and funding. Without these details, the statistic is unverifiable.

A nursing professor tells students: 'I've been practicing for 30 years and I've never seen a patient harmed by that medication.' This claim should be assessed with which critical lens?

Answer: Personal clinical experience, however extensive, is not a substitute for systematic evidence and can be subject to survivorship bias

Even extensive personal experience is subject to survivorship bias, recall bias, and is inferior to controlled systematic studies when assessing drug safety.

A medical journal publishes a randomized controlled trial with a p-value of 0.049. A subsequent letter argues the result is not credible because the pre-specified primary outcome was changed after data collection. This argument MOST directly challenges:

Answer: Internal validity — outcome switching inflates false-positive rates and undermines the statistical credibility of the finding

Post-hoc outcome switching (HARKing) inflates the false-positive rate and directly undermines the internal validity and credibility of the reported p-value.

A pharmaceutical sales representative presents a graph showing that patients on their drug lived on average 2.1 months longer. The survival benefit was statistically significant (p=0.001). Which credibility question is MOST important?

Answer: Is 2.1 months of additional survival clinically meaningful given the drug's side effects and cost?

Statistical significance does not equal clinical significance. A 2.1-month benefit must be weighed against quality of life, side effects, and cost to assess real-world credibility.

A public health official states: 'Since we introduced the vaccine campaign, hospitalizations fell by 45%, proving the vaccine works.' Which credibility flaw is MOST apparent?

Answer: Post hoc ergo propter hoc — assuming causation from temporal sequence without controlling for other factors

Temporal precedence alone does not establish causation. Other concurrent factors (seasonal variation, herd immunity, behavior changes) may have contributed to the decline.

A health sciences student finds an article claiming that herbal supplement Z cures type 2 diabetes, published on a website with no author names, no institutional affiliation, and no citation list. The MOST appropriate credibility assessment is:

Answer: The article is unverifiable and lacks basic indicators of credibility such as authorship, affiliation, and referenced evidence

Authorship, institutional affiliation, and cited references are foundational credibility markers. Their absence makes a claim unverifiable.