Evaluation Flashcards

A study finds that cities with more libraries have lower crime rates. A mayor proposes building more libraries to reduce crime. What should a careful evaluator point out?

Answer: The correlation may be explained by a third variable such as education funding or income levels

A third variable (confound) like wealth or educational investment could explain both the prevalence of libraries and lower crime, making a causal inference unwarranted.

When evaluating a deductive argument, what does it mean for the argument to be 'valid' but 'unsound'?

Answer: The conclusion follows logically from the premises, but at least one premise is false

Validity concerns logical structure; soundness additionally requires true premises — so a valid argument with a false premise is unsound.

A financial advisor says: 'My last ten clients who followed this strategy made money, so it must be a winning strategy.' What evaluation concern applies?

Answer: Survivorship bias / hasty generalization

Ten successful clients may not represent all clients, and clients who lost money may not be mentioned, making this an unrepresentative and potentially biased sample.

Which of the following is the strongest evidence that an evaluation framework is being applied consistently?

Answer: The same criteria are used regardless of which option they favor

Consistent application of criteria regardless of outcome ensures the evaluation is objective and not selectively applied to favor a predetermined conclusion.

An argument contains the premise: 'All experts agree that this treatment works.' What should an evaluator examine before accepting this?

Answer: Whether 'all experts agree' is actually true and whether the experts are relevant and credible

Claims of universal expert consensus should be verified; appeals to authority are only strong when the experts are genuinely relevant and the consensus is real.

A law firm argues: 'This precedent was established 100 years ago, so it must still be valid today.' Which evaluative weakness does this expose?

Answer: Appeal to tradition — age alone does not establish current validity

An appeal to tradition mistakes longevity for correctness; precedents may need revision as society, science, or circumstances change.

In evaluating a scientific claim, why is peer review considered an important quality indicator?

Answer: Peer review subjects research to independent expert scrutiny before publication

Peer review provides an independent check on methodology, reasoning, and evidence quality before research enters the scientific record.