Software Testing Research & Evidence-Based Practice 3

Question 1

Which research method involves observing software teams in their natural work environment without manipulating variables?

Accepted Answer

Ethnographic study

Answer

Ethnographic studies observe practitioners in their real environment to understand behaviors and practices without researcher intervention.

Question 2

When a testing research paper reports a 'p-value < 0.05', what does this conventionally indicate?

Accepted Answer

The result is statistically significant under the null hypothesis at a 5% threshold

Answer

A p-value below 0.05 means there is less than a 5% probability of observing the result if the null hypothesis were true, meeting the conventional significance threshold.

Question 3

What is 'publication bias' and why is it relevant to evidence-based software testing?

Accepted Answer

The tendency for positive results to be published more often, skewing the available evidence

Answer

Publication bias inflates apparent effectiveness of techniques because negative or null results are less likely to be published.

Question 4

In a survey-based software testing study, what is 'response bias'?

Accepted Answer

Participants answer in ways they believe are expected rather than reflecting their true experience

Answer

Response bias occurs when participants skew answers toward socially desirable or expected responses rather than honest ones.

Question 5

What does 'effect size' measure in a software testing experiment, and why does it matter beyond p-values?

Accepted Answer

The practical magnitude of a difference between groups, independent of sample size

Answer

Effect size quantifies how large a difference is in practical terms, whereas a p-value only indicates whether it is statistically distinguishable.

Software Testing Practice Test

Software Testing Practice Test

Software Testing Research & Evidence-Based Practice 3