Software Testing Research & Evidence-Based Practice 2 — Questions and Answers
Question 1: What does a systematic literature review (SLR) in software testing research primarily aim to do?
- Summarize all available evidence on a topic using a defined, repeatable protocol (Correct answer)
- Publish a single expert opinion on best testing practices
- Create new test tools based on researcher preference
- Identify bugs in a specific software product
Correct answer: Summarize all available evidence on a topic using a defined, repeatable protocol
An SLR uses a rigorous, predefined protocol to identify, evaluate, and synthesize all relevant research on a topic.
Question 2: In research terminology, what is the difference between a hypothesis and a research question?
- A hypothesis is a testable prediction; a research question is an open inquiry (Correct answer)
- A research question must be answered with statistics; a hypothesis cannot
- They are interchangeable terms in software testing research
- A hypothesis is used only in qualitative studies
Correct answer: A hypothesis is a testable prediction; a research question is an open inquiry
A hypothesis makes a specific, testable prediction, while a research question frames an open-ended inquiry to be explored.
Question 3: Which metric is most commonly used in software testing research to measure the effectiveness of a test suite?
- Mutation score (Correct answer)
- Lines of code
- Number of test cases written
- Developer experience level
Correct answer: Mutation score
Mutation score measures the percentage of injected faults (mutants) that a test suite detects, directly reflecting its fault-detection effectiveness.
Question 4: What is a 'quasi-experiment' in the context of software engineering research?
- A study that lacks full random assignment of participants to conditions (Correct answer)
- An experiment conducted entirely in a simulation environment
- A study that uses only qualitative data
- An informal pilot test run before the main study
Correct answer: A study that lacks full random assignment of participants to conditions
A quasi-experiment lacks full randomization, meaning subjects are not randomly assigned to treatment and control groups.
Question 5: Which of the following best describes 'external validity' in a software testing experiment?
- The degree to which results can be generalized beyond the study's specific context (Correct answer)
- The accuracy of the measurement instruments used
- Whether the study was peer-reviewed before publication
- The extent to which results replicate within the same study
Correct answer: The degree to which results can be generalized beyond the study's specific context
External validity concerns how well findings generalize to other settings, populations, or software projects.
Question 6: In evidence-based software testing, what is the primary purpose of a 'control group'?
- To provide a baseline for comparison against the experimental treatment (Correct answer)
- To test the software under adversarial conditions
- To validate regulatory compliance
- To ensure the test team has sufficient staffing
Correct answer: To provide a baseline for comparison against the experimental treatment
A control group does not receive the experimental treatment and serves as the baseline against which the treatment group is compared.
Question 7: What does 'triangulation' mean when applied to qualitative software testing research?
- Using multiple data sources or methods to increase confidence in findings (Correct answer)
- Applying geometry to model software defect distributions
- Running the same test case three times to confirm failure
- Comparing results from three independent test automation tools
Correct answer: Using multiple data sources or methods to increase confidence in findings
Triangulation strengthens qualitative findings by cross-validating them through multiple perspectives, methods, or data sources.
Software Testing Research & Evidence-Based Practice 2
What does a systematic literature review (SLR) in software testing research primarily aim to do?