PC PC Test Development and Validation 1 — Questions and Answers
Question 1: The first step in developing a psychometrically sound certification exam is typically:
- Writing test items
- Conducting a job task analysis (Correct answer)
- Establishing cut scores
- Administering pilot tests
Correct answer: Conducting a job task analysis
A job task analysis (or practice analysis) identifies the knowledge, skills, and abilities required for the role, providing the content blueprint for the exam.
Question 2: Content validity evidence in test development is best established through:
- Correlating test scores with a criterion measure
- Expert review confirming items represent the intended content domain (Correct answer)
- Factor analysis of item responses
- Calculating internal consistency
Correct answer: Expert review confirming items represent the intended content domain
Content validity is demonstrated when subject matter experts confirm that test items adequately and representatively sample the defined content domain.
Question 3: A test blueprint (table of specifications) in certification exam development specifies:
- The passing score for each examinee
- The proportion of items covering each content area (Correct answer)
- The statistical difficulty of every item
- The identities of item writers
Correct answer: The proportion of items covering each content area
A test blueprint maps content areas to the number or percentage of items, ensuring the final exam adequately represents all relevant domains.
Question 4: In psychometric test development, construct validity refers to:
- The degree to which a test measures the theoretical construct it purports to measure (Correct answer)
- The consistency of scores across repeated administrations
- Whether test content matches the curriculum
- The accuracy of scoring procedures
Correct answer: The degree to which a test measures the theoretical construct it purports to measure
Construct validity is the extent to which a test accurately measures the theoretical psychological attribute or construct it was designed to assess.
Question 5: Which method is used to set a defensible passing score on a professional certification examination?
- Norm-referencing to the top 50% of candidates
- Standard-setting methods such as the Angoff method (Correct answer)
- Setting the passing score at the median
- Using the item mean as the cut score
Correct answer: Standard-setting methods such as the Angoff method
The Angoff method and similar standard-setting procedures use expert judgment to establish criterion-referenced passing scores based on minimally competent performance.
Question 6: Pilot testing or field testing new psychometric items primarily serves to:
- Finalize the test blueprint
- Collect item performance data before operational use (Correct answer)
- Establish inter-rater reliability
- Determine certification eligibility requirements
Correct answer: Collect item performance data before operational use
Pilot testing exposes new items to a representative sample to gather empirical data on difficulty, discrimination, and potential item flaws before live administration.
PC PC Test Development and Validation 1
The first step in developing a psychometrically sound certification exam is typically: