Psychometrics and Scaling Methods Quiz

Test Development Lecture 5 - Psych Assessment

• This lecture covers test development, a process encompassing five stages.
• Objective: Students will be able to identify test development concepts, understand scientific test construction, and create test items using item analysis.

Stages of Test Development

• Conceptualization: Initial stage; determining the test's purpose, target population, and measurement objectives. Includes preliminary research on the construct (emerging behavior pattern) to form a test prototype. Pilot work is the generalized term for this research. Items are evaluated to determine suitability for the final test.

• Construction: Developing the test's format and questions. Various methods, such as rating scales, expert rankings, equal-appearing intervals, absolute scaling (based on age groups), and Likert scales, help establish the measurement criteria. Guttman scales, empirical keying, paired comparisons, and categorical scaling are also referenced as potential methods.

• Tryout: The test is administered to a representative sample to evaluate its effectiveness and identify problematic areas. The sample should mirror the target population. The number of participants should be at least 20 per item.

• Item Analysis: Evaluating individual items to assess quality and identify areas needing correction. Measures include calculating the difficulty index of items, their reliability index, and the discrimination between high and low scorers. Qualitative item analysis, examining item interaction, further evaluates them. Factors such as guessing, fairness, and speed tests are also considerations during analysis. 'Think Aloud' test administration, where respondents verbalize their thoughts during the process, might be employed. Expert panels assess these items for effectiveness for different populations, especially underserved ones.

• Revision: Making refinements to the test based on previous results. Factors considered for revision may include outdated material, cultural changes, changes in norms (standards), validity (degree of accuracy), reliability (consistency), or theoretical adjustments. Revision may involve cross-validation on new populations or co-validation using the same sample. Quality assurance is crucial through mechanisms like Anchor Protocols—established by a high-authority scorer for standardized scoring processes. Also, identify and correct scoring discrepancies.

• Considerations: Considerations regarding writing test items include determining content coverage, appropriate item formats, and number of items per content area. Clarity in measuring the intended concepts is important. Test items should avoid complexity, double-barreled ideas, and be appropriate for the intended test takers. Positively and negatively worded items should be mixed in.

Scoring Models:

• Cumulative: Scores represent the number of items a person answers correctly or agrees with, reflecting the degree of the target construct.
• Class/Category: Places individuals into specific categories for descriptions or predictions based on the results.
• Ipsative: Individuals choose between equally socially acceptable alternatives, emphasizing comparisons rather than absolute scores.

Assignment

• Construct various test types. -Five-item binary scale for motivation -Five-item Likert scale for selfishness -Five-item semantic differential scale for COVID-19 response attitude toward the Philippine government -Five-item Guttman scale for attitude toward depression