Norming and Standardization in Test Development

Inter-Rater Reliability

• A measure has good inter-rater reliability if it yields identical ratings among several raters using nominal scales.
• The kappa statistic is used to assess the level of agreement among raters.

Internal Consistency

• It measures the degree to which each test item measures the same construct.
• It is the intercorrelations among the items.
• If all items on a test measure the same construct, then it has good internal consistency.
• Formulae used to measure internal consistency: Spearman-Brown, Kuder-Richardson, and Cronbach’s alpha.

Formulae for Internal Consistency

• Spearman-Brown Formula: estimates the correlations between the two halves of a test if each half had been the length of the whole test and had equal variances.
• Cronbach’s coefficient alpha: confirms that a test has substantial reliability in case the two halves of a test have unequal variances.
• Kuder-Richardson 20 (KR20) Formula: used for calculating the reliability of a test in which the items are dichotomous or scored as 0 or 1.

Validity of Test Results

• Test results can be affected by extraneous factors such as non-standardized administration, inexperienced examiner, noisy testing room, scared examinee, or careless scoring.
• Standardized procedures in test administration are essential to ensure reliable test results.

Clinical Interviewing

• Client-centered interview: nondirective and unstructured, behaviors are brought about by choice and self-change.
• Behavioral interview: directive and structured, environment or external conditions are the causes of behavior.
• Diagnostic interview: to develop a specific diagnosis based on the DSM-IVTR, DSM-V, or ICD-10.
• Informal or exploratory interview: to assess a person’s socioemotional functioning, coping styles, social support, etc.

Types of Clinical Interviews

• Structured Clinical Interviews: specific sets of questions are presented in a particular order, e.g. Schedule for Affective Disorders and Schizophrenia (SADS) and Child Assessment Schedule (CAS).
• Mental Status Examination: reviews the major systems of psychiatric functioning, assessing domains such as appearance, attitude, behavior, mood and affect, speech, thought processes, thought content, perceptions, cognition, insight, and judgment.
• Mini Mental Status Exam: a 30-point questionnaire used to assess the presence or absence of cognitive impairment.

Draw-A-Person (DAP) Test

• Stance: indicates concern regarding the need to control threatening impulses, degree of security the person feels in their environment.
• Emotional Indications: power and strength, power strivings, left shoulder has feminine indications while right shoulder has male indications.
• Koppitz's indications of problems with socioemotional functioning:
  • Poor integration of parts or figures
  • Shading of face
  • Shading of body and/or limbs
  • Shading of hands and/or neck
  • Gross asymmetry of limbs
  • Slanting figure, axis of figure tilted by 15 degrees or more
  • Tiny figure, two inches or less
  • Big figure, nine inches or more
  • Transparencies
  • Tiny head, less than 1/10th of the total figure
  • Crossed eyes, both eyes turn in or out

Norming and Standardization

• Norming: the process of deriving norms, may be modified to describe a particular type of norm derivation.
• Standardization: the process of administering a test to a representative sample of test takers for the purpose of establishing norms.

Sampling to Develop Norms

• Probability sampling: the entire population is known, each individual has a specifiable probability of selection, and sampling occurs by a random process based on probabilities.
• Non-probability sampling: the population is not completely known, individual probabilities cannot be known, and the sampling method is based on common sense or ease but still maintains representativeness and avoids bias.

Age-Related Norms

• Certain tests have different normative groups for particular age groups, e.g. most IQ tests.
• Agerelated norms are used when the norms of a particular age group cannot be used on a younger or older age group.

Classical Test Score Theory

• Assumes that each person has a true score that would be obtained if there were no errors in measurement.
• Measuring instruments are imperfect, therefore the observed score for each person almost always differs from the person’s true ability or characteristic.