Psychology Assessments and Research Methods

Arts Degree Advice Session

• Wednesday, September 14th
• Time: 6:00 PM - 7:00 PM
• Location: AMB-1021, O'hEocha Theatre
• Registration: Careers Connect

Research Methods in Personality

• Data and Design

Psychology's Emphasis on Method

• Goal: Continuously improve tentative answers to questions.
• Focus: Thinking and seeking new knowledge.
• Research: Exploration of the unknown, requires data gathering.

Measuring Personality

• Psychological triad (thoughts, feelings, behaviors) cannot be directly observed.
• Personality is inferred from observable aspects.
• No perfect measures exist.

Sources of Information

• S Data: Self-report (thoughts, feelings, etc.)
• I Data: Informant data (based on information from others like partners, parents, friends, teachers)
• L Data: Life data (verifiable life history like marriage, employment)
• B Data: Behavioral observations (lab and real-life contexts).

Self-reports (S Data)

• Open-ended questions
• Surveys/questionnaires
• High face validity

Ratings by others (I data)

• Direct observer report
• Peer ratings

Advantages of S Data

• Large amount of information
• Self-awareness, introspection
• Access to internal experiences
• Causal force (self-efficacy, self-verification)
• Simple and easy to collect

Disadvantages of S Data

• Bias
• Overly positive or negative views
• Desire for privacy, faking
• Errors (fish-and-water effect, memory distortion, lack of self-insight)
• Carelessness

Advantages of I Data

• Large amount of information from many perspectives
• Real-world basis
• Common sense understanding of context
• Insight into causal force (reputation and expectations)
• Definitional truth

Disadvantages of I Data

• Limited behavioral information
• Lack of access to private experience
• Error (remembering extremes/unusual/emotionally arousing behaviors)
• Bias (letter of recommendation effect, prejudices, and stereotypes).

Life Outcomes (L) Data

• Obtained from archival records or self-report.
• Assess the long-term results or consequences of personality traits.
• Archival records Advantages/disadvantages.
• Results, "residue", of personality.

Advantages and Disadvantages of L Data

• Objective and verifiable data.
• Intrinsic importance and psychological relevance.
• Potential disadvantage: multidetermination (many factors impact life outcomes).

B-Data (Behavioral Data): Naturalistic Observation

• Observations in natural settings (or as close as possible)
• Hybrid approaches (e.g., social media, self-reports on naturalistic observations)
• Experience sampling method, diary studies, ambulatory assessment, ecological momentary assessment.

Advantages and Disadvantages of Natural B Data

• Realistic representation of behavior.
• Disadvantages: Difficulty, rarity of desired contexts

B-Data (Behavioral Data): Laboratory B Data

• Create situations to observe behavior.
• Examine reactions to situations.
• Represent real-world contexts difficult to observe.
• Physiological measures (involuntary responses).

Advantages & Disadvantages of Laboratory B Data

• Wide variety of contexts
• Appearance of objectivity
• Disadvantages: Difficult, expensive, uncertain interpretation

Discussion: S Data and I Data

• Aspects of personality likely and unlikely to be accurately reported.
• Accuracy of evaluations from peers, coworkers, and parents.

Mixed Types of Data

• Data commonly don't fit into one category.
• Multiple data types are relevant to personality.
• Each type has advantages and disadvantages.

Research Design

• What method to use?

Case Study

• Involves closely studying events or a person within real-life context.
• Generates new ideas tested in larger studies using various methods.
• Clinical experience building theories.

Correlational Research

• Looks at the relationship between two variables.
• Focuses on variations, correlation, and individual differences.
• Correlation does not equal causation.

Experimental Research

• Manipulates independent variable (IV) under controlled conditions observing dependent variable (DV).
• Focuses on central tendencies, not variations.
• Direct causal effects of IV on DV are analyzed.
• Cannot manipulate personality traits.
• Quasi-experiments use existing personality traits to assign participants to conditions.
• Experimental research complexities: uncertainty about manipulation, third-variable problem, difficulty in creating realistic levels of traits.

Personality Assessment

• Treatment planning.
• Ethical issues and insurance reimbursement.
• Diagnosis and behavior cluster summaries.
• Assessing client behavior.

Personality Tests: Objective Tests

• Specific questions (yes/no, true/false, multiple choice).
• Computer-administered.
• Quick scoring, analysis, and interpretation.
• Validity and subjectivity of test items.
• Number of items and reliability (aggregation principle and Spearman-Brown formula).

Constructing Personality Tests

• Rational method: Questions directly related to constructs.
• Factor analysis: Statistical method to group related items into factors.

Two Approaches to Assessment Development

• Rational (Theoretical): Starts with a concept, selects items, tests validity and reliability.
• Empirical (Data-Based): Empirically driven from many items, statistical methods for item selection.

Methods of Objective Test Construction:

• Rational: Items identical meaning for test-taker and creator, self-assessment capability, accurate reporting, validating items. Most common.
• Factor analytic: Create long list of objective items, administer to many subjects, analyze items that cluster together (factors), name and describe the factor.
• Empirical: Generate items, divide subjects into groups, administer test, compare answers from the different groups, cross validation.

When Are Different Methods Used?

• Rational approach used in theory building, empirical approach used in practical needs such as vocational interests/occupational assessment.

Methods of Objective Test Construction (combined methods)

• Generate items using the rational method, analyze responses with factor analysis, and correlate factors with independent criteria.

Projective Assessments

• Person responds to vague stimuli, interpreting how they relate to personality.
• Everyone expected to have different interpretations.
• Assessment is now moving away from norm-referenced tests.
• Structured interviews rather than "tests" are encouraged.

Projective Assessments: Types

• TAT (Thematic Apperception Test) —Story creation from pictures.
• Draw-A-Person test: Drawing reveals aspects of personality.
• Rorschach inkblot test — Reactions to inkblots reveal unconscious mind.

Projective Methods

• Often used by clinical psychologists linking to Freud's ideas.
• B-data.
• Word association/Sentence completion tests- used in projective methods.

Advantages of Projective Methods

• Helpful in breaking the ice (initial contact with subject)
• Skilled clinicians can sometimes extract nuanced information not found in controlled research.
• Limited support for validity of the tests, notably the Rorschach test.

Disadvantages of Projective Methods

• Validity evidence is scarce.
• Expensive and time-consuming to administer.
• Psycholgists cannot be sure about the meaning of responses.
• Other, less expensive tests might give more useful results, and these are better used.
• Inappropiate use and practice.

Projective Tests - Critique

• "Nobody agrees..." — No objective scoring, meaning dependent on responders.

Objective vs. Projective Tests - Validity/Reliability

• Which is more valid and reliable: Objective vs Projective?

Modern Personality Tests?

• Combination of data and methods, possibly including data from social media.

Facebook Likes and Personality

• Facebook likes (average 170 per person) analyzed to predict personality traits such as social openness and extraversion but also intelligence, race, religion, sexual orientation and political beliefs.
• Demographic information also collected, such as age, gender, relationship status, political views and more.

Personality Tests: Examples