Psychology Assessments and Research Methods

Arts Degree Advice Session

• Wednesday, September 14th
• Time: 6:00 PM - 7:00 PM
• Location: AMB-1021, O'hEocha Theatre
• Registration: Careers Connect

Research Methods in Personality

• Data and Design

Psychology's Emphasis on Method

• Goal: Continuously improve tentative answers to questions.
• Focus: Thinking and seeking new knowledge.
• Research: Exploration of the unknown, requires data gathering.

Measuring Personality

• Psychological triad (thoughts, feelings, behaviors) cannot be directly observed.
• Personality is inferred from observable aspects.
• No perfect measures exist.

Sources of Information

• S Data: Self-report (thoughts, feelings, etc.)
• I Data: Informant data (based on information from others like partners, parents, friends, teachers)
• L Data: Life data (verifiable life history like marriage, employment)
• B Data: Behavioral observations (lab and real-life contexts).

Self-reports (S Data)

• Open-ended questions
• Surveys/questionnaires
• High face validity

Ratings by others (I data)

• Direct observer report
• Peer ratings

Advantages of S Data

• Large amount of information
• Self-awareness, introspection
• Access to internal experiences
• Causal force (self-efficacy, self-verification)
• Simple and easy to collect

Disadvantages of S Data

• Bias
• Overly positive or negative views
• Desire for privacy, faking
• Errors (fish-and-water effect, memory distortion, lack of self-insight)
• Carelessness

Advantages of I Data

• Large amount of information from many perspectives
• Real-world basis
• Common sense understanding of context
• Insight into causal force (reputation and expectations)
• Definitional truth

Disadvantages of I Data

• Limited behavioral information
• Lack of access to private experience
• Error (remembering extremes/unusual/emotionally arousing behaviors)
• Bias (letter of recommendation effect, prejudices, and stereotypes).

Life Outcomes (L) Data

• Obtained from archival records or self-report.
• Assess the long-term results or consequences of personality traits.
• Archival records Advantages/disadvantages.
• Results, "residue", of personality.

Advantages and Disadvantages of L Data

• Objective and verifiable data.
• Intrinsic importance and psychological relevance.
• Potential disadvantage: multidetermination (many factors impact life outcomes).

B-Data (Behavioral Data): Naturalistic Observation

• Observations in natural settings (or as close as possible)
• Hybrid approaches (e.g., social media, self-reports on naturalistic observations)
• Experience sampling method, diary studies, ambulatory assessment, ecological momentary assessment.

Advantages and Disadvantages of Natural B Data

• Realistic representation of behavior.
• Disadvantages: Difficulty, rarity of desired contexts

B-Data (Behavioral Data): Laboratory B Data

• Create situations to observe behavior.
• Examine reactions to situations.
• Represent real-world contexts difficult to observe.
• Physiological measures (involuntary responses).

Advantages & Disadvantages of Laboratory B Data

• Wide variety of contexts
• Appearance of objectivity
• Disadvantages: Difficult, expensive, uncertain interpretation

Discussion: S Data and I Data

• Aspects of personality likely and unlikely to be accurately reported.
• Accuracy of evaluations from peers, coworkers, and parents.

Mixed Types of Data

• Data commonly don't fit into one category.
• Multiple data types are relevant to personality.
• Each type has advantages and disadvantages.

Research Design

• What method to use?

Case Study

• Involves closely studying events or a person within real-life context.
• Generates new ideas tested in larger studies using various methods.
• Clinical experience building theories.

Correlational Research

• Looks at the relationship between two variables.
• Focuses on variations, correlation, and individual differences.
• Correlation does not equal causation.

Experimental Research

• Manipulates independent variable (IV) under controlled conditions observing dependent variable (DV).
• Focuses on central tendencies, not variations.
• Direct causal effects of IV on DV are analyzed.
• Cannot manipulate personality traits.
• Quasi-experiments use existing personality traits to assign participants to conditions.
• Experimental research complexities: uncertainty about manipulation, third-variable problem, difficulty in creating realistic levels of traits.

Personality Assessment

• Treatment planning.
• Ethical issues and insurance reimbursement.
• Diagnosis and behavior cluster summaries.
• Assessing client behavior.

Personality Tests: Objective Tests

• Specific questions (yes/no, true/false, multiple choice).
• Computer-administered.
• Quick scoring, analysis, and interpretation.
• Validity and subjectivity of test items.
• Number of items and reliability (aggregation principle and Spearman-Brown formula).

Constructing Personality Tests

• Rational method: Questions directly related to constructs.
• Factor analysis: Statistical method to group related items into factors.

Two Approaches to Assessment Development

• Rational (Theoretical): Starts with a concept, selects items, tests validity and reliability.
• Empirical (Data-Based): Empirically driven from many items, statistical methods for item selection.

Methods of Objective Test Construction:

• Rational: Items identical meaning for test-taker and creator, self-assessment capability, accurate reporting, validating items. Most common.
• Factor analytic: Create long list of objective items, administer to many subjects, analyze items that cluster together (factors), name and describe the factor.
• Empirical: Generate items, divide subjects into groups, administer test, compare answers from the different groups, cross validation.

When Are Different Methods Used?

• Rational approach used in theory building, empirical approach used in practical needs such as vocational interests/occupational assessment.

Methods of Objective Test Construction (combined methods)

• Generate items using the rational method, analyze responses with factor analysis, and correlate factors with independent criteria.

Projective Assessments

• Person responds to vague stimuli, interpreting how they relate to personality.
• Everyone expected to have different interpretations.
• Assessment is now moving away from norm-referenced tests.
• Structured interviews rather than "tests" are encouraged.

Projective Assessments: Types

• TAT (Thematic Apperception Test) —Story creation from pictures.
• Draw-A-Person test: Drawing reveals aspects of personality.
• Rorschach inkblot test — Reactions to inkblots reveal unconscious mind.

Projective Methods

• Often used by clinical psychologists linking to Freud's ideas.
• B-data.
• Word association/Sentence completion tests- used in projective methods.

Advantages of Projective Methods

• Helpful in breaking the ice (initial contact with subject)
• Skilled clinicians can sometimes extract nuanced information not found in controlled research.
• Limited support for validity of the tests, notably the Rorschach test.

Disadvantages of Projective Methods

• Validity evidence is scarce.
• Expensive and time-consuming to administer.
• Psycholgists cannot be sure about the meaning of responses.
• Other, less expensive tests might give more useful results, and these are better used.
• Inappropiate use and practice.

Projective Tests - Critique

• "Nobody agrees..." — No objective scoring, meaning dependent on responders.

Objective vs. Projective Tests - Validity/Reliability

• Which is more valid and reliable: Objective vs Projective?

Modern Personality Tests?

• Combination of data and methods, possibly including data from social media.

Facebook Likes and Personality

• Facebook likes (average 170 per person) analyzed to predict personality traits such as social openness and extraversion but also intelligence, race, religion, sexual orientation and political beliefs.
• Demographic information also collected, such as age, gender, relationship status, political views and more.

Personality Tests: Examples

This quiz covers key concepts in psychology regarding the limitations of correlational studies, the importance of experimental research, and the different types of validity in assessments. Test your understanding of how various research methods contribute to psychological evaluations and the interpretation of personality assessments.