Week 12 -The Sense of Embodiment in Virtual Reality - Copy PDF

Human Machine Interaction II: Virtual Embodiment in Social Interaction Cassandra Crone 26 October 2023 PSYU3333 (Week 12) What’s on? ´ Implicit/Unconscious Bias ´ A review ´ Perspective Taking ´ Embodiment ´ Some research… Unconscious Bias ´ What’s involved? ´ Attitudes ´ Stereotypes ´ Learning...

Human Machine Interaction II: Virtual Embodiment in Social Interaction Cassandra Crone 26 October 2023 PSYU3333 (Week 12) What’s on? ´ Implicit/Unconscious Bias ´ A review ´ Perspective Taking ´ Embodiment ´ Some research… Unconscious Bias ´ What’s involved? ´ Attitudes ´ Stereotypes ´ Learning ´ Socialisation ´ cultural & communal norms ´ Systemic Influences: education, law, workplace, media, religion, architecture ´ Recall: “Attitudes are formed through repeated association of concepts” Implicit = Unconscious Which of these are based in physical appearance characteristics? ´ Race ´ Age ´ Physical Disability ´ and sometimes mental impairment ´ Class ´ clothing brands, plastic surgery, homelessness ´ Gender ´ and sometimes, sexual orientation and religion ´ Other observable factors include: ´ Tone of voice, accent, gesture/posture/eye contact They way someone looks impacts how they are treated in any social setting or social interaction Take for example the Halo Effect aka ‘attractive people get more benefits’ Remembering that attractiveness is based on white, western, patriarchal standards Implicit bias impacts perception, decision making, and behaviour The results… ´ Prejudice ´ Discrimination ´ Microaggressions ´ Higher conviction rates ´ Reduced access to ´ government benefits ´ healthcare ´ education ´ Less likely to complete university courses ´ Less likely to be hired and/or promoted Bias in the workplace (as a model) But what about D&I policies and training? ´ Success of interventions and diversity training is transient (Bezrukova et al., 2016) (acute then disappears) ´ Limited consultation with marginalised people in these endeavours ´ “Nothing about us without us” ´ These processes do not secure equitable hiring practices (Diekman et al., 2015; Isaac et al., 2009) ´ Tokenism ´ Does not change workplace culture ´ Hiring personnel rarely receive ongoing training or feedback on the accuracy of their decisions (Hauenstein & McCusker, 2017; Melchers et al., 2011) ´ Implicit biases influence decision making ´ Under conditions of uncertainty/ambiguity (Barrantes & Eaton, 2018; Heilman, 2012) ´ During selection tasks (Cartwright et al., 2017) So, what can be done? ´ From Tutorial 5: ´ Introduce objectivity (e.g., marking criteria, hiring criteria) & accountability ´ Proactively identify ambiguity, inclusion, exclusion in policies ´ Blinded applications ´ However, this can unexpectedly reduce selection of marginalised candidates (Breda & Hillion, 2016; Hiscox et al., 2017) ´ Does not consider impact of systemic issues ´ Physically identifying factors are rarely concealed during face-to-face interactions ´ (e.g., interviews, day-to-day job responsibilities, performance reviews) ´ Such factors should not need to be concealed ´ Blinded Interviews ´ Low acceptability from both sides ´ AI Hiring ´ AI is also programmed with bias So, what can be done? Immersive perspective taking may offer a solution Perspective Taking: 1) the ability to look beyond your own point of view, so that you can consider how someone else may think or feel about something 2) the act of perceiving a situation or understanding a concept from an alternative point of view, such as that of another individual 3) put yourself in someone else’s shoes To do this successfully, you must have some understanding of others' thoughts, feelings, motivations, and intentions Thinking back to our workplace model ´ Yes, many D&I trainings and interventions include perspective taking ´ This usually involves imagining what it might be like to be someone else ´ Now consider the issue of implicit/unconscious bias To do this successfully, you must have some understanding of others' thoughts, feelings, motivations, and intentions ´ Now consider again, the role of implicit bias ´ Limitations… ´ How thoroughly can one really imagine what it’s like to be Black, transgender, disabled, etc? ´ How can we control for bias in someone’s imagination? So, what can be done? Immersive perspective taking may offer a solution Writing Video or Online Gaming Virtual Reality Who has previously experienced virtual reality? Just once or twice Moderate (e.g., a few times a year) Frequent (e.g., once a month+) Hold up…I thought we were talking about bias, perspective taking, and the workplace? Correct…let’s bring it all together Virtual Reality Perspective Taking (VRPT) & Embodiment What is embodiment? ´ Embodiment has many definitions, depending on context ´ In psychology, embodiment generally refers to a sense of something in the body ´ Emotion ´ Sensation ´ Reactions ´ Stress ´ In virtual reality, however, embodiment is a perceptual illusion! Embodiment – a definition Embodiment refers to a perceptual illusion of ownership over a virtual body (or avatar) Perceptual Illusion: in this case we have visual input from virtual reality that replaces visual input from the real world (If you’ve ever been in virtual reality, you will have experienced the somewhat strange sensation of returning to the real world) Avatars! Becoming an Avatar Let’s use our imagination Three Key Components • Body Ownership - user’s perception of owning the virtual body and perceiving the virtual avatar as the source of any bodily sensations • Agency - the conscious intention and performance of actions, including the sense of controlling one’s body movements • Location - of the self-avatar, relative to the user’s own body; the virtual avatar must be co-located with the participant The less embodied someone feels in VR, the less effective the VR experience will be Why do we care about embodiment? ´ Proteus Effect ´ People change their behaviour in line with their character ´ People change their behaviour according to stereotypes Using Embodiment to Reduce Social Bias Virtual Reality Perspective Taking (VRPT) Proteus Effect: Behaviour is known to change to be consistent with virtual representations of oneself Embodiment studies show reductions in bias for age, race, socioeconomic status, and binary gender (Banakou et al., 2018; Crone & Kallen, 2022; Hasler et al., 2017; Herrera et al., 2018; Oh et al., 2016; Peck et al., 2013; Yee & Bailenson, 2006) Greater empathy and prosocial behaviour following virtual embodiment (Crone & Kallen, 2022; Herrera et al., 2018; van Loon et al., 2018) In particular, when experiments include simulations of ecologically valid behavioural or interpersonal tasks (Groom et al., 2009; Herrera et al., 2018; van Loon et al., 2018) 30 Virtual Reality Perspective Taking (VRPT) Compared to traditional perspective taking Greater reduction in bias (Herrera et al., 2018; Oh et al., 2016) Greater increase in empathy and prosocial behaviour (Herrera et al., 2018) More direct and immersive PT experience Removes subjectivity of traditional PT exercises Allows for experimental manipulations that are not otherwise possible High experimental control AND ecological validity (Blascovich et al., 2002) Removes need for confederate in interpersonal interaction tasks Participants interact with virtual avatars the same way they would with people in the real world (Rosenthal-von der Pütten et al., 2014; Rosenthal-von der Pütten et al., 2013; von der Pütten et al., 2010) Even when avatars are not very realistic (Rossen et al., 2008) Eye-tracking and motion-tracking data collection 31 Summary Embodiment -Body Ownership -Agency -Co-location Previous Work -Race -Age -SES -Binary Gender Outcomes -Empathy -Affiliation -Prosocial Behaviour -Unconscious Bias Summary ´ Embodiment is: ´ A perceptual illusion ´ A sense of ownership about a virtual body ´ Becoming an avatar ´ First-person virtual reality experience ´ Motor control and agency ´ Influences behaviour ´ Can modulate social stereotypes So what are we actually doing with VRPT and Embodiment? ´ Applying embodiment (of a virtual avatar) to a different social identity, which typically carries social stigma (i.e., an outgroup member) ´ Forces perspective taking in a way that is not just visceral and immersive… ´ …but also, which allows exposure to more than what someone can simply imagine… ´ …based on their pre-existing knowledge and experiences (i.e., learned implicit attitudes – associations, values, beliefs) Current Research Validation Study Target Measure – Embodiment Questionnaire (Gonzalez-Franco & Peck; 2018) Purpose/Construct Sense of Ownership over a Virtual Body (aka Embodiment of a Virtual Avatar) Total Score Image 4* Subscales Body Ownership Agency Location External Appearance Underlying Theory: Embodiment is comprised of Body Ownership, Motor Control/Agency, Co-Location (of the body in space) Research Questions 1. Do specified constructs differ between invivo and VR? 2. Do scores on the target measure (and its subscales) correlate with scores on validated measures (of the same constructs)? General Procedure In-Vivo Exercises (x3) Agency Questionnaires Virtual Reality Exercises (x3) Self-Report Battery 38 Virtual Environment Virtual Avatars Exercise 1 - Kinaesthetic Sense Kinaesthetic and Visual Imagery Questionnaire (KVIQ) 7* Basic Movements Forward trunk flexion, Knee extension, Neck flexion/extension “Rate the clarity of the image” No image – Image as clear as seeing “Rate the intensity of the sensations” No sensation – As intense as executing the action Does kinaesthetic sense differ between in-vivo & VR? Exercise 2 – Proprioceptive Sense Proprioceptive Acuity Test Joint Reproduction Test Image Ipsilateral Matching 10 Trials (5 right, 5 left) Contralateral Matching 10 Trials (5 right, 5 left) Targets: 30°, 60°, 90°, 120°, 150° Randomised between right and left sides 20 Trials Total Participant is blindfolded Can we induce proprioceptive drift in VR? Exercise 3 – Embodiment Distractor Yoga/Tai Chi Image 1-Minute “Please follow along to the best of your ability” Purpose: full-body movement in VR = sense of embodiment Measures of Convergent Validity Total Score Rubber Hand Illusion Questionnaire (RHIQ) Body Ownership Subscale Scale of Body Connection (SBC) Agency Subscale Sense of Agency Rating Scale (SOARS) large posi+ve correla+on small nega)ve correla)on state agency small posi)ve correla)on Location Subscale Proprioceptive Acuity Test Performance VR only small-moderate correla-on External Appearance Subscale Scale of Body Connection (SBC) very small correla,on *Interpersonal Reactivity Index (IRI) *Questionnaire of Cognitive and Affective Empathy (QCAE) 44 Future Spoilers – a sidebar about empathy External Appearance Subscale Scale of Body Connection (SBC) *Interpersonal Reactivity Index (IRI) *Questionnaire of Cognitive and Affective Empathy (QCAE) • Interaction Quality (with the demonstrator) predicted External Appearance • But only meaningful for those with low Affective Empathy • Although gender was not a significant predictor… …you might guess that men were overall less empathetic 45 Do specified constructs differ between in-vivo and VR? KVIQ Kinaesthetic Subscale t(85) = –3.23, p < .001** Better performance in VR KVIQ Visual Subscale t(85) = 2.19, p = .031* Better performance in-vivo State Agency Involuntariness t(85) = –.0.12, p = .903 No differences between invivo & VR State Agency Effortlessness t(85) = 0.06, p = .953 No differences between invivo & VR Additionally, TRAIT Agency did not differ Proprioceptive Drift DO SPECIFIED CONSTRUCTS DIFFER BETWEEN IN-VIVO AND VR? Proprioceptive Drift was induced in VR by moving the participants arms 20cm laterally (relative to their actual arm position) Ipsilateral (Dominant) z = –4.18, p < .001** Ipsilateral No differences in perceived sense of location during the proprioceptive acuity task (Nondominant) Contralateral t(84) = 1.72, p = .090 (Dominant) Less accurate in VR, indicating some influence of drift Contralateral Greater influence on the ipsilateral trials (Nondominant) z = –4.34, p < .001** z = –1.74, p = .082 z = –2.13, p = .034* Do specified constructs differ between invivo and VR? In plain language, what does this mean? • Visually imagining the virtual body was more difficult (compared to imagining the real body) • Proprioceptive drift can be induced in VR • While VR can easily alter visual perception… • …interoception remains affected by the physical world (vestibular, kinaesthetic, proprioceptive senses) • Overall, the virtual body experience is NOT the same as the physical body experience Do scores on the target measure correlate with scores on validated measures? Mostly… …no Except: Total Score & RHIQ r = .70** Large positive correlation! We also did a PCA revealing 3 subscales: Embodiment, Disembodiment, Agency Which align with the same subscales in the RHIQ (i.e., embodiment, disembodiment) And the state agency measure 49 Do scores on the target measure correlate with scores on validated measures? Summary • Results are somewhat expected ― Cognitive theory (including the Rubber Hand Illusion) tells us that agency and location comprise body ownership ― Explicit self-report measures for location & body ownership do not exist ― The agency scale was validated for use in hypnosis research • Although scales chosen for convergent/divergent validity were related to the subscale constructs… …there was no support for a measure that nominates these three factors as valid subscales • Overall, the items should ask about body ownership, agency, and self-location, but the total score is the best measure of embodiment ― In line with a breadth of Rubber Hand Illusion research 50 Why does this matter for a class on social interaction in the modern world? • Earlier, we talked about embodiment for social identity, stigma, etc. • Embodied VR interactions are also used in clinical psych, forensic psych, and developmental psych (e.g., learning) ― See Tutorial 6 for more info • BUT… ― There have been four measurement scales created, to date ― None have been psychometrically validated ― These are being created primarily in Computer Science and Big Tech (see Tutorial 6) • Psychological scientists are using embodied approaches anyway! ― Yes, we have a foundation in RHI research ― But we are doing this either 1) with a non-validated measure or 2) simply assuming the approach works and not measuring embodiment at all What considerations might there be for serving vulnerable communities? 51 Current Research Women in STEM Gender Bias and STEM SCIENCE, TECHNOLOGY, ENGINEERING, MATH (STEM) Men comprise 80% of the STEM workforce in Australia and 70% internationally (Department of Industry, 2019; European Commission, 2019; National Science Foundation, 2020) 20.8% of women compared to 79.2% of men complete their STEM degrees (Department of Industry, 2019) Largest disparities in engineering and IT (Department of Industry, 2019) 12.4% women in engineering; earn 11% less 28% women in IT; earn 20.2% less Women frequently report experiences of gender bias in the STEM workplace (Robnett, 2016; Williams et al., 2016) Causes of Gender Disparity? SCIENCE, TECHNOLOGY, ENGINEERING, MATH (STEM) Multifaceted Issue Meta-Analysis: 470 papers between 1985 and 2018 (Alviolo et al., 2020) Contributing Factors: Individual, Family, Social, Educational, and Labour-Economic Evidence suggests gender bias favouring men, rather than disadvantaging women, is an overarching factor Causal link between gender bias and gender gaps in STEM (Moss-Racusin et al., 2018) Specifically, women’s disengagement Series of hiring studies shows preference for men over women Men are chosen more often Men consistently rated more competent Men given more benefits (Eaton et al., 2019; Milkman et al., 2015; Moss-Racusin et al., 2012; Reuben et al., 2014; Steinpress et al., 1999) Role of Embodiment SCIENCE, TECHNOLOGY, ENGINEERING, MATH (STEM) Little research on embodiment for gender 1) Greater gender bias on an IAT for males embodied as female during a movement task (Lopez et al., 2019) 2) Greater gender bias on an IAT for all participants following any gendered embodiment within a virtual office (Schulze et al., 2019) **Note reliance on IAT** **Fully immersive virtual reality** When participants are exposed to a workplace simulation that directly involves bias from a co-worker 3) Reduced gender bias on an IAT for participants embodied as women (Beltran et al., 2021) **Implements ecologically valid simulation** **Computer-based immersion** Simulated Interview Study Embodiment - female/male interviewer Participants rate each candidate on Competence, Hireability, Likeability, Empathy Ecologically valid simulation of everyday social & professional interaction Conducted online AND in fully immersive virtual reality Participant conducts two interviews – one male candidate, one female candidate (randomised presentation) Participant reads every second interview question, then observes the candidate responses and co-interviewer 58 Online vs VRPT Computer-Based Embodiment VR Embodiment ´ First-person perspective ´ First-person perspective ´ Two-dimensional immersion (size of computer screen) ´ Three-dimensional immersion (360-degree surround) ´ Uses mouse/trackpad to view the virtual environment ´ Uses eye-gaze and head movement to view the virtual environment AND own body ´ Full-body motor control ´ Co-location of body in space Role of Embodiment SCIENCE, TECHNOLOGY, ENGINEERING, MATH (STEM) Computer-Based Embodiment 1) Women are more empathetic than men (Not a finding specific to this study but rather, a broad effect) 2) Non-significant trend: men in a male avatar prefer the male candidate men in a female avatar prefer the female candidate 3) Higher empathy predicts better candidate ratings VRPT 1) Replicated effects for empathy 2) Replicated trend for men, but also observed the same trend for women 3) Empathy mediates the relationship between embodiment and candidate ratings **VRPT is more effective than computer-based embodiment** **Empathy may be an underlying factor in VRPT outcomes** Role of Embodiment SCIENCE, TECHNOLOGY, ENGINEERING, MATH (STEM) Two Key Conclusions: 1) Participants identify with the avatar gender they’ve been assigned to embody Broader application – participants come to perceive themselves (temporarily) as members of the out-group 2) If women are already more empathetic than men… …VRPT interventions targeting bias against women in male dominated industries may be most effective for men Broader application – such interventions may be best targeted at individuals in positions of power and privilege (Consider, in particular, evidence that shows a negative response to D&I processes from dominant group members – e.g., reverse racism, perceived unfair advantage) Current Research VRPT for Gender Diversity Study 1 DYADIC INTERACTION You will be playing the role of Alex today. You graduated from university and then worked in the information technology field as a programmer for five years before seeking new employment. You identify as a transgender man and have a masculine appearance. Your preferred pronouns are he/him. According to your previous supervisor at Research Technology Labs, you have demonstrated a great degree of technical knowledge but have been passed over for regular promotions, mostly due to your lack of managerial skills. Adapted from Dray et al. (2020) Virtual Environment {QUESTION SET A} 1. Please share with me what the terms diversity, equity, and inclusion mean to you? 2. Do you think that workplace policies should include statements on allyship, such as how to respond to sexist, racist, or homophobic behaviours? Why or why not? 3. In your opinion, should sharing gender pronouns be incorporated in workplace settings? Why or why not? OBJECT PASSING TASK GENDER IDENTITY DISCLOSURE {QUESTION SET B} 1. In your opinion, what is the most challenging aspect of working in a diverse, equitable, and/or inclusive environment? 2. Allyship refers to members of privileged in-groups making efforts to advance the interests of marginalised groups. Do you think allyship is important in the workplace? Why or why not? 3. Do you think it is important to understand gender diversity? Why or why not? OBJECT PASSING TASK Object Passing Neutral Trials Critical Trials Front/Back Right/Left EMBODIMENT (EQ) EMOTIONAL STATE (VAS) INTERPERSONAL INTERACTION RATINGS EXPLICIT GENDER BIAS (ASI) SOCIAL DESIRABILITY (SDS) TRAIT EMPATHY (QCAE) Preliminary Results ´ All participants show more empathy at post-disclosure ´ But only trans-embodied participants ´ report similarity & closeness ´ are more coordinated with the confederate ´ Possible mediation ´ Embodiment predicts empathy, which in turn predicts closeness (but not similarity) ´ Men are less empathetic and report more explicit bias (overall; across three measures) ´ Women are less ego-centric (at post-disclosure) ´ Possible effect for trans-embodied men, but only p = .057 ´ Results highlight the role of perspective taking via effects for transembodied participants (compared to cis) ´ Possible influence of intergroup contact (but no control group) Study 2 COOPERATIVE/COMPETITIVE PERFORMANCE AND DECISION MAKING AVATAR CUSTOMIZATION TRANSGENDER EMBODIMENT RPG TEAM BUILDING EVENT W/ PREPROGRAMMED AVATAR Performance Tasks Cooperative Competitive Distractor The Great Bathroom Debate Preliminary Results ´ Empathy increases throughout the interaction ´ Only empathy at final timepoint predicts reduction in heteronormative attitudes and beliefs (i.e., explicit bias) ´ No gender-empathy effect here! ´ Men are less empathetic, but this effect is attenuated when accounting for baseline levels of trait empathy (i.e., affective/cognitive) ´ Meaning, all participants benefit from increased empathy & reduced explicit bias ´ Mediation also found ´ Like previous study, embodiment predicts empathy, which in turn predicts lower explicit bias Preliminary Results ´ We also conducted a DFA ´ See Lecture 11 fractals & Tutorial 6 for more info ´ All participants exhibit less determinism (i.e., more pink noise) during the conflict scenarios ´ Overall, men exhibit more flexible movement dynamics (i.e., more pink, less deterministic) ´ Not yet sure about the role of contact here, but there are some indications ´ Complexity of results beyond the scope of this lecture Preliminary Results ´ However, strong evidence for perspective taking ´ And again, we see that this type of intervention might work best for men ´ That is, if we assume less determinism/more pink noise is good… …and that it indicates an opportunity to introduce a perturbation (i.e., a change in the system)… …we can think about what other perturbations/ changes/ experimental constraints might be effective ´ If the bathroom conflict is effective and represents a real-world experience for TGD people, what else might work? Let’s wrap it up! ´ The purpose of this research is to induce perspective taking for marginalised identities via virtual embodiment to modulate implicit bias ´ Practical application/Broad scope: implement VRPT as an intervention among decision makers, particularly those with privilege ´ e.g., CEOs, management, interviewers, recruiters, policy makers, barristers, judges, GPs, medical practitioners, psychologists, and more ´ Long-term aim: systemic change by targeting the forms of bias that contribute to underrepresentation Invisibility Fragmentation/Isolation (othering) Stereotyping Linguistic Bias Imbalance/Selectivity Cosmetic Bias/Tokenism Unreality/Ignorance See you in class! [email protected]