03-VR-Human_Factors.pdf

Virtual Reality Prof. Dr. Oliver Staadt Chair in Visual Computing 1 3. Human Factors 2 Why are Human Factors Important? Perceptual and motor capabilities determine the range within which input and output technologies should perform Human senses are informationa...

Virtual Reality Prof. Dr. Oliver Staadt Chair in Visual Computing 1 3. Human Factors 2 Why are Human Factors Important? Perceptual and motor capabilities determine the range within which input and output technologies should perform Human senses are informational channels with different possible rates of information transfer How to best present and display information to facilitate understanding Keep in mind intersensory interactions, adaptation, simulator sickness (cybersickness) Predict the VR system’s affective in uence on how it addresses the user’s capabilities 3 fl  Important Human Factors Visual perception Auditory perception Haptic perception Olfactory perception Vestibular sense Implications for virtual environments Virtual presence 4 Human Information Processing (According to Card et al. 1986a) The Human Eye 6  Human Vision Human eye responds to a narrow band of electromagnetic radiation 400nm to 700nm, overall peak at 559nm Well-matched to spectral emission of sunlight 7  Human Vision (cont’d) Responds to single photon up to uxes one trillion times stronger At any given time, response is in a roughly 2 log unit range; Adaptation is required Eye is in constant motion Saccades at about 4 Hz Stabilized images disappear Saccadic suppression renders these motions largely unnoticed Convergence 8 fl  Rods and Cones Cones densely packed in the foveal region High spatial acuity in fovea Three kinds of cones, sensitive to different wavelengths “long”: 575nm “middle”: 535nm “short”: 445nm Trichromatic vision No rods in the fovea 9  Brightness and Contrast No display can match full range of perceptible brightness levels Few photons to brightness levels 12 orders of magnitude higher Eye must adapt over brightness continuum Target/background contrast requirement varies with background illumination For bright backgrounds, target/background contrast roughly constant As background dims, target/background contrast must increase Brightness in uences acuity and color 10 fl Photopic and Scotopic Vision 11  Color Perception Scotopic system responds to luminance not color, so night scenes require only limited use of color Daylight, photopic scenes require full color Cone receptors produce sensation of color Retinal periphery tiled with less-dense, panchromatic, averaging rods Rods are very sensitive to motion, low light levels Trichromatic color generation Choice of primaries yields color gamut Due to tristimulus color crosstalk, no trichromatic gamut can generate full range of perceptible colors. 12  Field of View (FoV) Angle subtended by viewing surface from viewer location FoV of single eye: 150° The horizontal elds of both eyes overlap in the center (binocular eld of view) Area of overlap: 120° with 30-35° monocular vision on each side Combined horizontal eld of view: 180-200° Vertical eld of view is 120-135° for both eyes “Typical” desktop display: 40°x32° @ 46cm Use eye, head, and body movements to keep viewed objects within the foveal region (region of maximal resolution) 13 fi fi fi fi  Spatial Resolution Number of pixels, pixel pitch, angular resolution Foveal FoV subtends only 1-2° of visual eld Visual saccades give illusion of visual eld at foveal resolution Human visual acuity is 0.5-1 min of arc 1 arc min resolution allows you to distinguish detail of 1 mm @ 3 m To match this, requires “typical” desktop display of 4800x3840 (18.4 megapixels) “4K” display standards Ultra High De nition TV (UHD): 3840 × 2160 (8.2 megapixels) Full Ultra HD (FUHD): 7680 × 4320 (33.2 megapixels) 14 fi fi fi  Spatial Resolution (cont’d) Consider a visual display screen that is 20’’ wide (horizontal, not diagonal), positioned 24’’ from the viewer. How many pixels across one scan line would it take to match human visual acuity? Human visual acuity: 0.5 arc min (120 cycles/deg.) to 1 arc min (60 cycles/deg.) Horizontally subtended angle α = 2 atan(20/2)/24) = 45.24° 120α = 10,858 pixels and 60α = 5429 pixels 15  Refresh and Update Rates Refresh rate is frequency at which display redraws imagery Update rate Frequency at which new imagery is computed Rule of thumb: 10-15 Hz required for “smooth” animation Critical fusion frequency (CFF) Depends on display brightness, ambient lighting, position in visual eld Typical room: 60 Hz NTSC: 30 Hz (2x60 Hz interlaced elds), PAL: 25 Hz (2x50 Hz interlaced elds) Interlaced stereo 16 fi fi fi  Perception of Depth Monocular (pictorial) depth cues Only one eye Insensitive to focusing Oculomotor cues Binocular disparity and stereopsis Motion Cues 17  Monocular Depth Cues Linear perspective Aerial perspective (atmospheric attenuation) Interposition (occlusion) Closer objects have higher Textural gradients color saturation Proximity-luminance covariance Shadows Brighter areas are perceived Size and past experience closer E.g., two objects of known size Height in the visual eld (relative to horizon) 18 fi Linear Perspective “Rue de Paris, temps de pluie” (Gustave Caillebotte) 19  Interposition „Madonna del Magni cat“ (Sandro Botticelli) 20 fi Textural Gradients “Rue de Paris, temps de pluie” (Gustave Caillebotte) 21 Aerial Perspective 22 Shadows 23 Size and Past Experience „Scolls“ (Gustave Caillebotte) 24 Height in the Visual Field „Küste bei Portrieux“ (Eugene Boudin) 25  Oculomotor Cues Accommodation Physical stretching and relaxing of the lens Parallel rays entering the relaxed eye will focus on the retina Relaxed eye has a depth of eld of 6 m to in nity To focus objects within 6 m it is necessary to alter the optical system of the eye Vergence Rotation of the eyes (convergence: inward rotation corresponding to viewing closer object) Muscular feedback in converging and focusing the eyes is cue to the depth of viewed object Relatively weak, but coupled depth cues! Have to be decoupled in HMDs 26 fi fi Binocular Disparity and Stereopsis Stereo pairs – Each eye sees different image generated at different viewpoints Binocular disparity is greatest for close objects and least for distant objects Binocular rivalry: Visual system often suppresses one of the images (dominant eye) Avoid binocular rivalry in synthetic scenes! Stereopsis is not always necessary for depth perception (other cues!) Stereopsis effects operate over a limited range (9 m for peripheral viewing of static scenes) Stereoscopic cues operate out to 500 m in the fovea 27 Stereoscopic Vision (Virtual and Augmented Reality (VR/AR), Doerner et al., 2022)  Additional Cues with Motion Relative motion Closer objects move more in the visual eld Distant objects move less Motion parallax Very important for depth perception at extended ranges Rotation of objects 29 fi 30 30 Relative Importance of Depth Cues Empirical studies and models Weighted additive model (Wickens, Todd, and Seidler, 1989) Stereopsis, interposition, and motion parallax are most dominant Texture gradient, proximity-luminance covariance, and perspective are not as strong, but important in combination with strong cues Past experience and highlighting are quite weak 31  Implications for Virtual Environments Speci cation of an “ideal” HMD: Horizontal FOV of 180°; each eye having a 150° with binocular overlap of 120° Vertical FOV of 135° Display at resolution of 1 arc min (limiting resolution of the visual system) Display the full range of discernible colors No distortions or optical aberrations (exact point-to-point correspondence between both eyes) Lightweight and comfortable to use Upper limit, but technically not feasible (today) 32 fi  Auditory Perception Air vibrations (rapid changes in air pressure) are converted to mechanical vibrations in the middle ear Acoustical characteristics of sound Amplitude: Magnitude of the pressure variation Frequency: Pressure variation rate Phase Acoustic re ex: Adaptation to high-intensity sounds; temporarily reduced auditory sensitivity Acoustic stimuli necessarily have a temporal component Constant sounds drop out of conscious awareness Sounds are perceived from sources in all directions 33 fl  Auditory Localization Different factors in uence our ability to perceive the location of sound sources Interaural level difference: Difference in volume of sound reaching ears Interaural time difference: Time difference of sound reaching ears caused by “slow” speed of sound Pinna ltering and HRTFs Motion cues Doppler effect: Frequency shift resulting from relative motion between sound source and observer Changing volume: Sound is perceived to be approaching when volume gradually increases (and vice versa) 34 fi fl Head-Related Transfer Functions Sound pressure that arbitrary source x(t) produces at ear drum Head-Related Impulse Response (HRIR): Impulse response h(t) from source to ear drum Head-Related Transfer Function (HRTF): Fourier transform of HRIR HRTF captures all of the physical cues to source localization Based on HRTF for left and right ear, binaural signals can be synthesized from monaural source HRTFs can be measured 35 Implications for Virtual Environments Auditory display/soni cation largely underused in virtual environments No direct contact to observer required (as opposed to force/tactile perception) Sound perceived from any direction (not restricted to visual eld) The ears can guide the eyes to point of interest outside of visual eld 36 fi fi fi  Haptic Perception Tactile perception Product of receptors under skin surface Different receptors for skin covered/not covered by hair Perception of thermal, and mechanical stimuli Force perception Receptors in muscles and joints Perception of movement, position, and torque of limbs and other body parts Varying joint angels and muscular length 37 Haption Virtuose 6D Implications for Virtual Environments Tactile and force feedback are important for certain tasks/ applications So far more research on value of force feedback than on tactile feedback Tactile displays mainly used in non-VE contexts (tactile vision substitution) Force feedback systems used in teleoperation systems Surgery simulation Haptic perception is one of our least understood senses 39  Olfactory Perception Perception of odor using the sense of smell Not very well studied in the eld of human-computer interaction/virtual environments Promise of future olfactory displays? Online performance tasks Teleoperation Smell associated with state of equipment (e.g., overheating) Of ine training tasks Veridicality Fire ghter training 40 fl fi fi Olfactory Display 41 42 43 43  Vestibular Sense Senses movement of the head Rotation Linear acceleration Gravity Vestibular illusions due to vestibular adaptation Inversion: Being upside-down in zero gravity Coriolis illusion: As body rotates around a vertical axis, tilting head forward produces illusion of falling to one side Post-rotatory sensations (graveyard spin): after prolonged rotation, deceleration produces illusion of turning/falling in opposite direction Require motion platforms in VE Vehicle simulators Con icts between vestibular sense and other sensory channels can cause cybersickness 44 fl 45 45  Virtual Presence Degrees of presence “part of a synthetic experience”: Aware of outside world “immersed in the experience”: Feel part of the actual environment No objective measure for presence Visual cue can improve immersion ( rst-person views) Breakdown of presence sensation Person feels tired HMD to heavy Unnatural movement in VE 46 fi  Important Factors of Presence Seeing parts of one’s own body Virtual body parts lower feeling of presence See-through HMD Surround-screen VE (CAVE) High resolution and large FoV: User should be able to move eyes to see objects outside of peripheral vision Familiarity of VE or scene 47  Physiological Measures of Presence UNC’s Pit Room Test-bed for measuring the effects of a stress-inducing VE: Border around a 20’ cutout in the oor – subjects try to walk across Issues what contributes to the experience - have you measured what you think you have measured? how much impact does the inability to see your own body have? 48 fl Next time: Tracking 49