Cognition Module 2 Prof. Serino PDF

Cognition Module 2 Prof. Serino Created @October 3, 2024 Tags Human-System Interaction The fields that work together to create intuitive, efficient, and enjoyable user experiences in technology...

Cognition Module 2 Prof. Serino Created @October 3, 2024 Tags Human-System Interaction The fields that work together to create intuitive, efficient, and enjoyable user experiences in technology Cognitive psychology Studies mental processes to understand how users think and process information crucial for understanding human mental models of AI and designing an intuitive AI interface Cognitive Ergonomics Applies cognitive psychology in “real-life’ to optimize human well-being and system performance essential for optimizing cognitive load in complex AI interactions and ensuring user well-being Human-computer interaction Focuses on the design and use of computer technology, centered on the interfaces between people and computers, optimizes performance Provides foundational principles for designing effective human- AI dialogs and interactions Interaction design Creates engaging interfaces with well-developed behaviors, focusing on user goals and experience Critical for creating engaging and meaningful interfaces for AI systems that adapt to user needs 1. Cognitive psychology Cognition Module 2 Prof. Serino 1  Cognitive psychology is a disciplined concerned with internal mental processes, such as perception, attention, language, memory, learning, emotional processes, decision-making, problem-solving and creativity how do people perceive various shapes; why do they remember some facts and forget others; how we experience emotions 2. Cognitive ergonomics concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other system elements traditionally applied to optimize performance in the workspace relevant topics: mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress, and training 3. Human-computer interaction draws on the fields of computer science, psychology, cognitive science and organizational and social sciences in order to understand how people use and experience interactive technology 4. Interaction design Interaction design is the umbrella form for interface design, software design, user-centered design, product design, web design, experience design (UX) It is fundamental to all disciplines, fields, and approaches concerned with researching and designing intelligent systems for people Cognition Module 2 Prof. Serino 2  All these fields converge to address the unique challenges of Human-AI Interaction, ensuring systems that are not just intelligent, but also usable, ethical, and human-centered Cognitive processes Attention Perception Memory Learning Reading, speaking and listening Problem- solving, planning, reasoning and decision-making Perception how we interpret sensory stimulation Design should be easily perceivable, text should be legible, icons should be easy to distinguish and read (accessibility) There are issues connected to accessibility, as people perceive things differently Cognition Module 2 Prof. Serino 3  Weller’s experiment (ебаный текст сначала массой а потом с рамочками,и с рамочками легче) found that people took less time to locate items for information that was grouped Some argue that too much white space on web pages is detrimental to the search process makes it harder to find information The Gestalt theory of perception Popular theory of perception developed in the 1920’s in Germany According to Gestalt theory, we perceive objects according to some shapes, patterns, well-organized patterns, rather than separate components Still popular in design Proximity refers to the visual tendency to group things together that are close to each other. The group will generally be perceived as a single unit. AI Interface example : Grouping related AI-generated suggestions or options in chatbots or recommendation systems Similarity refers to the visual tendency to similar elements regardless of their proximity to each other. They can be grouped by color, shape or size AI Interface example: Using consistent visual styles to differentiate AI- generated content from human input in collaborative tools Continuity refers to the visual tendency to create continuous figures. The mind continues a pattern or form beyond its given information. Continuity occurs mostly in the perception of lines and refers to the tendency to “carry the line forward” AI Interface Example: Designing smooth transition in AI-driven data visualizations or predictive text interfaces Closure is the tendency to perceive incomplete objects as complete and to close or fill gaps and to perceive asymmetric stimuli as symmetric Cognition Module 2 Prof. Serino 4  AI Interface example: Displaying partial results while AI is processing, indicating progress and expected outcomes Applying these principles can enhance user understanding, improve interaction, and create more intuitive AI interfaces Our perception of the world around us is not a true depiction of what is actually there Three factors bias our expectations: the past: our experience the present: our current context the future: our goals Bias by experience: People expect and desire consistency, if you have two buttons on many pages of a website in a certain place, and then switch them on the last page, most people wouldn’t notice the switch immediately Context is important Bias by goals: Our goals influence where we look. The sensitize us to what we see Perception and design implications: Avoid ambiguity Ensure all users interpret your design in the same way. Use clear icons and symbols Be Consistent Place controls in consistent locations. Use consistent shapes, colors, fonts, etc. across the interface Understand goals Recognize that different users may have different goals. Ensure you interface directs users to the right goal for their needs Cognition Module 2 Prof. Serino 5  Attention Attention is the ability to detect and respond to stimuli. It refers to how we actively process specific information present in our environment. Attention enables us to be selective in terms of the mass of competing stimuli, but limits our ability to keep track of all events When the visual information, like text, is bunched together, it makes it harder to find exactly what you’re looking for. If you group it into categories visually, the search is easier and faster Attention is a limited resource. Multitasking can cause people to lose their train of thought, make errors, and need to start over. Ophir et al. (2009) compared heavy vs. light multitaskers: Heavy multitaskers are more prone to being distracted than those who infrequently multitask Heavy multitaskers are easily distracted and find it difficult to filter irrelevant information Lotteridge et al. (2015) conducted another study involving writing an essay under two conditions: relevant or irrelevant information Heavy multitaskers were easily distracted but able to put this to good use if the distracting sources were relevant to the task at hand Irrelevant information was found to impact task performance negatively In sum The main reason why multitasking is thought to be detrimental to human performance is that it overloads people’s capacity to focus Apps have been designed to help people get back on track or avoid getting distracted. Many are designed to block or limit distracting sources, such as notifications, newsfeeds and social media. Cognition Module 2 Prof. Serino 6  An example is FocusMe which claims to “wall off online temptation”. It can help people improve willpower so they can develop better digital habits. Driving and phones: Driving is very demanding, and drivers are prone to being distracted and causing accidents. Drivers’ reaction times are longer to external events when talking on the phone in a car. Drivers often try to imagine what the other person’s face is like - the person whom they are speaking to. The visual imagery competes for the processing resources also needed to enable the driver to notice and react to what is in front of them on the road The same goes for a hands-free phone or talking with the front-seat passenger. However, the driver and front-seat passenger can observe jointly what is happening in front of the road and will moderate or cease their conversation in order to switch their full attention to a potential or actual hazard Attention and AI-interface design implications: Contextual Salience (Prominence) Make information salient when it needs to be attended to at a given task stage. Use color, ordering, spacing, underlining, sequencing, and animation AI Example: Highlight AI-generated insights or recommendations relevant to the current step in a data analysis workflow Avoid Clutter Avoid cluttering visual interfaces with too much information. Present only necessary data at each stage. AI Example: In an AI-powered dashboard, use progressive disclosure to reveal detailed AI analyses only when requested Effective Switching Design different ways to support effective switching and returning to an interface Cognition Module 2 Prof. Serino 7  AI Example: In a conversational AI, provide clear visual cues for switching between topics and returning to previous conversations Memory Memory is the ability to retain information over time The three stages of memory are: Encoding - consists of placing a piece of information in memory Storage - when the information is stored in memory Retrieval - occurs when the information is recovered from storage There are three different memory stores: Sensory store Short-term memory Long-term memory According to the Atkinson-Shiffrin Theory (1968) sensory inputs first go into sensory memory. Then, with attention the input is transferred into short-term memory and with rehearsal it ends up being stored in long-term memory Sensory memory is a large and transient memory store, meaning that it can hold large amounts of information but for a limited amount of time. The small portion of memory that gets attention is transferred ount of the sensory store to short-term memory With short-term memory, you are conscious of the information and it is readily accessible, but subject to decay (over a period of around 20 seconds The information can be prevented from decay if it is rehearsed, that is repeated over and over Information that is rehearsed or that undergoes other forms of processing (elaboration), is transferred to long-term memory Long-term memory is an unlimited store of information generally available to us According to George Miller (1956), people can generally retain up to 5-9 items in perfect order. This principle has been applied in interaction design when considering how many options to display. Cognition Module 2 Prof. Serino 8  Depending on a task and available screen estate, this principle can be applied to HCI. However, People can scan bullets, tabs, and menu items for the one they want They don’t have to recall them from memory, having only briefly heard or seen them So you can have more than nine items in the interface. For instance, when it comes to history lists of websites visited Sometimes a small number of items is good, like for a smart watch screen where space is limited Post-completion errors Byrne and Bovair (1997) studied the cognitive errors associated with using ATNs to make cash withdrawals from bank accounts. Post-completion errors occurred when the user completed their task of withdrawing cash but failed to remove their bank card from the ATM A common post-completion error is when people write an email with a line “see the attachment file” but then forget to attach the said file (for that GMail asks if you meant to attach files when you write in your message that the message was attached) Post-completion errors tend to happen when users have an additional step to perform after their primary task The performance of many everyday cognitive tasks requires short-term retention and simultaneous manipulation of material. Psychologists use the term “working memory” to refer to the system responsible for the temporary storage and concurrent processing of information Baddeley’s model of working memory An interactive system that links incoming perceptual information to long-term memory Cognition Module 2 Prof. Serino 9  Central executive: Coordinates the activity of the phonological loop and the visuospatial sketchpad Example of processes: maintaining and updating task goals monitoring and correcting errors inhibiting irrelevant information retrieving information from long-term memory Visual spatial sketchpad: Dedicated to the storage and maintenance of visuospatial information It retains information coded in a visuospatial form It provides a “virtual” environment for physical simulation, calculation, visualization Phonological loop: Responsible for the short-term retention of material coded in a phonological format Episodic buffer: A limited capacity storage system responsible for integrating information from several sources (e.g. binding verbal and visuospatial information) to create a unified memory (episode) It links working memory to long-term memory and perception Cognition Module 2 Prof. Serino 10  Working memory helps us with different operations in everyday life and it has a role in many cognitive functions: comprehension, learning, reasoning, problem- solving, and reading to perform… But it has a small capacity when dealing with new information, so it can be easily overloaded Performance depends on: an individual’s working memory capacity the cognitive load made by a task Cash withdrawals in ATMs: Byrne and Bovair found that the post-completion error only occurred when the load on working memory was high In these circumstances, the presence of an additional step overloads limited working memory capacity It is worth noting that the occurrence of post-completion errors led to the redesign of ATMs, with the result that they now only dispense cash after first returning the card to the user Working memory and design implications Mitigation strategies: Automate final steps when possible Provide clear reminders Use forced functions to prevent task completion without the final step For example: automatically saving AI-generated results, showing a prominent “Save and Exit” button, or preventing closure of the analysis window without saving Long-term memory: Recognition vs. Recall Recall is a mental process by which we bring to our present consciousness the past stimuli without presenting the physical stimuli Cognition Module 2 Prof. Serino 11  Recognition is a mental process by which we recognize previously seen stimuli from new ones Implications for design: See and choose is easier than recall and type. Show users their options and let them choose among them, rather than force users to recall their options and tell the system what they want Use pictures where possible to convey function Make authentication information easy to recall: we can give users the freedom to create passwords they can remember and challenge questions/hints for which they can easily remember the correct response authentication methods that do not rely on users to recall the authentication data would seem to be a solution External cognition The extended mind (Clark and Chalmers, 1998) was a philosophical account of how artifacts and objects (including digital technologies) can be considered an extension of our minds Knowledge and cognition are not confined to the brain - other objects within the environment are part of an extended mind Theory of external cognition by Scaife and Rogers, 1996, was concerned with designing new technologies and interfaces that would empower our cognitive abilities while reducing cognitive effort Cognitive offloading Cognitive offloading is a common strategy to prevent forgetting and to avoid the effort of remembering Examples include the use of diaries, reminders, calendars, notes, shopping lists, to-do lists, post-its, piles, marked emails External representations: Remind us that we need to do something (for example, to buy something for mother’s day) Cognition Module 2 Prof. Serino 12  Remind us of what to do (for instance, buy a card) Remind us of when to so something (for example, send a card by a certain date) An obvious area where technology can be designed to help remind us Cognitive frameworks These are used to explain and predict user behavior when interacting with a technology Based on theories of behavior Focus is on mental processes that Take place Most well known are: Mental models Gulfs of execution and evaluation Embodied cognition and embodies interactions Mental Models People develop an understanding of a system through learning about using it. This knowledge is sometimes described as a mental model: How to use a system (what to do next) What to do with unfamiliar systems or unexpected situations (how the system works) People make inferences using mental models of how to carry out tasks Craik (1943) described mental models as internal constructions of some aspect of the external world enabling predictions to be made. The process involves unconscious and conscious processes (imagery and analogies are activated) People use their mental models to: Reason about a system Figure out how to interact with a system, how it works Figure out what to do when things go wrong Cognition Module 2 Prof. Serino 13  But mental models are incomplete, constantly evolving, and not accurate at representing Users “see” the system through mental models. They rely on mental models during usage. Mental models can support users’ interaction There are various forms of mental models. The designers also have mental models, which do not necessarily match the users’ mental models How can interfaces be designed to help people build better mental models? Visibility - The more visible a function is, the more the users will notice and use it make relevant parts visible make what has to be done obvious Feedback - Sending information back to the user about what has been done Includes sound, highlighting, animation and combinations of these Constraints - Restricting the possible actions that can be performed Helps prevent users from selecting incorrect options Three main types (Norman, 1999): Physical: the object’s size or shape prevents or stops certain actions from occurring Cultural: localized signals Logical: relies on common sense (e.g. gravity) Mapping: Relationship between controls and their movements and the results in the world The control buttons are mapped better onto the sequence of actions of fast rewind, rewind, play and fast forward Cognition Module 2 Prof. Serino 14  Consistency: Designing interfaces to have similar operations and use similar elements for similar tasks, for example, always use ctrl key plus the first initial of the command for an operation - ctrl + C Main benefit is that consistent interfaces are easier to learn and use Internal consistency refers to designing operations to behave the same within an application Difficult to achieve with complex interfaces External Consistency refers to designing operations, interfaces, etc., to be the same across applications and devices Very rarely the case, based on different designer’s preference Most successful in product families (e.g. MS Office) Affordances: Refers to a functional attribute of an object that allows people to know how to use it (e.g. a mouse button invites pushing, a door handle affords pulling) Direct perception of functional properties allows to understand, without any particular cognitive elaboration, which objects of our environment are useful to reach a given goal Norman (1988) used the term to discuss the design of everyday objects. Since then, has been popularized in interaction design to discuss how to design interface objects (e.g. scrollbars to afford moving up and down, icons to afford clicking on) Gulfs of execution and evaluation The notions of gulfs provided a framework to explore potential mappings and mismatches between how a system was designed to work and how a person understands how to do a task using it Cognition Module 2 Prof. Serino 15  The gulfs are an early conceptual framework in HCI The ‘gulfs’ explicate the gaps between the user and the interface The gulf of execution - the distance from the user to the physical system The Gulf of Execution refers to the gap between a user's goal or intention and the actions available in the system to achieve that goal. It highlights the difficulty users may experience in figuring out how to perform a desired action. Causes Poorly designed interfaces that do not make available actions clear. Ambiguous or hidden controls that confuse users. Lack of logical mapping between controls and their effects. Examples 1. Poor Button Placement: A user wants to print a document but cannot find the "Print" button because it is buried in a submenu. 2. Confusing Navigation: An e-commerce site requires users to go through multiple unrelated steps to add items to their cart. The gulf of evaluation - the distance from the physical system to the user The Gulf of Evaluation refers to the gap between the system's feedback and the user's ability to interpret and understand it. It highlights the difficulty users may face in understanding the system's current state or the outcome of their actions. Causes Vague or unclear system feedback. Overly technical error messages that users cannot decipher. Lack of real-time status updates or progress indicators. Examples Generic Error Messages: A system displays "Error 404" without explaining that a webpage could not be found. No Feedback: A user clicks "Submit," but there’s no confirmation, so they are unsure if the action was successful. Bridging the gulfs can reduce cognitive effort required to perform tasks Cognition Module 2 Prof. Serino 16  Can reveal whether interface increases or decreases cognitive load and whether it is obvious what to do next (Norman 1986; Hutchins et al. 1986) Embodied interaction An embodied view of cognition was popularized in HCI community by Dourish He used the term embodied interaction to describe an approach to interaction design that places an emphasis on understanding and incorporating our relationship with the world around us, both physical and social, into the design and use of interactive systems Embodied metaphors Interaction models based on embodied metaphors Cognitive structures of higher-order thinking emerge from recurrent patters of bodily or sensory-motor experience. Such recurrent patterns in bodily experiences are also referred to as image or embodied schemata. A metaphor allows us to understand or experience one concept (target domain) in terms of another (source domain). When the source domain involves schemata that we have arisen from bodily experiences, we call them embodied schemata and the metaphors, embodies metaphors. Based on children’s early experiences combining movement and sound perception, Juntunen and Hyvönen suggest a metaphorical link between body movement and abstract sound concepts such as pitch or volume. This link relies on embodied metaphors, which enable children to understand abstract (sound) concepts in terms of concrete (embodied) concepts Cognition Module 2 Prof. Serino 17  For example, children can understand the concept of volume (soft versus loud) in terms of concrete, movement-related concepts (for example, slow versus fast or up versus down) Design for mental models and generative AI Generative AI poses new challenges for users, and designers must carefully consider how to leverage the mental model framework to help users understand how a system works and how their actions affect it Strategy 1: Orient the user to generative variability Help the user understand the AI system’s behavior and that it may produce multiple and different outputs for the same input AI Example: Google Gemini provides answers in multiple drafts, indicating that it came up with multiple different answers for the same question Strategy 2: Teach effective use Help the user learn how to use the AI system effectively by explaining features and examples through in-context mechanisms and documentation Example: DALL-E provides curated examples of generated outputs and the prompts used to generate them. Adobe Photoshop provides pop-ups and tooltips to introduce users to its Generative Fill feature. Strategy 3: Understand the user’s mental model Build upon the user’s existing mental models and evaluate how they think about your application: its capabilities, limitations, and how to work with it effectively Example: Evaluating a Q&A application, you might ask the user, “how did the system answer your question about who the current president is?” Answers such as, “it looked it up on the web” might indicate a need to educate users about hallucination issues Strategy 4: Teach the AI system about the user Cognition Module 2 Prof. Serino 18  Capture the user’s expectations, behaviors, and preferences to improve the AI system’s interactions with them Example: ChatGPT provides a form for “Custom Instructions” in which users provide answers to questions such as, “Where are you based?”, “What do you do for work?”, and “What subjects can you talk about for hours?” In this way, users teach ChapGPT about themselves to receive more personalized responses Cognition Module 2 Prof. Serino 19