Interaction Design Evaluation

Testing and Evaluation

• Testing and evaluation are crucial for interaction design
• They help identify areas for improvement in performance/efficiency
• This includes measuring time and steps needed to complete tasks
• Accuracy of user performance must be assessed, including fatal and recoverable errors
• Recall measures how much information is remembered after periods of non-use
• Emotional response is an important factor, considering user confidence and stress levels
• User recommendations are also important feedback measures

Evaluation Goals

• Assessing system functionality is vital
• Identifying the impact on the user is important
• Pinpointing specific user problems is important for improvement
• Designers need to understand if their designs meet users' needs and expectations

Usability

• Usability refers to a product's ease of use by a specific user group to achieve their goals effectively and efficiently, with satisfaction within a specific context
• Usability involves measures of ease of learning, efficiency of use, memorability, error frequency/severity and subjective satisfaction scores

User Evaluation

• User-based evaluations are considered the most reliable and valid usability assessments
• Participants complete tasks with the technology
• Success rate and speed of task completion are often recorded, depending on the evaluator's focus

Categories of Evaluation

• Lab/controlled studies, such as usability testing, are common experimental research methods
• Field/in the wild studies are observational
• Observation methods, like think-aloud protocols, video analysis, are also used
• Other methods include questionnaires, interviews, and focus groups

Evaluation Methods

• The evaluation approach influences the methods used
• Usability testing often involves users, interviews, participatory design, and questionnaires
• These tests are usually conducted in labs or natural settings
• The goal is to assess interface usability for the target user population
• Field studies, which do not involve laboratory settings, often involve observations and interviews
• Qualitative data is usually generated in field studies

Qualitative vs. Quantitative

• Evaluation methods can use quantitative or qualitative approaches
• Quantitative methods involve gathering numerical data, like time of completion, error count
• Qualitative methods use words to describe user experience, in interviews, open-ended questionnaires

Other Considerations

• Reliability/reliability shows similar results with repeated studies
• Validity measures the accuracy of the method in measuring intended factors
• Bias may influence experiments, as expectations, beliefs, and preconceived notions may affect outcomes.
• Scope boundaries the limits and extent of a study

Questionnaires

• Choose between creating a new or adapting an existing questionnaire to assess perceived usability
• Reputable questionnaires provide more reliable results than new ones
• Existing questionnaires include the Questionnaire for User Interface Satisfaction (QUIS), the Computer System Usability Questionnaire (CSUQ), the System Usability Scale (SUS) and questionnaires focused on robotic perceptions, and task load index measurement tools

Observations

• Observing users interacting with technology and recording notes are common practice
• Minimizing interaction during observation can enhance validity
• This method simulates real-world usage conditions

Interviews

• Interviews involve one-on-one interactions with users based on prepared questions
• Structured interviews use all prepared questions; semi-structured interviews use a mix, and unstructured ones focus on broad topics
• Interviews can explore user contexts and issues more fully
• They can provide insights into how users perceive and use products

Focus Groups

• This discussion-based group interview is often used in marketing research
• Participants are often selected for their particular characteristics
• They tend to be informal, driven by open-ended questions
• Focus groups may use a questionnaire as part of the process

Think Aloud Protocols

• This method observes user's thought process while completing tasks, asking them to verbalize their thoughts
• Simplicity, useful insights, and realistic use patterns are advantages
• Subjectivity and selective description may bias results

Experimental Research

• A single-variable test modifies a variable while other factors remain constant
• Controlled experiments compare results between an experimental and control group
• Terms like independent variables (test conditions), dependent variables (measured outcomes), and confounding variables (uncontrolled variables) are used

Independent/Dependent Variables

• Independent variables can be changed
• Dependent variables are to be observed and measured
• Categories of independent variables in HCI research include interface types, feedback types, and menu types;
• Dependent variables typically measure time, performance, accuracy, and error rates or subjective ratings

Confounds/Confounding Variables

• A confound is an extraneous variable that can affect the relationship between the independent and dependent variables
• It must be considered and controlled in experimental designs
• Factors such as time, soil types, and genetics can be confound variables in usability studies

Experimental Design Within/between Subject

• Within-subject designs involve each participant completing experiments in all conditions, potentially showing learning transfer but being less costly, and less likely to suffer from user-variation issues
• Between-subject designs assign different participants to only one condition, limiting learning transfer but potentially requiring more participants

Method (Procedure/Materials/Setup/Measurements/Participants)

• The procedure in HCI studies outlines the steps involved in data collection
• The materials used include computers, microphones, screens, and tangible objects, among others
• The setup section details the participant's location and other pertinent factors
• Data collection methodology is detailed including what is measured and how
• Participants' demographics are documented

This quiz covers essential concepts in testing and evaluation for interaction design. It explores how to assess user performance, functionality, and emotional responses to improve usability. Gain insights into measuring system efficiency and understanding user needs.