Are You a UX Design Expert?

Aspects to Evaluate in System Design and Development

• Personal music services must be evaluated based on the ease of music selection and storage for users.
• Navigation and attractiveness of clothes are core concerns in evaluating a website for selling clothes.
• Trustworthiness and security of credit card details are important aspects in evaluating both systems.
• Remote studies of online behavior and living labs can be used to evaluate natural interactions of participants in different settings.
• Evaluation timing depends on the product type and development process, ranging from new concepts to upgrades and rapidly changing markets.
• Formative evaluations are conducted during design to ensure design ideas meet user requirements.
• Summative evaluations assess the success of a finished product, while upgrades may focus on improving specific aspects.
• Rapid iterations of product development often embed evaluations into short cycles of design, build, and test.
• Government websites and certain products, such as aircraft navigation systems, have to be evaluated according to set standards.
• The WCAG 2.1 provides guidelines for designing accessible websites.
• Lab-based experiments using a car simulator are necessary to evaluate a new car seat's effectiveness in a safe setting.
• The new car seat's feedback mechanism must be tested in a more natural setting, such as a race track or safe training circuit for new drivers.

Types of Evaluation in Human-Computer Interaction

• Experiments and user tests are designed to control what users do, when they do it, and for how long to reduce outside influences and distractions.
• Usability testing involves collecting data using a combination of methods in a controlled setting to determine whether an interface is usable by the intended user population.
• Usability testing involves investigating how typical users perform on typical tasks, comparing the number and kinds of errors that users make between versions, and recording the time it takes to complete a task.
• The data collected in usability testing can be supplemented by observation at product sites to collect evidence about how the product is being used in the workplace or in other environments.
• Usability testing is a fundamental, essential HCI process and has been a staple of companies for many years.
• Optimal performance levels and minimal levels of acceptance are generally specified, and current levels are noted in a usability specification.
• Testing in a laboratory may also be done when it is too disruptive to evaluate a design in a natural setting.
• Living labs have been developed to evaluate people’s everyday lives, for example, to investigate people’s habits and routines over a period of several months.
• The goal of field studies is to evaluate products with users in their natural settings.
• Field studies are used primarily to help identify opportunities for new technology, establish the requirements for a new design, and facilitate the introduction of technology or inform deployment of existing technology in new contexts.
• Living labs are being developed that form an integral part of a smart building that can be adapted for different conditions to investigate the effects of different configurations of lighting, heating, and other building features on the inhabitant’s comfort, work productivity, stress levels, and well-being.
• Citizen science, in which volunteers work with scientists to collect data on a scientific research issue, can also be thought of as a type of living lab, especially when the behavior of the participants, their use of technology, and the design of that technology are also being studied.

Field Studies and Living Labs

• Field studies evaluate products in natural settings to identify opportunities for new technology, establish design requirements, or facilitate technology deployment.
• Smart buildings, such as the Smart Living Lab in Switzerland, are used as living labs to investigate the effects of different building configurations on inhabitant experiences.
• Mobile living labs, where people are fitted with wearable devices to measure health, enable more people to be studied over longer periods and in difficult-to-observe locations.
• Citizen science, where volunteers work with scientists to collect data on scientific research issues, can be considered a type of living lab.
• Lab in the Wild is an online site that hosts volunteers for research projects and can be studied over time as a type of living lab.
• The concept of a living lab aims to be both natural and experimental, which poses the dilemma of how artificial to make the natural setting.
• Observation, interviews, and interaction logging are methods used in field studies to record events and conversations without affecting participant behavior.
• In-the-wild studies look at how new technologies or prototypes are deployed and used by people in various settings, such as outdoors, in public places, and in homes.
• Disruptive technologies are prototypes deployed in the wild to determine how they displace existing technologies or practices.
• Giving up control in the wild makes it difficult to anticipate what will happen and be present when something interesting occurs.
• Virtual field studies take place in multiuser games, online communities, and chat rooms to examine social processes such as collaboration, confrontation, and cooperation.
• Online field studies are popular in geological and biological sciences and can supplement studies in the field.

Methods for Conducting User Interface Evaluations

• Evaluations can be conducted without involving users by using inspection methods such as heuristic evaluation, walkthroughs, cognitive walkthroughs, analytics, and models.
• Heuristic evaluations involve applying knowledge of typical users to identify usability problems based on rules of thumb, while cognitive walkthroughs simulate a user's problem-solving process to evaluate designs for ease of learning.
• Analytics is a technique for logging and analyzing data to understand and optimize web usage, while models are used to compare the efficacy of different interfaces for the same application.
• Combinations of methods across categories can be used to obtain a richer understanding of user interface evaluations.
• Controlled settings allow for testing hypotheses about specific features of the interface, while uncontrolled settings provide unexpected data that can provide different insights into people's perceptions and experiences of using new technologies.
• Opportunistic evaluations are informal and do not require many resources, and can be conducted early in the design process to provide designers with feedback quickly about a design idea.
• Early evaluations are important because they confirm whether it is worth proceeding to develop an idea into a prototype.
• Opportunistic evaluations with users can also be conducted to hone the target audience and make subsequent evaluation studies more focused.
• The original heuristics used in heuristic evaluation were for screen-based applications, but they have been adapted to evaluate web-based products, mobile systems, collaborative technologies, computerized toys, and information visualizations.
• One problem with using heuristics is that designers can sometimes be led astray by findings that are not as accurate as they appeared to be at first.
• Cognitive walkthroughs have been used to evaluate smartphones, large displays, and other applications, and focus on evaluating designs for ease of learning.
• Analytics can be particularly useful for evaluating design features of a website, and learning analytics have gained prominence for assessing the learning that takes place in MOOCs and OERs.