Usability Testing

Evaluation Studies: From Controlled to Natural Settings

• Usability testing is traditionally done in controlled laboratory settings to test the usability of products, including apps and digital products.

• Usability testing involves collecting data about users' performance on predefined tasks, using a combination of methods such as video recordings, think-aloud technique, and user satisfaction questionnaires.

• Performance times and the number of different types of actions carried out by users are the two main performance measures in usability testing.

• Usability labs are custom-built and consist of recording equipment, observation rooms, and monitoring equipment. They can be expensive and labor-intensive to run and maintain.

• Mobile and remote usability testing equipment is an alternative to usability labs and includes video cameras, laptops, and eye-tracking devices that can be set up temporarily in an office or other space.

• Remote, unmoderated usability testing is popular in large companies and allows many users to be tested at the same time in real-world settings.

• The number of users involved in usability testing varies depending on budget and schedule constraints, with 5 to 12 users being an acceptable number.

• Usability testing can be done in natural settings such as people's homes, schools, work, and leisure environments, through field studies.

• Usability testing measures whether a product is usable by the intended user and whether users are satisfied with their experience.

• User testing was conducted by usability specialists on the iPad when it first came onto the market to evaluate interactions with websites and apps specifically designed for the iPad.

• The study used usability testing with think-aloud and an expert review to evaluate user expectations for the iPad compared to the iPhone.

• The first study found that users preferred using apps to browsing the web on the iPad, and the second study examined additional usability issues.Usability Testing on iPad: Participants, Tests, and Procedures

• The study aimed to identify typical usability issues users encounter when using apps and websites on the iPad.

• Seven participants were recruited, all experienced iPhone users who had used a variety of apps for at least three months.

• Participants were considered typical users who represented the range of those who might purchase an iPad and included different age and gender groups.

• Participants were asked to sign an informed consent form before taking part in the study.

• The consent form described the study's terms and conditions, length, compensation, confidentiality, and the right to withdraw from the study at any time.

• Participants were invited to explore any application they found interesting on the iPad and comment on their experience.

• A moderator observed and took notes while the participants worked on their own.

• The sessions were video-recorded and lasted about 90 minutes each.

• Participants were asked to open specific apps or websites, explore them, and carry out one or more tasks assigned in a random order.

• More than 60 tasks were chosen from over 32 different sites, some of which were not specifically designed for the iPad.

• The presentation order of apps and websites was balanced to avoid bias.

• The study ensured the confidentiality of participants' data and promised not to disclose their identity or share data with marketers or anyone other than the researchers.

Usability Testing of iPad: Methods, Findings, and Recommendations

• The study aimed to examine how participants interacted with the iPad when using apps and websites to help designers and developers determine whether specific websites need to be developed for the iPad.
• Usability in this study involved the efficiency, effectiveness, safety, ease of learning, ease of remembering, good utility, creativity, motivation, helpfulness, and satisfaction of the iPad.
• The tasks outlined in the study, such as shopping, reading, planning, and finding a recipe, were a small sample of the total set prepared by the researchers to cover common activities that people engage in during their everyday lives.
• The testing setup used a camera to record the participant's interactions and gestures when using the iPad, a webcam to record their facial expressions and think-aloud commentary, and software called Morae to synchronize the two data streams.
• The participants were able to interact with websites on the iPad but faced various usability problems, such as small links and difficult-to-read fonts, which were classified according to well-known interaction design principles and concepts.
• Some participants got lost in the application because they tapped the iPad too much and could not find a back button or get back to the home page.
• Based on the study findings, the researchers made various recommendations, including supporting standard navigation, and provided a report publicly available to app developers and the general public.
• A year later, the researchers did more tests with iPad users who had at least two months of experience using an iPad and found that even though many recommendations were implemented, there were still some problems, such as accidentally touching something and not finding the starting point.
• The first study was done with participants who did not have direct experience with an iPad, while the second study was done with participants who had at least two months of experience using an iPad.
• The first set of tests in 2010 illustrates how the researchers had to adapt their testing method to fit within a tight time period, and designers and researchers often have to modify how they go about user testing for various reasons.
• Using multiple methods in usability testing can reveal different usability problems, and the importance of not taking for granted that a method used with one group of participants will work with another group, particularly when working with people from different cultures.
• The study did not address how the iPad would be used in people's everyday lives, which required a field study where observations were made of how people use iPads in their own homes, at school, in the gym, and when traveling, but this did not happen due to lack of time.

Conducting Experiments and Hypothesis Testing in User Interface Design

• Hypotheses are specific predictions about user performance with an interface feature based on theories or previous research findings.
• Variables in a hypothesis can be independent (manipulated by the researcher) or dependent (a measure of user performance).
• A null hypothesis is derived to test the alternative hypothesis, stating that there is no effect between the independent and dependent variables.
• One-tailed hypotheses state a specific effect, while two-tailed hypotheses specify a difference without direction.
• Experimental design involves controlling other variables to prevent them from influencing the findings.
• Different-participant design randomly allocates participants to each experimental condition, minimizing individual differences but requiring more participants.
• Same-participant design involves all participants performing in all conditions, requiring fewer participants but necessitating counterbalancing to avoid bias.
• Matched-participant design matches participants based on user characteristics and randomly allocates them to each condition.
• Response times, total task completion times, and number of errors are measured to compare user performance across conditions.
• Statistical tests, such as t-tests, are used to determine if differences between conditions are significant.
• Rigorous hypothesis testing increases confidence in interface design decisions.
• Usability testing can be done in person with a designer and participant, as seen in the video "Rocket Surgery Made Easy" by Steve Krug.