HCI Usability Measurement PDF

User Interface Design (UX/UI) Usability test The key to developing highly usable software is employing the expression, “test early and often”, is particularly appropriate when it comes to usability testing 2  A/B Testing A...

User Interface Design (UX/UI) Usability test The key to developing highly usable software is employing the expression, “test early and often”, is particularly appropriate when it comes to usability testing 2  A/B Testing A/B Testing lets you compare two versions of an ad strategy by changing variables such as ad images, ad text, audience or placement. We show each version to a segment of your audience and ensure nobody sees both, then determine which version performs best. Before you select a variable to test, choose a hypothesis for your test. For example, you might hypothesize that a custom audience strategy will outperform an interest-based audience strategy for your business. 8  How can the usability of an interactive system be test? 1. Gathering feedback (Using user) I. completion rate time II. Effectiveness (error rate ) III. Efficiency (task time) IV. Satisfaction 2. Cognitive and predictive models (Without user)Less expensive than user testing. 10 I. Completion Rate time Completion Rate is measured in terms of task time. that is, the time (in seconds and/or minutes) the participant takes to successfully complete a task. The time taken to complete a task can then be calculated by simply subtracting the start time from the end time as shown in the equation below: Task Time (second) = End Time – Start Time 11 12 13 i- Effectiveness Time-Based Effectiveness (error rate) it is calculated by assigning a binary value of ‘1’ if the test participant manages to complete a task and ‘0’ if he/she does not. it can be collected during any stage of development. Effectiveness can thus be represented as a percentage by using this simple equation: 14 Example: Calculation of effectiveness Let: 5 users perform a task using the same system. At the end of the test session, 3 users manage to achieve the goal of the task while the other 2 do not. Using the above equation, the overall user effectiveness of the system is worked out as follows: Number of tasks completed successfully = 3 Total number of tasks undertaken = 5 Inserting the above values into the Effectiveness equation 15 ii. Time-Based Efficiency Where: N = The total number of tasks (goals) R = The number of users nij = The result of task i by user j; if the user successfully completes the task, then nij = 1, if not, then nij = 0 tij = The time spent by user j to complete task i. If the task is not successfully completed, then time is 16 measured till the moment the user quits the task Example: Calculation of time-based efficiency Suppose there are 4 users who use the same product to attempt to perform the same task (1 task). 3 users manage to successfully complete it – taking 1, 2 and 3 seconds respectively. The fourth user takes 6 seconds and then gives up without completing the task. Taking the above equation: N = The total number of tasks = 1 R = The number of users = 4 User 1: Nij = 1 and Tij = 1 User 2: Nij = 1 and Tij = 2 User 3: Nij = 1 and Tij = 3 User 4: Nij = 0 and Tij = 6 Placing the above values in the equation: 17  It is also very easy to graphically represent the completion rate, for example as a stacked bar graph as shown in the diagram below 18 Exercise Suppose there are 3 users who use the same product to attempt to perform the same two task. 3 users manage to successfully complete first task it – taking 1, 2 and 3 seconds respectively. The second task taking 1,2,10 but second user gives up without completing the task. Calculate :Effective time for each task 20 Satisfactions User satisfaction is measured through standardized satisfaction questionnaires which can be administered after each task and/or after the usability test session. Task Level Satisfaction Test Level Satisfaction 22 Task Level Satisfaction After users attempt a task (irrespective of whether they manage to achieve its goal or not), they should immediately be given a questionnaire so as to measure how difficult that task was. Typically consisting of up to 5 questions, these post- task questionnaires often take the form of Likert scale ratings and their goal is to provide insight into task difficult 23 Test Level Satisfaction Test Level Satisfaction is measured by giving a formalized questionnaire to each test participant at the end of the test session. This serves to measure their impression of the overall ease of use of the system being tested. For this purpose, the following questionnaires can be used (ranked in ascending order by number of questions): 24 26 27 Cognitive and predictive models (Without user) 30  Cognitive modeling It is an area of computer science that deals with simulating human problem solving and mental task processes in a computerized model by understanding the cognitive processes, and explain how these processes interact. The processes that the brain uses to accomplish tasks including: – learning, remembering, thinking, predicting, problem solving, decision making, Planning. Provide a way of evaluating products or designs without directly involving users. 31 GOMS model 34 It is a cognitive model of human performance that can be used to improve the efficiency of human-machine interaction by identifying and eliminating unnecessary user actions.  GOMS model GOMS stands for (Goals, Operators, Methods, and Selection). – Methods that are used to achieve specific Goals. – These methods are then composed of Operators at the lowest level. – The operators are specific steps that a user performs and are assigned a specific execution time. – If a goal can be achieved by more than one method, then Selection rules are used to determine the proper Method. 35 36 Example 1:GOMS model Discuss the GOMS-model of editing text (cut,copy,delete)in word processors Goal : Edit text (initial situation) Select : Method A: move text Operator: cut Method B: copy text Operator: copy Method C: delete text operator :(del) 37 Exercise 2:GOMS model Discuss the GOMS-model of mobile phone to make a phone call 40 41 GOMS Model Goal: make a phone call –Select: Method A: type number 42 –Operator1: Open the "Phone" application –Operator 2: Dial the number you want to call –Operator 2: Press "Call" Method B: contact –Operator1: Open the "Phone" application –Operator 2: Call an existing contact –Operator 2: Press "Call" Disadvantages GOMS model GOMS only applies to skilled users. It does not work for beginners. the lack of account for errors, even skilled users make errors but GOMS does not account for errors Mental workload is not addressed in the model, making this an unpredictable variable. User personalities, habits or physical restrictions (for example disabilities) are not accounted for in any of the GOMS models. All users are assumed to be exactly the 47 same. Predictive models: Provide a way of evaluating time without directly involving users. 1. KLM-GOMS Less expensive than user 2. Fitts’ Law testing. 3. Hick’s law 48 1- Keystroke-level model (KLM) Fast, approximate way to calculate time taken to complete a task. A task is broken into a series of subtasks 49 Example: Consider a task such as “delete a file” Perhaps there are two ways to do the task: 1. Mouse + menu selection 2. Keyboard + command entry The KLM can predict the time for each method Total predicted time is the sum of the subtask times. KLM Operators operation Notation Description time Keystrokes (K) mouse click, or 0.08s button press Pointing (P) for moving mouse 1.1s to target (or as determined using Fitts' Law) 50 Homing (mouse) or (H) for moving hand 0.4s H(keyboard between mouse and keyboard Drawing : (D) 0.9n + 0.16l (where n = number of segments, l = total length of segments) System response (Sys) as estimated based time on system Mental operators (M) to think about task 1.35s Total time using KLM model=Summing together 51 52 53 54 55 A KLM Prediction Assumptions Simple world Single-tasking 56 No interruptions Single method to complete a task. It Can be improved by incorporating Fitts’ Law, and by using tables for keystroke speeds for individual keys Q: Use KLM to compare different designs for: goal: Select file and drag it trash can (assume your hand on K.B, Do it with mental operators) Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5s Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard)  Goal: delete file Select: Method A: using mouse Operator1:moving hand to mouse(H) Operator2:pointing file(P) Operator3:select file (p) Operator 4: right click (p) Operator5:drage mouse in sub menu(p) Operator6:select delete (p) 60 Operator7: click on delete(k) Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5s Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard)  Goal: delete file Select: Method B: using K.B Operator1:moving hand to select direction arrow(H) Operator2:select file(k) Operator3:moving hand to delete key(H) Operator4:press delete (k) 62 Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5s Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard) KLM benefits and disadvantages? Benefits? easy to model Can be done from just pictures or ideas (i.e. before an interface is built) Drawbacks? Some time estimates are inherently variable (typing speed) Doesn’t model: Errors and Learning time  Test yourself ! 1. Predict the task completion time to: book in train from Alex to Cairo. Assumption: Hands on keyboard Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5 Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard) Mental M 1.35s operators to think about task Test yourself ! Goal: book in train Select: Method A:K.B Method B:Mouse  Test yourself ! Goal: book in train Select: Method A:K.B Operator1: tab to first text box (k) Operator 2:type Alex(4 k) Operator 3:tab to second text box(k) Operator 4: type Cairo(5k) Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5 Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard) Mental M 1.35s operators to think about task  Test yourself ! Goal: book in train Select: Method A:K.B Operator1: tab to first text box (k) Operator 2:type Alex(4 k) Operator 3:tab to second text box(k) Operator 4: type Cairo(5k) Description Operation time Keystrokes K 1.2s per keystroke, mouse click, or button press Pointing P 0.5 Homing H 0.4s (mouse or for moving hand between mouse and keyboard keyboard) Mental M 1.35s operators to think about task 2- Fitts' law It predicts time required to point a target on screen. (model the act of pointing) 69 By physically touching an object with a hand, finger or virtually By pointing to an object on a computer monitor using a pointing device. Fitts’ Law cont.'s MT = a + b log 2 ( (A / W) +1 ) 70 A W Hick’s law Predicts the time it takes to make a decision in selecting among possible choices. 71  Hick’s Law describes the time it takes for a person to make a decision as a result of the possible choices they have. According to the law, increasing the number of choices will increase the time it takes for a person to make a decision.