32 Questions
What is one of the goals of evaluation in the design life cycle?
To evaluate the level of system functionality
What is the primary focus of Cognitive Walkthrough analysis?
To identify problems using psychological principles
Where can evaluation occur?
In a laboratory, field, and/or in collaboration with users
What is one of the things that evaluation identifies?
Specific problems with the system
Who typically performs a Cognitive Walkthrough?
An expert in cognitive psychology
What is the purpose of forms in Cognitive Walkthrough analysis?
To guide the analysis
When should evaluation be considered?
At all stages of the design life cycle
What type of evaluation is proposed by Polson et al.?
Cognitive Walkthrough
What aspect of user interaction is considered in Cognitive Walkthrough?
Impact of interaction on the user
What is one of the things that evaluation evaluates?
The effect of the interface on user satisfaction
What cognitive processes are considered in Cognitive Walkthrough?
Cognitive processes required for task completion
What is the ultimate goal of Cognitive Walkthrough analysis?
To ensure the design leads the user to make correct goals
Who proposed the Heuristic Evaluation method?
Nielsen and Molich
What is the purpose of Heuristic Evaluation?
To identify usability issues
What is one of the heuristics mentioned in the text?
System behavior is predictable
What is the goal of debugging in Heuristic Evaluation?
To fix errors
What is Review-based evaluation used for?
To support or refute parts of a design
What is important to ensure when using Review-based evaluation?
That the results are transferable to new designs
What is a key advantage of laboratory studies in evaluating user participation?
Specialist equipment is available
What is a limitation of field studies in evaluating user participation?
Distractions and noise
What is required for evaluating implementations?
An artefact, such as a simulation or prototype
What is a key characteristic of experimental evaluation?
Evaluator chooses the hypothesis to be tested
What is an advantage of model-based evaluation?
Not mentioned in the text
What is a limitation of laboratory studies in evaluating user participation?
Lack of context
What is the purpose of a hypothesis in an experiment?
To frame a prediction of the outcome
What is an independent variable in an experiment?
A characteristic changed to produce different conditions
What is a dependent variable in an experiment?
A characteristic measured in the experiment
What is the main advantage of a within-groups design?
It is less costly and less likely to suffer from user variation
What is a null hypothesis?
A statement of no difference between conditions
What is an example of an independent variable in an experiment?
Interface style
What is the purpose of a null hypothesis?
To state no difference between conditions
What is an example of a dependent variable in an experiment?
Error rate
Study Notes
Evaluation
- Evaluates the usability and functionality of a system
- Occurs in laboratory, field, and/or in collaboration with users
- Evaluates both design and implementation
Goals of Evaluation
- Evaluates level of system functionality
- Evaluates effect of interface on user
- Identifies specific problems
Evaluating Designs
- Cognitive Walkthrough: evaluates design on how well it supports user in learning tasks
- Heuristic Evaluation: evaluates design based on usability criteria (heuristics)
- Review-based evaluation: uses written reviews to support or refute parts of design
Cognitive Walkthrough
- Evaluates design on how well it supports user in learning tasks
- Performed by expert in cognitive psychology
- Expert 'walks through' design to identify possible problems using psychological principles
- Forms used to guide analysis
- For each task, walkthrough considers:
- What impact will interaction have on user?
- What cognitive processes are required?
- What learning problems may occur?
Heuristic Evaluation
- Proposed by Nielsen and Molich
- Evaluates design based on usability criteria (heuristics)
- Examples of heuristics:
- System behavior is predictable
- System behavior is consistent
- Feedback is provided
- Heuristic evaluation 'debugs' design
Review-based Evaluation
- Results from written review used to support or refute parts of design
- Care needed to ensure results are transferable to new design
Evaluating through User Participation
- Laboratory studies:
- Advantages: specialist equipment available, uninterrupted environment
- Disadvantages: lack of context, difficult to observe several users cooperating
- Field Studies:
- Advantages: natural environment, context retained
- Disadvantages: distractions, noise
Evaluating Implementations
- Requires an artefact: simulation, prototype, or full implementation
- Experimental evaluation:
- Controlled evaluation of interactive behavior
- Evaluator chooses hypothesis to be tested
- A number of experimental conditions are considered, which differ only in the value of some controlled variable
- Changes in behavioral measure are attributed to different conditions
Experimental Factors
- Subjects:
- Who: representative, sufficient sample
- Variables:
- What to modify and measure
- Hypothesis:
- What you'd like to show
- Experimental design:
- How you are going to do it
Variables
- Independent variable (IV):
- Characteristic changed to produce different conditions
- E.g. interface style, number of menu items
- Dependent variable (DV):
- Characteristics measured in the experiment
- E.g. time taken, number of errors
Hypothesis (Thesis or Theory)
- Prediction of outcome
- Framed in terms of IV and DV
- E.g. "error rate will increase as font size decreases"
- Null hypothesis:
- States no difference between conditions
- E.g. "no change with font size"
Assessing the usability and functionality of a system, evaluating design and implementation, and testing in laboratory, field, and collaborative settings.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
