Usability Evaluation Methods

Podcast Beta

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is a primary advantage of the Think Aloud method?

It eliminates all subjectivity in evaluations.

It does not alter task performance.

It can show how the system is actually used. (correct)

It requires extensive expertise from the evaluator.

Which of the following is a significant disadvantage of interviews as a data collection method?

They are easier to analyze rigorously.

They are often informal and subjective. (correct)

They can be structured to fit any context.

They reach a large audience quickly.

In what way are cooperative evaluations an improvement over traditional think aloud protocols?

They eliminate the need for user input.

They allow for more user criticism of the system. (correct)

They follow a strict question format.

They require less time and are more expensive.

What is a characteristic weakness of questionnaires in data collection?

They often lack depth due to fixed questions. Signup and view all the answers

What is an essential consideration when designing a questionnaire?

Determining what information is necessary. Signup and view all the answers

What is a primary disadvantage of using laboratory studies for evaluation?

It is difficult to observe multiple users cooperating. Signup and view all the answers

Which evaluation method involves experts assessing if usability criteria are violated?

Heuristic Evaluation Signup and view all the answers

What is a key advantage of field studies in evaluating system usability?

The ability to retain contextual factors. Signup and view all the answers

In the context of user participation, if both advantages and disadvantages are present, which environment may be least appropriate for testing?

A noisy environment with many distractions. Signup and view all the answers

What advantage does the laboratory study offer for system evaluations?

Availability of specialist equipment. Signup and view all the answers

Which cognitive evaluation method focuses on supporting users in learning tasks?

Cognitive Walkthrough Signup and view all the answers

What should be carefully considered when applying results from model-based evaluations?

Whether results apply to a new design. Signup and view all the answers

What is the purpose of formative evaluation in the design process?

To gather user feedback for future product revisions. Signup and view all the answers

What is the primary purpose of an experimental evaluation?

To evaluate specific aspects of interactive behavior Signup and view all the answers

Which of the following correctly differentiates between independent and dependent variables?

Independent variables are modified, dependent variables are the result of these modifications Signup and view all the answers

In hypothesis formulation, what does the null hypothesis represent?

A prediction of no difference between experimental conditions Signup and view all the answers

What defines a within groups experimental design?

Each subject is tested under various conditions Signup and view all the answers

What is a characteristic of discrete data in statistical analysis?

It consists of a finite number of values. Signup and view all the answers

Which experimental factor pertains to the participants involved in a study?

Subjects Signup and view all the answers

What is a primary consideration when selecting a statistical technique for data analysis?

The type of data collected Signup and view all the answers

Why is it important to save original data before performing statistical analysis?

To facilitate recalculation in case of errors Signup and view all the answers

Study Notes

Evaluation

Test usability and functionality of a system
Can be conducted in the lab, field, and/or in collaboration with users
Evaluates design and implementation
Should be considered at all stages of the design lifecycle
Formative v.s. Summative evaluation

Goals of evaluation

Assess the extent of system functionality
Assess the effect of the interface on the user
Identify specific problems

Evaluating Designs

Cognitive Walkthrough
Heuristic Evaluation
Model-based Evaluation

Cognitive Walkthrough

Proposed by Polson et al.
Evaluates design on how well (or easy) it supports users in learning tasks
Usually performed by an expert in cognitive psychology
Can be performed by an expert 'walking through' a design to identify potential problems using psychological principles
Forms can be used to guide analysis
Example: Command line interface v.s. WIMP

Heuristic Evaluation

Proposed by Nielsen and Molich
Usability criteria (heuristics) are identified
Design examined by experts to see if these criteria are violated
Example heuristics: System behaviour is predictable, system behavior is consistent, and feedback is provided.
Heuristic evaluation 'debugs' design.

Model-based evaluation

Results from the literature used to support or refute parts of a design
Care is needed to ensure results are transferable to a new design
Cognitive models used to filter design options
- GOMS (goals, operators, methods, and selection) prediction of user performance
Design rationale can also provide useful evaluation information

Evaluating through user participation

Laboratory studies

Advantages:
- Specialist equipment available
- Uninterrupted environment
Disadvantages:
- Lack of content
- Difficult to observe several users cooperating
Appropriate:
- If a system's location is dangerous or impractical for constrained single-user systems to allow controlled manipulation of use.

Field Studies

Advantages:
- Natural environment
- Context retained (though observation may alter it)
Disadvantages:
- Distractions
- Noise
Appropriate:
- Where content is crucial and difficult to implement in a lab.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Explore various evaluation techniques used to assess system usability and functionality. This quiz covers methods such as cognitive walkthroughs and heuristic evaluations, emphasizing their importance throughout the design lifecycle. Understand how to identify specific problems and improve user interface design.