User Study Methods - Part 1-2 PDF

CS 5140 Human Factors in Computing 2 Qualitative Study No Number Involved Explanations Experiences Stories Observations 3 Qualitative Study No Calculation Aims to understand the HUMAN MIND, BEHAV...

CS 5140 Human Factors in Computing 2 Qualitative Study No Number Involved Explanations Experiences Stories Observations 3 Qualitative Study No Calculation Aims to understand the HUMAN MIND, BEHAVIOR, and CHARACTERISTICS To develop a deep and holistic understanding of phenomenon 4 Which one is the Qualitative Data? 5 Which of these studies can be a Qualitative Study? Why do certain age-group of people What age-group of people prefer prefer using Instagram over Facebook? Instagram over Facebook? 6 How to Conduct Qualitative Study? General Steps for Qualitative Studies Determine Data Identify and Recruit Determine Objectives Develop Focal Points Collection Methods Participants and Design Materials Understand how the Questions that focus and design principles that scope the project: you have used in your design is helping users How is your design make better user helping users combat experience the usability issues? How can you improve the designs to make the user experience better? 8 Design Improvement and Evaluation Parameters Measure the User Experience of the products Comprehensive impression of user experience Measure classical usability aspects ○ Efficiency, ○ Perspicuity, ○ Dependability, etc. 9 Design Improvement and Evaluation Parameters 10 Design Improvement and Evaluation Parameters 11 Design Improvement and Evaluation Parameters 12 Design Improvement and Evaluation Parameters 13 Design Improvement and Evaluation Parameters Many more… 14 Example 1. Which parts of the design do you think have made it attractive? a. How would you evaluate our use of font and color in making the design visually appealing? b. Please suggest the changes in design that you think will make it more attractive. [Direct Involvement] 15 General Steps for Qualitative Studies Identify and Recruit Determine Objectives Develop Focal Points Participants 16 Participants Recruitment Strategy Convenience and Snowball Sampling 17 Participants Recruitment Strategy Flyers 18 General Steps for Qualitative Studies Determine Data Identify and Recruit Determine Objectives Develop Focal Points Collection Methods Participants and Design Materials Interview 19 Interview 1. Interview guidelines 2. Do’s and Don’ts of interview 20 Interview Guidelines One-on-one Interview Conduct an interview with the target audience to understand the effectiveness of your design. 22 One-on-one Interview In-person Call Video Call Mediums 23 Types of Interviews Open-ended / Unstructured Structured Semi-structured 24 Unstructured Interviews Unfold like a conversation No specific question No agreed order General topics to cover 25 Structured Interviews Predetermined Questions Questions asked in an order Interviewer knows what to look for from each questions 26 Semi-structured Interviews In between structured & unstructured Flexibility for open-ended follow-up as situation evolves 27 Types of Interviews 28 Interview: Do’s and Don’ts Do’s: Be organized before you start Audio/video equipment ○ audio/video recorder, microphone, extra batteries, tripod Note taking equipment Instruments: interview scripts, designs 30 Do’s: Primary and Secondary Interviewer There is often too much for one person to do! Primary ○ Usually the person who has contacted the participant and guides the discussion Secondary ○ Responsible for most data capture (all recording devices, primary notes, artifact collection) 31 Do’s Structure the time Give participants context Use props and visuals (e.g., prototypes, photos) Listen and make eye contact Be attentive, respectful, sympathetic, and flexible Give the participant time to think 32 Do’s: Post session Debrief immediately with partner/team Check your recordings and label media Make a log of all the items from the session (audio/video tapes, notes…) 33 Don’t: Ask Leading Questions Example: Leading: Why would you prefer to use our product? What is wrong with this question??? 34 Don’t: Ask Leading Questions Example: Leading: Why would you prefer to use our product? Better: What are your thoughts about using our product? 35 Don’t: Ask Closed Questions Instead Ask Open-ended Questions Example: Closed: Do you like coffee? What is wrong with this question??? 36 Don’t: Ask Closed Questions Instead Ask Open-ended Questions Example: Closed: Do you like coffee? Open-ended: What are your thoughts on coffee? 37 Don’t: Use Double-Barreled Questions Example: Double-barreled: What do you like about coffee and new coffee products? What is wrong with this question??? What is wrong with this question??? 38 Don’t: Use Double-Barreled Questions Example: Double-barreled: What do you like about coffee and new coffee products? Better: What do you like about coffee products? 39 Don’t Do not pre-suppose answer Avoid asking long questions Do not use jargons Do not interrupt unnecessarily 40 In-class Activity 41 Demographic Questions 42 Demographic Questions Background information on the participants Context for the collected data, allowing researchers to describe their participants and better analyze their data 43 Purpose of Demographic Questions Obtain a clear picture of who participated in the study Accurately describe the sample Categorize the data into different demographic groups during analysis 44 Writing Demographic Questions Carefully wording the questions 45 Writing Demographic Questions Carefully wording the questions Providing answer choices that will elicit the information that they seek 46 Writing Demographic Questions Carefully wording the questions Providing answer choices that will elicit the information that they seek Do not invade the privacy of participants, provide an option: ‘Prefer not to answer’ 47 Writing Demographic Questions Carefully wording the questions Providing answer choices that will elicit the information that they seek Do not invade the privacy of participants, provide an option: ‘Prefer not to answer’ Provide exhaustive choices: ‘Other’ 48 Writing Demographic Questions Carefully wording the questions Providing answer choices that will elicit the information that they seek Do not invade the privacy of participants, provide an option: ‘Prefer not to answer’ Provide exhaustive choices: ‘Other’ Provide enough range so that the data is differentiated and meaningful 49 Writing Demographic Questions Carefully wording the questions Providing answer choices that will elicit the information that they seek Do not invade the privacy of participants, provide an option: ‘Prefer not to answer’ Provide exhaustive choices: ‘Other’ Provide enough range so that the data is differentiated and meaningful Use mutually exclusive answer choices so that selecting one answer would negate other choices 50 Demographic Questions Placement Where? ○ Place demographic questions at the end of an interview Why? ○ Will not be primed 51 Demographic Questions Placement Where? ○ Place demographic questions at the end of an interview Why? ○ Will not be primed There are exceptions! 52