02-1 Needfinding.pdf

Needfinding Assoc Prof. Jeannie S. Lee What do users need? INF2002 1. What do people do now? 2. What values and goals do people have? 3. How are these particular activities embedded in a larger ecology? 4. Similarities and differences across people 5....and other types of context, like...

Needfinding Assoc Prof. Jeannie S. Lee What do users need? INF2002 1. What do people do now? 2. What values and goals do people have? 3. How are these particular activities embedded in a larger ecology? 4. Similarities and differences across people 5....and other types of context, like time of day INF2002 Why needfinding? Uncover needs that people have which they may or may not be aware of Guide innovation efforts Identify someone to design for Discover the emotions that guide behaviors INF2002 Requirements Definition Work to understand a user's experience by learning about their lives Define the detailed requirements for the system being developed Gather data about what users need to do or accomplish Analyze and represent data Interpret the results and use that in design decisions Iterative process INF2002 Some data gathering techniques Observation Direct (Field vs. Controlled) Indirect Ethnography Surveys & Questionnaires Interviews Focus Groups INF2002 Observation Watch users do what they do Typically from a distance Video recording May require editing or coding the video later Transcription Take lots of photos, notes, sketches Focus on specific task-relevant behaviors in notes Later convert to abstract subtasks INF2002 Hawthorne Effect Tendency to perform or perceive differently when one knows they are being observed Users may become nervous and may not behave the way they do normally May be thinking more about being observed than doing the task INF2002 Direct Observation Researcher is the observer, recording what is being watched Controlled Setting Controlled environment, possibly lab-based experiment Field Setting In the natural environment of the user Information gained is impossible to gather in the lab INF2002 Controlled vs. Field Settings Controlled (Lab) Setting Field Setting INF2002 Indirect Observation Researcher relies on the reported observation of others (incl. self-observations) Observing the effects or results of the behavior rather than the behavior itself E.g. food diary, app logging usage INF2002 Ethnography Techniques based in sociology and anthropology – the study of humans Deeply contextual inquiry “Wallow in the data” “Live among” the users Understanding the full complexity of behavior, in its complete social context INF2002 INF2002 Surveys & Questionnaires Subjective answers in a quantitative format Mix of qualitative and quantitative formats Questions Exploratory vs. confirmatory Open-ended vs. categorical (exhaustive) INF2002 Example Survey Questions Rank the importance of each of these tasks (give a list of tasks) List the four most important tasks that you perform (this is an open question) List the pieces of information you need to have before making a decision about X, in order of importance Are there any other points you would like to make? (open-ended opinion question; good way to end) INF2002 Question Formats Objective Questions Quantifiable, countable Check boxes to select one or more options Multiple-Choice questions Yes/No, True/False questions Subjective questions Response open ended, in one or more sentences Semantic Scales 1- lowest score to 5- highest score Likert Scale Agree Strongly - Agree Somewhat - Indifferent - Disagree Somewhat - Disagree Strongly INF2002 Sematic Differential Scale INF2002 Likert Scale INF2002 Online Surveys Responses can be tracked Data analysis is easier Reach a wider audience easily Errors can be corrected Survey Monkey https://www.surveymonkey.com/ Google Forms https://www.google.com.sg/forms /about/ INF2002 Surveys & Questionnaire Tips Do not make questionnaire very long Use fewer but better questions Ensure anonymity and clear motivation Start from research goals INF2002 Surveys & Questionnaire Issues Which users to target, how to target? Cheap but difficult to administer Response rate may be low Response bias Data entry or analysis issues Questions may be ambiguous Users may be dishonest INF2002 Interviews Engage the user more than by just watching Rich data directly from the user, adds a lot of context Meet and ask questions face-to-face May be video or audio recording May require editing or coding the video later Transcription of audio Issues of bias and honesty INF2002 Interviews Structured interviews There is a set of questions (script) Efficient, but requires training Unstructured There is no script Might be difficult to follow and conduct Inefficient, but requires no training Semi-structured There is a script Interviewer can explore answers more by deviating from script Good balance, often appropriate INF2002 Interview preparation Objectives, goals, research questions Methods E.g. phone, in person, email, etc. Interview questions Design questions carefully, to get good data (not too vague) Test questions before asking People: Moderator/interviewer; participant(s); scribe(s) INF2002 Interview Phases Introduction Warm-up (ramp up) Interview “proper” Cool-down (glide) Summary and wrap-up INF2002 Effective Interview Questions How do you perform task x? Why do you perform task x? What information do you need to…? Who do you need to communicate with to…? What do you use to…? What happens after you…? What is the result or consequence of NOT…? (Gordon & Gill, 1992; Graesser, Lang, & Elofson, 1987) INF2002 Interviewing Tips Manage the conversation Focus on data, stay on track Listen more, talk less Use examples, but not too long Set expectations and boundaries, establish trust and comfort (“safe zone”), then get to the tough questions Always meta-monitor the situation INF2002 Focus Groups Structured interview with groups of individuals (3 to 10 persons) Use several different groups with different roles or perspectives Audio or video record, with permission Manage the interaction Facilitate and motivate people to talk Avoid few people dominating the discussion Focus on preferences and views, not performance Relatively low cost, quick way to learn a lot INF2002 INF2002 Other Methods Document mining Often contains description of how the (current) tasks should be done Users may not necessarily follow them E.g. Standards documents, Manuals, Histories, Best Practices Data logging Automatically tracking how the task is being done Keystroke/mouse clicks, timers, logs of transactions, physical location, movement trackers Competitive product review Similar and existing products What are the good and bad ideas? INF2002 Analyzing & Reporting Data Digest and represent the data Infer results from collected data Use figures, charts, statistics and descriptions Use data to improve new design INF2002 Summary Define the detailed requirements for the system being developed Gather data about what users need to do or accomplish Analyze and represent data Interpret the results and use that in design decisions INF2002 Methods Summary Technique Description Data collected Pros Cons Direct Both quantitative ? ? ? Observation and qualitative Indirect Observation No disturbance to user, can be done ? ? ? for extended periods Surveys & ? ? ? ? Questionnaires Interviews ? Mostly qualitative ? ? Focus Groups Mostly qualitative, ? ? ? less quantitative INF2002 Video Observation Rich’s Commute (Stanford D.School video) Write down everything that you hear that is important to the person being interviewed INF2002 Video Debrief 1. What did you learn about the user? 2. What surprised you about the user? 3. Did the user ever contradict themselves? 4. If you could ask additional questions what would you ask? 5. How can we use what we learned in this video to inform designing for this user? INF2002