Evaluation Terms Quiz
32 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the difference between data analytics and web analytics?

  • Data analytics measures website traffic, while web analytics draws inferences from large volumes of raw data.
  • Data analytics is used to analyze click data, while web analytics predicts user behavior.
  • Data analytics draws inferences from large volumes of raw data, while web analytics measures website traffic through analyzing users’ click data. (correct)
  • Data analytics and web analytics are the same thing.

    • What is ecological validity?

  • How well a method produces the same results on separate occasions under the same circumstances.
  • Whether the evaluation method measures what it is intended to measure.
  • How many people are involved in evaluating a product.
  • How the environment in which an evaluation is conducted influences or even distorts the results. (correct)

    • What is expert review?

  • When knowledge of typical users is applied, often guided by heuristics, to identify usability problems.
  • When someone with usability expertise and knowledge of the user population reviews a product for potential problems. (correct)
  • When hypotheses about an interface or other dimension are tested through controlled experiments.
  • When hundreds, thousands, or even millions of people evaluate a product or take part in an experiment.

    • What is the purpose of informed consent forms?

    <p>To describe the study and participants' rights.</p> Signup and view all the answers

    What is the purpose of formative evaluation?

    <p>To check that the product fulfills requirements and continues to meet users’ needs.</p> Signup and view all the answers

    What is the difference between reliability and validity?

    <p>Reliability refers to how well a method produces the same results on separate occasions under the same circumstances, while validity concerns whether the evaluation method measures what it is intended to measure.</p> Signup and view all the answers

    What is crowdsourcing?

    <p>When hundreds, thousands, or even millions of people evaluate a product or take part in an experiment, either in person or online.</p> Signup and view all the answers

    What is analytical evaluation?

    <p>When models are used to predict user behavior.</p> Signup and view all the answers

    What is heuristic evaluation?

    <p>When someone with usability expertise and knowledge of the user population reviews a product for potential problems.</p> Signup and view all the answers

    What is bias?

    <p>When the results are distorted, and can happen when expert evaluators, researchers, or interviewers selectively gather data or influence responses.</p> Signup and view all the answers

    What are controlled experiments used for?

    <p>To test hypotheses about an interface or other dimension.</p> Signup and view all the answers

    What is the purpose of formative evaluation?

    <p>To check that the product fulfills requirements and continues to meet users’ needs.</p> Signup and view all the answers

    What is the key difference between data analytics and web analytics?

    <p>Data analytics draws inferences from large volumes of raw data while web analytics measures website traffic</p> Signup and view all the answers

    What is the purpose of analytical evaluation?

    <p>To predict user behavior</p> Signup and view all the answers

    What is the purpose of controlled experiments?

    <p>To test hypotheses about an interface or other dimension</p> Signup and view all the answers

    What is the purpose of crowdsourcing in evaluation?

    <p>To evaluate a product through the input of a large number of people</p> Signup and view all the answers

    What is ecological validity?

    <p>The influence of the evaluation environment on the results</p> Signup and view all the answers

    What is expert review or crit?

    <p>A method of evaluation that involves someone with usability expertise and knowledge of the user population reviewing a product for potential problems</p> Signup and view all the answers

    What is the purpose of formative evaluation?

    <p>To check that the product fulfills requirements and continues to meet users’ needs</p> Signup and view all the answers

    What is heuristic evaluation?

    <p>A method of evaluation that involves applying knowledge of typical users to identify usability problems</p> Signup and view all the answers

    What is the purpose of informed consent forms?

    <p>To describe the study and participants' rights</p> Signup and view all the answers

    What is reliability?

    <p>The ability of an evaluation method to produce the same results on separate occasions under the same circumstances</p> Signup and view all the answers

    What is validity?

    <p>The degree to which an evaluation method measures what it is intended to measure</p> Signup and view all the answers

    What is bias?

    <p>The distortion of results due to selective gathering of data or influencing responses</p> Signup and view all the answers

    What is the difference between qualitative and quantitative data analysis?

    <p>Qualitative analysis involves non-numerical data and subjective interpretation, while quantitative analysis involves numerical data and statistical methods.</p> Signup and view all the answers

    What is the purpose of data cleansing?

    <p>To check for any anomalies that might be erroneous.</p> Signup and view all the answers

    What is the common mistake in data analysis, interpretation, and presentation?

    <p>Making claims that go beyond what the data can support.</p> Signup and view all the answers

    What is the difference between mean, median, and mode?

    <p>Mean is the average value, median is the middle value, and mode is the most frequent value.</p> Signup and view all the answers

    What is the difference between inductive and deductive analysis?

    <p>Inductive analysis involves letting the themes emerge from the data itself, while deductive analysis involves choosing a frame beforehand.</p> Signup and view all the answers

    What is the purpose of critical incident analysis?

    <p>To identify significant subsets of data for more detailed analysis.</p> Signup and view all the answers

    What is the purpose of grounded theory?

    <p>To construct a theory around the phenomenon of interest using empirical data of any kind.</p> Signup and view all the answers

    What is the focus of interaction analysis?

    <p>To analyze verbal and non-verbal interactions between people and artifacts to understand how knowledge and action are used within an activity.</p> Signup and view all the answers

    Study Notes

    Defining Evaluation Terms and Other Considerations

    • Data analytics is used to draw inferences from large volumes of raw data, while web analytics is used to measure website traffic through analyzing users’ click data.
    • Analytical evaluation models and predicts user behavior, and can refer to heuristic evaluation, walk-throughs, modeling, and analytics.
    • Controlled experiments are used to test hypotheses about an interface or other dimension, with aspects such as tasks, time, and environment controlled.
    • Crowdsourcing involves hundreds, thousands, or even millions of people evaluating a product or taking part in an experiment, either in person or online.
    • Ecological validity refers to how the environment in which an evaluation is conducted influences or even distorts the results.
    • Expert review or crit involves someone with usability expertise and knowledge of the user population reviewing a product for potential problems.
    • Formative evaluation is done during design to check that the product fulfills requirements and continues to meet users’ needs.
    • Heuristic evaluation involves applying knowledge of typical users, often guided by heuristics, to identify usability problems.
    • Informed consent forms describe the study and participants' rights, and are mandatory in many universities and major organizations.
    • Reliability refers to how well a method produces the same results on separate occasions under the same circumstances, while validity concerns whether the evaluation method measures what it is intended to measure.
    • Ecological validity concerns how the environment in which an evaluation is conducted influences or even distorts the results.
    • Bias occurs when the results are distorted, and can happen when expert evaluators, researchers, or interviewers selectively gather data or influence responses.

    Data Analysis, Interpretation, and Presentation

    • Qualitative and quantitative data and analysis are different.

    • A combination of qualitative and quantitative approaches is common in data analysis.

    • Initial reactions or observations from the data involve identifying patterns or calculating simple numerical values such as ratios, averages, or percentages.

    • Data cleansing is important to check for any anomalies that might be erroneous.

    • Interpretation of the findings often proceeds in parallel with analysis, but it is important to make sure that the data supports any conclusions.

    • Making claims that go beyond what the data can support is a common mistake in data analysis, interpretation, and presentation.

    • Finding the best way to present findings depends on the goals and audience for whom the study was performed.

    • Quantitative data is in the form of numbers, while qualitative data is in the form of words and images.

    • All forms of data gathering may result in qualitative and quantitative data.

    • Quantitative analysis uses numerical methods, while qualitative analysis focuses on the nature of something and can be represented by themes, patterns, and stories.

    • Initial processing of data involves transcription, expansion of notes, entry of answers to close-ended questions into a spreadsheet, clean up data, filter into different data sets, synchronization between data recordings, and records of behavior.

    • Basic quantitative analysis techniques include averages and percentages, and there are three types of average: mean, median, and mode.Basic Data Analysis Techniques

    • Data analysis is the process of examining data to extract meaningful insights and conclusions.

    • There are two basic approaches to data analysis: quantitative analysis and qualitative analysis.

    • Quantitative analysis involves numerical data and statistical methods, while qualitative analysis involves non-numerical data and subjective interpretation.

    • Quantitative data can be analyzed using spreadsheet software like Excel or Google Sheets, while qualitative data requires manual analysis and interpretation.

    • The design of survey questions affects the kind of analyses that can be performed and the conclusions that can be drawn.

    • Close-ended questions with fixed answer options can be analyzed quantitatively, while open-ended questions require qualitative analysis.

    • Basic quantitative analysis involves finding averages, identifying outliers, and producing graphical representations of the data.

    • Data visualization tools like heatmaps can create more sophisticated representations of the data.

    • Outliers are usually removed from the main data set because they distort general patterns, but they may also be interesting cases to investigate further.

    • Basic qualitative analysis involves identifying themes, categorizing data, and analyzing critical incidents.

    • Critical incident analysis is a way to isolate subsets of data for more detailed analysis.

    • The first step in qualitative analysis is to gain an overall impression of the data and start looking for interesting features, topics, repeated observations, or things that stand out.Basic Qualitative Data Analysis Techniques

    • Qualitative data can be analyzed inductively or deductively, depending on the data obtained and the goal of the study.

    • The objective of qualitative data analysis is to classify elements of the data in order to gain insights toward the study’s goal.

    • Identifying themes (thematic analysis) takes an inductive approach, while categorizing data takes a deductive approach.

    • Analysis is often performed iteratively, and it is common for themes identified inductively then to be applied deductively to new data.

    • Reliability of the analysis can be achieved by training another person to do the coding and calculating the inter-rater reliability.

    • Using more sophisticated analytical frameworks to structure the analysis of qualitative data can lead to additional insights that go beyond the results of basic techniques.

    • Thematic analysis is a widely used analytical technique that aims to identify, analyze, and report patterns in the data.

    • After an initial pass through the data, the next step is to look more systematically for themes across participants’ transcripts, seeking further evidence both to confirm and disconfirm initial impressions in all of the data.

    • An overall narrative is starting to emerge from the set of themes, and some of the original themes may not seem as relevant and can be removed.

    • Affinity diagrams are a common technique for exploring data, identifying themes, and looking for an overall narrative.

    • Inductive analysis is appropriate when the study is exploratory, and it is important to let the themes emerge from the data itself.

    • Categorization is appropriate when the analysis frame is chosen beforehand, based on the study goal.Analytical Frameworks for Qualitative Data Analysis

    • Usability studies can use think-aloud protocols to identify usability problems and categorize them for analysis.

    • Critical incident analysis helps to identify significant subsets of data for more detailed analysis by focusing on incidents that are significant or pivotal.

    • Six different analytical frameworks can be used for qualitative data analysis, ordered by granularity.

    • Conversation analysis focuses on recordings of spoken conversations to provide insights into how conversations are managed and progress.

    • Discourse analysis focuses on recordings of speech or writing to understand how words are used to convey meaning.

    • Content analysis examines any form of text to determine the frequency of items appearing in a text at varying levels.

    • Interaction analysis uses video recordings to analyze verbal and non-verbal interactions between people and artifacts to understand how knowledge and action are used within an activity.

    • Grounded theory constructs a theory around the phenomenon of interest using empirical data of any kind.

    • Systems-based frameworks focus on large-scale and heterogeneous data involving people and technology to provide insights about organizational effectiveness and efficiency.

    • Tuomas Kari et al. used critical incident analysis to identify types of behavior change induced in players of the game Pokémon GO.

    • Elise Grison et al. used critical incident analysis to investigate factors influencing travelers' choices of transport mode in Paris.

    • In a task of identifying the next available theater performance or movie to attend, critical incidents may include searching social media, discovering a favorite movie is playing, and needing a credit card to purchase tickets.

    Studying That Suits You

    Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

    Quiz Team

    Related Documents

    chap9.pdf

    Description

    Test your knowledge on evaluation terms and considerations with this informative and engaging quiz. From data analytics to ecological validity, this quiz covers various evaluation models and techniques used in the field of user experience. Whether you're a seasoned UX professional or just starting out, this quiz will provide valuable insights into the evaluation process and how to ensure reliable and valid results. So, put your thinking cap on and see if you can ace this evaluation terms quiz!

    More Like This

    Class XI Accountancy Evaluation-1
    5 questions

    Class XI Accountancy Evaluation-1

    WieldyAmazonite avatar
    WieldyAmazonite
    Valuation Terms Quiz
    22 questions

    Valuation Terms Quiz

    SustainableTurkey avatar
    SustainableTurkey
    Research, Evaluation, and Policy Key Terms Quiz
    34 questions

    Research, Evaluation, and Policy Key Terms Quiz

    FondMonkey75 avatar
    FondMonkey75
    Key Aspects of Summative Assessment Quiz
    20 questions

    Key Aspects of Summative Assessment Quiz

    ThumbsUpKnowledge avatar
    ThumbsUpKnowledge
    Use Quizgecko on...
    Browser
    Open
    Browser
    Browser