Data Collection Methods - Module II.4 PDF

Course structure i. Foundations ii. The empirical research process 1. Research question, theory, and hypotheses 2. Research design 3. Sampling and measurement 4. Data collection 5. Hypotheses testing 6. Dissemination iii. Research ethics...

Course structure i. Foundations ii. The empirical research process 1. Research question, theory, and hypotheses 2. Research design 3. Sampling and measurement 4. Data collection 5. Hypotheses testing 6. Dissemination iii. Research ethics Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 1 Learning goals of this module  Learning goals – … get to know the different data types – … create surveys – … conduct interviews – … use secondary data  Readings – Trochim, W., Donnelly, J. P., & Arora, K. (2020). Research Methods: The Essential Knowledge Base. Chapters 7 and 3 Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 2  Technische Universität München Data types Classification of research data Primary data: data that is newly collected for the purpose of Primary data the study vs Secondary data Secondary data: data that has already been collected, often for some other purpose and by third-parties Self-reported data: data that is provided by the subject Self-reported vs Observed Observed data: data that is collected through studying the subject’s behavior Quantitative data: data that is represented numerically Quantitative vs Qualitative data Qualitative data: data that cannot readily be expressed numerically (e.g., interview transcripts, email correspondence) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 4  TUM School of Management Technische Universität München Question to you  What are drawbacks of self-reported data?  Lack of accuracy / honesty  Introspective ability  Difficulties in interpretation of questions Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 5  Technische Universität München Data collection methods: surveys Definition and forms of surveys Survey: A measurement tool used to gather self-reported data from subjects by asking questions  Types of survey research  Questionnaire: instrument that is typically completed by the subject (self- administered)  Interview: researcher asks the participants questions and then goes on to complete the instrument based on their answers (researcher- administered) (Bryman and Bell (2011), p. 54) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 7 Questionnaire vs interview: considerations Questionnaire Interview           Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 8 Survey types In-home/ office Interview/surveying at where the subject is based (at home, at the office). May also be computer-supported Mall-intercept Ask subjects to participate in study while at the mall/shopping center Phone Interview Via phone Group Survey Survey is sent to key contact person (usually with prior agreement) who distributes to a group (e.g., a group of school children) Drop-off Subjects are handed the survey, may return at their on time Online Internet tools are used to have participants complete the survey online Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 9 Questionnaires: design  Structure – Invitation (letter) – Welcome: encourage and instruct respondents – Legitimation – Incentive: results, prize draw, … – Provide roadmap for the questionnaire – Actual questionnaire  Keep in mind – Must be clear because no clarification possible – Trust must be established because you are asking for sensitive information (Peró et al. (2015)) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 10  TUM School of Management Technische Universität München Question to you  What are advantages of using eMail/Internet-based questionnaires over paper-based questionnaires, what are disadvantages? Advantages Disadvantages  Inexpensive to administer  Prone to manipulation  Fast results  Junk mail syndrome  Easy to modify  Problems with less knowledgeable computer users  Respondents can reply in their own time and write as much/little as they choose Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 11 Questionnaires: implementation with Qualtrics Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 12 Questionnaires: implementation with SoSci Survey Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 13 Questionnaires: checklist What questions about a certain phenomenon should be asked? Content Is it better to ask one or more questions about a certain phenomenon? Are the respondents sufficiently knowledgeable to answer the questions? Are open-ended or closed questions more appropriate? Format For closed questions: What answer options should be provided? How can scientific terms be translated into the language of the respondents? Wording How can the unambiguousness of questions be secured? Are the questions neutrally worded? Do the introductory questions motivate respondents participate in the survey? Sequence Does the sequence of questions create a pattern of certain answers? Does the questionnaire seem structured? Does the questionnaire appear professional? Impression Can the questionnaire be completed easily? Do the respondents understand the questions? Pretest Do the respondents go through specific cognitive patterns? (Homburg and Krohmer 2008, p. 43) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 14 Interviews […] Researcher develops a series of questions or a series of points of interest to discuss and explore with the interviewees (Quinlan et al. (2018), p. 153)  Characteristics – Less structure required (i.e., conversation-driven instead of fixed questions) – Dominance of open-ended questions – Substantial interest in the perspective of the interviewee (i.e., to better understand their opinion and rationales) – Encouragement of digressions (i.e., digressions can reveal relevant insights of which interviewer was not yet aware of) – Flexibility and freedom to change/expand interview during the conversation – Encouragement of giving detailed answers – Participating role of the interviewer (i.e., requesting clarification or examples) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 15 Strengths of interviews  Establish a rapport with interviewee  Observe interviewees’ responses to questions  Individual perspectives and experiences  Addressing sensible topics Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 16 Role of the interviewer  Locate and enlist cooperation of respondents  Motivate respondents  Clarify any confusion/concerns  Observe quality of responses Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 17 Questionnaires: checklist Phase in which the relation between the discussion partners is established Arrival Start with small-talk Clearly define your objective and your methods Presentation of Research Guarantee confidentiality Project Request permission for recording the conversation Inquires about background information (e.g., age, education, company size) for interview adaptation Introductory Question Ask about issues that are of special interest for the interview (avoid that these issues overshadow the entire interview) Conduct the main part of the interview, in terms of asking pre-defined Main Part questions or follow-up questions based on the answers of the interviewee Indicate approaching to the end of interview 5 to 10 minutes before the actual conclusion Closing the Conversation Ask if all the focal aspects were sufficiently covered Thank for cooperation and emphasize data confidentiality Make notes, e.g., about body language, interview environment, etc. Transcribe the interview Follow Up Request approval of transcript or quotes from interviewee Identify new questions and integrate them in the interview Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 18 Interviews: best practices  Pay attention and show interest  State open questions (“W-Questions”)  Emphasize that there are no right and wrong statements  Pay attention to the body language and the color of voice  Manage interview time  Short, precise questions  Allow time for answering questions  Refrain from personal inferences and assumptions  Use questions to state assumptions about what was said  Do not comment, interrupt, or summarize the answers  Use silence as a technique Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 19 Types of interviews  By number or type of interviewees – Individual – Group interviews  By media – Online interviews – In-person interviews  By interview method – Unstructured, semi-structured, fully structured – Expert interviews – Think-aloud – Critical-incident technique Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 20  TUM School of Management Technische Universität München Question to you  Which challenges could a researcher face when conducting interviews? – Requires the capacity to engage with interviewee in an emphatic way – Time-consuming – Can be exhausting Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 21 Expert interviews  Respondents make no statements about themselves but act as Arrival key informants  Specific purpose: reconstruction of expertise  Particular challenges: – Recruiting a sufficient number of experts – Threat of conversation being dominated by the expert – Selection of experts: corporate decision makers, consultants – Respondent bias – Potential abuse (e.g., to get information about competitors) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 22 Think Aloud-interviews  Respondents are assigned to a task with the instruction to freely articulate their thoughts while solving it – “Try to think aloud. I guess you often do so when you are alone and working on a problem.” (Duncker 1926)  Application area: exploration of cognitive processes – Cognitive processes when being exposed to advertisements – Cognitive processes when completing questionnaires – Processing of competitive information  Important: warm-up phase to get used to the research method  Simple tasks  Fifteen-minute deadline: if no adequate verbalization takes place, abort Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 23 Think Aloud-interviews  Diverse degrees of verbalization by respondents: – Level 1: Simple reproduction of thoughts – Level 2: Description of thoughts – Level 3: Explanation of thoughts (Ericsson and Simon 1984)  Important: warm-up phase to get used to the research method  Simple tasks  Fifteen-minute deadline: if no adequate verbalization takes place, abort Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 24 Systematic errors and their sources in survey research Systematic error (bias) Administrative Respondent error error Sample Data Interviewer Interviewer Non-response selection processing Response bias cheating errors error error error Unconscious Deliberate Social Common Auspices Interviewer Extremity Acquies- desirability method bias bias bias cence bias bias bias (Quinlan et al. 2018, p. 276) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 25  TUM School of Management Technische Universität München Question to you  What could be examples of research settings particularly prone to social desirability biases? Consumer survey on eco-friendly consumerism: Choosing products/services that are eco-friendly 93% Boycotting companies that act unethically towards the environment 91% Willingness to spend more on eco-friendly products/services 74% Participating in voluntary environmental activities 68% Persuading others to similar product/service consumption habits 66% Percent who strongly-somewhat agree Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 26  Technische Universität München Data collection methods: observation Observation  Observation: the systematic observation, recording, description, analysis and interpretation of subject’s behavior (Schouten and McAlexander 1995) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 28 Types of observations Researcher takes part in activity Participant Complete as observer participant Researcher’s Researcher’s identity is identity is revealed concealed Observer as Complete participant observer Researcher observes activity (Schouten and McAlexander 1995) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 29  TUM School of Management Technische Universität München Question to you  What are advantages of covert observations, what are challenges? Advantages of covert observation Disadvantages of covert observation  “Subject’s knowledge that (s)he is  Ethical problems participating in a scholarly search may  No informed consent by confound the investigator’s data” (Webb el participants al. 1966,p.13)  Violation of privacy  Awareness of the observation  Blowing the cover endangers the whole  Change in behavior, i.e., due to research project (anxiety of researcher) impression management  Access to organizations often tied to  Role adjustment specific qualifications  Guessing of the research  Strong time commitment necessary objectives and corresponding change in behavior  Condition changes  Changes that are done to enable the observation in the first place (Kepper (2008), p. 207) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 30 Data generated by observation  Primary observations: are those where you would note what happened or what was said at the time  Secondary observations: are statements by observers of what happened or was said  Experiential data: data on your perceptions and feelings as you experience the process you are researching (Schouten and McAlexander 1995) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 31 Objects of observation Phenomenon Example Arrival Physical action A worker’s movement during an assembly process Verbal behavior Statements made by airline travelers while waiting in line Expressive behavior Facial expressions, tones of voices, and forms of body language Spatial relations and Proximity of middle managers’ offices to the president’s locations office Temporal patterns Length of time it takes to execute a stock purchase order Physical objects Percentage of recycled materials compared to trash Verbal and pictorial records Number of illustrations appearing in a training booklet (Quinlan et al. (2018), p. 234) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 32 Quantifying observation: structured observation Behavior of the staff member Observed Comments Smiles and makes eye contact with the consumer yes Greets the customer in a friendly manner no Gives the customer undivided attention throughout the yes transaction Suggests extra items that have not been ordered by yes the customer Place items on clean tray with trayliner facing yes customer Ensures that customer is told where all relevant extras yes (e.g. cream, sugar) are located Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 33 Structured observation  It can be applied by assistants after suitable training in the use of the measuring instrument  May be carried out simultaneously in different locations or in parallel  The easier the observation instrument to use and understand, the more reliable the results will be  The method allows the collection of data at the time they occur in their natural setting  Helps secure information that most participants would ignore because to them it was too mundane or irrelevant  Research results are limited to overt action or surface indicators from which the observer must make inferences Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 34  Technische Universität München Data collection methods: secondary data Examples of secondary data Secondary data Docu- Multiple Survey mentary source Conti- Non- Time- Written Area nuous Ad hoc written series Censuses material based & regular surveys material based surveys (Saunders et al. 2006) Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 36 Reasons for secondary data  Norm: In some research fields, secondary data is the norm  Economic: Collecting primary data can be costly, saves time  Reliability: Third-party ensures data consistency and accuracy  Replicability: Use of common datasets among researchers helps replicate findings Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 37 Finding secondary data  WRDS  Destatis  Kaggle  EIKON  Orbis  USPTO  EPO  EBSCO  Scopus Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 38 Evaluating the quality of secondary data 1. Assess overall suitability of data to search question(s) and objectives Pay particular attention to: Measurement validity Coverage including unmeasured variables 2. Evaluate precise suitability of data for analyses needed to answer research question(s) and meet objectives Pay particular attention to: Validity Reliability Measurement bias 3. Judge whether the use data based on an assessment of costs and benefits in comparison to alternative sources If you consider the data to be definitely unsuitable, do not proceed beyond this stage Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 39 Questions Professorship for Innovation & Digitalization (Prof. Dr. Jens Förderer) 40