URBS 260- 06B Data Collection (1).pdf

URBS 260 ANALYTICAL METHODS IN URBAN STUDIES W6B DATA COLLECTION TEACHER: DONNY SETO 2 TODAY’S AGENDA Data Collection 1. Secondary Data 2. Collecting Primary Data 1. Question types 2. Coding Frame 3. Methods 1. Self Completion Questionnaires 2. Interviews 3. Standardized Testing 4. Obse...

URBS 260 ANALYTICAL METHODS IN URBAN STUDIES W6B DATA COLLECTION TEACHER: DONNY SETO 2 TODAY’S AGENDA Data Collection 1. Secondary Data 2. Collecting Primary Data 1. Question types 2. Coding Frame 3. Methods 1. Self Completion Questionnaires 2. Interviews 3. Standardized Testing 4. Observation 3 SECONDARY DATA CRITERIA FOR ASSESSING THE QUALITY OF DOCUMENTS AND DATA Scott (1990) States 4 ways to measure Quality of Documents: • Authenticity • Genuine and of unquestionable origin • Credibility • Free from error or distortion • Representativeness • Typical of what it is supposed to represent • Meaning • Clear and comprehensible Smithsionian Channel, (2018) America in Color: These Men Risked Their Lives to Build 1920 New York Skyscrappers [Video File] Retrieved from: https://www.smithsonianchannel.com/videos/thesemen-risked-their-lives-to-build-1920s-new-york-skyscrapers/56921 PERSONAL DOCUMENTS: DIARIES, LETTER, AUTOBIOGRAPHIES Authenticity • Did the ‘author’ actually write the piece? Credibility • Is the report factual? • Does it reflect the true feelings of the author? Representativeness • Do the documents accurately reflect people of different classes and genders? Meaning • May be somewhat elusive 6 PHOTOGRAPHS Scott (1990) maintains that family photographs are of three types: • Idealization: formally posed • Natural portrayal: candid, not posed • Demystification: revealing, atypical portrayal Representativeness • The photographs available may not be typical of the event or episode depicted. • What is not photographed may be very significant. • Selection biases (filtering) • What is chosen to be photographed • What is chosen to be kept Photographs can be • used to illustrate points; • sources of data; • used as prompts to get people to talk. Social Context of Photographs • Ask the following questions to answer Authenticity, Credibility & Meaning • When were they taken? • Where were they taken? • Of whom were they taken? Vox (May 30, 2017) How Obsessive Artists Colorize Old Photos [Video Files]- https://www.youtube.com/watch?v=vubuBrcAwtY&feature=youtu.be GOVERNMENT DOCUMENTS • Includes census information, voting records, official reports, Hansard, etc. • Credibility, representativeness, and meaning are sometimes at issue. Statistic Canada (2016) OFFICIAL STATISTICS Includes government data, e.g., Statistics Canada, police statistics, etc. • • Advantages • Often based on populations, not samples. • Reactivity may be less pronounced because participants 0are not part of a study. • Chance for longitudinal and cross-cultural (international) studies Disadvantages • They record only individuals who have been processed by a government agency. • e.g., crime statistics: much crime goes unrecorded • Not reported • Perpetrator cautioned instead of charged • Discovered through police surveillance but no actual witness • The ‘dark figure’ in crime statistics 10 SECONDARY DATA ADVANTAGES SECONDARY DATA LIMITATIONS 13 PRIMARY DATA QUESTIONS OPEN OR CLOSED QUESTIONS? • One of the most significant considerations in survey research is whether to ask a question in an “open” or “closed” format • With an open question, respondents can reply however they wish • With a closed question, they are presented with a set of fixed alternatives from which they have to choose an appropriate answer OPEN QUESTIONS • Advantages • Allow for replies that the survey researcher might not have contemplated • Make it possible to tap the participants’ unprompted knowledge • Salience of particular issues for respondents can be examined • Can generate fixed-choice format answers • Disadvantages • More time consuming • Answers must be coded • Less convenient to compose an answer • May require transcribing • Face inter-interviewer variability CLOSED QUESTIONS • Advantages • Minimize intra-interviewer variability and inter-interviewer variability • May make it easier to understand question because the answers are provided • Closed questions can be answered quickly and easily, which reduces response rate issues • Disadvantages • Loss of spontaneity and authenticity because relevant answers may be excluded from the choices provided • • Respondents may differ in their interpretation of the wording of fixed responses • • Use open questions to generate the categories e.g., the meaning of “strongly” in “strongly agree” Respondents may not find a fixed response that they feel applies to them ISSUES WITH CLOSED QUESTIONS • Some prominent sources of error in survey research are: • poorly worded questions • interviewer error in asking a question • misunderstanding on the part of the interviewee • interviewee lapses in memory • interviewer error in recording information • mistakes in entering the data into a computer file • biases related to the characteristics (such as gender or “race”) of the interviewers and/or the interviewees 19 CODING FRAME CODING OPEN QUESTIONS CODING QUESTIONS Coding may be used in both qualitative and quantitative research. • It involves making themes or categories of behaviour to form the basis for codes (labels or titles given to the themes or categories). These themes or categories are derived from the collected data. There are three basic principles for coding in research: 1. The categories must not overlap with one another. 2. All possibilities that exist in the data must be exhausted. 3. There should be clear rules for what is included in categories and what types of answers relate to each category. CLOSED QUESTIONS 23 SELF-QUESTIONNARIE RULES FOR DESIGNING QUESTIONNAIRE • Keep the research questions in mind • Focus on exactly what you want to know • Be specific • e.g., standard of living, you would ask, how many children are living at home vs. how many children do you have? • Put yourself in the position of the respondent • How would you answer the question? TYPES OF QUESTIONS DESIGNING THE QUESTIONNAIRE DESIGNING THE QUESTIONNAIRE STRUCTURED INTERVIEWS STRUCTURED INTERVIEWS Interview that follows an Interview schedule • a formal list of questions that the interviewer must follow in detail • The questions must be asked in the order given • Structured interviews are used because they produce standardization in the asking of questions and the recording of answers . VARIATION SOURCES Intra-interviewer variability: an interviewer is not consistent in asking questions or recording answers (with the same respondent or a different one). Inter-interviewer variability: lack of consistency in asking questions or recording answers between different interviewers. INTERVIEWER EFFECTS • The characteristics of the interviewer may influence the responses given. • Sex, social class, and race of interviewer are key reactive issues. WHY STRUCTURED INTERVIEWS? • Reduces the chance of variation due to error on the part of the interviewer/interview process. • Enhances the accuracy and ease of data processing, because of the use of ‘closed’ or ‘fixed choice’ questions. • The interviewees ultimately put themselves into categories when they select answers 33 SEMI-STRUCTURED • The researcher has a list of questions or topics to be covered. • Offers a somewhat longer interview guide. • Interviewees are free to reply in any way they choose. • Questions may be asked out of order. • New questions may be devised and asked on the spot. • There is a clear focus on the topic of interest. • Useful when more than one interviewer. • Useful when there are several interviewees. SEMI-STRUCTURED INTERVIEWING, 35 UNSTRUCTURED • The researcher uses only a brief set of points to introduce topics. • The only thing defined is the broad topic of interest • Conversational • No more than a short interview guide • May be only a single question PREPARING AN INTERVIEW GUIDE • What is interesting about this issue? • Always open to new issues that may arise. • Unstructured • • Short list of issues Semi-Structured • Longer list of actual questions • Establish some degree of loose order to the questioning process • Broad questions related to the research issues • Language that is understood by the participants • No leading questions • Prompts to ensure sufficient personal information about the individual participant is collected to contextualize the data BEFORE THE INTERVIEW • Familiarize oneself with the setting to contextualize the data. • Have a good tape recorder and use it. • Use a quiet and private setting for interviews. • Use good interview techniques (active listening). • Be knowledgeable about the topic of the study; clear in asking simple and understandable questions • Be gentle, sensitive, and open, so that the interviewee can freely express their opinion and not be interrupted • Be flexible and steering: flexible enough to respond to the new themes raised by the participant, but steering to redirect discussion KINDS OF QUESTIONS • Introducing questions • ‘Please tell me about when your interest in X first began.’ • Follow-up questions • Elaborate on an answer. • ‘What do you mean by that?’ • Probing questions • Follow up what has been said through direct questioning. • ‘Can you say some more about that?’ • Specifying questions • ‘What did you do then?’ KINDS OF QUESTIONS • Direct questions • ‘Do you find it easy to keep smiling when serving customers?’ • Best left until later in the interview, in order not to influence the direction of the interview too much. • Indirect questions • (‘How do you feel about . . . ?’) and then ‘What do most people around here think of the ways that management treats its staff?’ CONDUCTING INTERVIEW FACE TO FACE • Face-to-face interviewing is the preferred method in academic research. • Data gathered through face-to-face interviews is argued to be a superior quality to that of telephone interviews • More than one interviewer is rare (cost) • Focus groups are more common but not always desirable • More desirable is the one-on-one interview • In a traditional interview, an interviewer stands or sits in front of the respondent • Asks a series of questions • Writes down or keys in the answers. • Good for academic social research TELEPHONE INTERVIEWS Strengths • Cheaper and quicker to administer, especially where the respondents are geographically spread out. • Easier to supervise to reduce interviewer errors up front. • Reduced issue with confidentiality, that exists with taped interviews. • Reduced bias arising from ‘interviewer effect’. Weaknesses • May exclude people without telephones (poor), ‘unlisted numbers’, cell phones, and the hearing impaired unless a computer random-digit-dialling program is used. (This is prohibited by law in some areas.) • Hard to sustain for long periods of time (20–25 minutes). • Personal interviews tend to be more effective with sensitive issues. • Less peripheral information available to work with to assess for understanding and other social conditions. • Difficult to be sure targeted respondent is the person actually answering the questionnaire. • No visual aids can be used to assist the interview. NUMERIS? Source: dennis.beier. (2014) Numeris [Video file]. Retreived from https://vimeo.com/110159066 44 DETACHED AND PARTICIPANT OBSERVATION OBSERVATION RESEARCH Complete observer – the observer takes a detached stance by not getting involved in the events, and uses unobtrusive observation techniques and remains ‘invisible’ either in fact or in effect (i.e. by being ignored). Observer-as-participant – the researcher is mainly an interviewer doing some degree of observation but very little participation. Participant-as-observer – the researcher engages fully in the life and activities of the observed, who are aware of his/her observing role. Complete participant – the researcher takes a full part in the social events but is not recognized as an observer by the observed. The complete participant is a covert observer. • TEST YOUR AWARENESS : WHODUNNIT? WHAT IS ETHNOGRAPHY? • Ethnography is a study of people and their culture in naturally occurring settings • Behaviour is observed in an unstructured way by carrying out indepth discussions and interviews with the people studied ETHNOGRAPHY: ELLEN ISAACS AT TEDXBROADWAY TEDx Broadway, (2013 March 1) Ethnography: Ellen Isaacs at TEDxBroadway [Video file]. Retrieved from https://www.youtube.com/watch?v=nV0jY5VgymI OVERT VERSUS COVERT ETHNOGRAPHY • Overt ethnography: The people being studied know they are being observed by a researcher • Covert ethnography: The people being studied do not know they are being observed by a researcher • Has many ethical concerns and is difficult to implement; overt ethnography is preferable ROLES FOR ETHNOGRAPHERS