Doing Research (An Introduction) - Reading Notes (PDF)

lOMoARcPSD|31736681 Doing Research (an introduction) - Reading Notes Doing Research, An Introduction (Universiteit van Amsterdam) Studeersnel wordt niet gesponsord of ondersteund door een hogeschool of universiteit Gedownload door Eugene (eugenezinchenkostudies@gmail...

lOMoARcPSD|31736681 Doing Research (an introduction) - Reading Notes Doing Research, An Introduction (Universiteit van Amsterdam) Studeersnel wordt niet gesponsord of ondersteund door een hogeschool of universiteit Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 🧠 DRI Notes Course DRI - 3801D2QPVY Date Oct 19, 2020 Materials Sem/Block/Week B1 S1 Type Notes Chapter 1 Science, Society and Social Research Definitions Learning Objectives Chapter 2 The Process and Problems of Social Research Definitions Learning Objectives Chapter 3 Research Ethics Definitions Learning Objectives Chapter 4 Conceptualisation and Measurement Definitions Learning Objectives Chapter 5 Sampling and Generalizability Definitions Learning Objectives Chapter 6 Causation and Experimental Design Definitions Learning Objectives Chapter 7 Survey Research Definitions Learning Objectives Chapter 8 Qualitative Methods Definitions DRI Notes 1 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Learning Objectives Chapter 9 Unobtrusive Methods Definitions Learning Objectives Chapter 10 Evaluation and Mixed-Methods Research Definitions Learning Objectives Chapter 13 Summarizing and Reporting Research Definitions Learning Objectives Chapter 1: Science, Society and Social Research Definitions Science: a set of logical, systematic, documented methods for investigating nature and natural processes Social science: use of scientific methods to investigate individuals, societies and social processes Selective observation: choosing to look at only things that are in line with our preferences/beliefs Inaccurate observation: an observation based on faulty perceptions of empirical reality Over-generalisation: concluding unjustifiably that what is true for some cases is true for all cases Descriptive research: defining and describing social phenomena of interest; the primary focus of the first research about some issue; quantitative methods Exploratory research: seeks to find out how people get along in the setting under question, what meanings they give their actions, and what issues concern them; learning "what is going on here?" and to investigate social phenomena without explicit expectations; qualitative methods DRI Notes 2 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Explanatory research: seeks to identify causes and effects of social phenomena and to predict how one phenomenon will change or vary in response to variation in some other phenomena Evaluation research: examines programs, policies, or other efforts to affect social patterns; can include elements of descriptive, exploratory and explanatory research Quantitative methods: methods such as surveys and experiments that record variation in terms of amounts; data that are treated as quantitative are either numbers or attributes that can be ordered by magnitude Qualitative methods: methods such as participant observation, intensive interviewing, and focus groups that are designed to capture social life as participants experience it rather than in categories predetermined by the researcher Positivism: the belief, shared by most scientists, that there is a reality that exists quite apart from our own perception of it, that it can be understood through observation, and that it follows general laws Constructivism: methodology based on questioning belief in an external reality; emphasizes the importance of exploring the way in which different stakeholders in a social setting construct their beliefs Triangulation: the use of multiple methods to study a research question; also used to mean that use of two or more different measures of the same variable Learning Objectives Describe the four common errors in everyday reasoning Observing - selective observation Generalising - over generalisation Reasoning - illogical reasoning Re-evaluating - resistance to change in light of new ideas Define social science and identify its limitations DRI Notes 3 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Social science can be disadvantaged by the errors in everyday reasoning but they can be avoided by relying on scientific methods to investigate individuals, societies and social processes Identify the four types of social research Descriptive research - defining and describing Exploratory research - exploring social phenomena without explicit expectations Explanatory research - identify cause and effect relationships Evaluative research - examines programs using elements of the other 3 Explain the difference between the orientations in the following two pairs: quantitative/qualitative; positivist/constructivist Quantitative methods - data summarised using numerical methods Positivism uses quantifiable scientific methods to arrive at conclusions about cause-and-effect relationships Qualitative methods - data collected in a more elaborate way designed to capture meanings attached by participants Constructivism uses qualitative methods as it tries to understand how people see reality Triangulation is the use of multiple methods to further the depth of research by gaining the advantages of several methods Chapter 2: The Process and Problems of Social Research Definitions The research process DRI Notes 4 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Social research question: a question about the social world that is answered through the collection and analysis of first-hand, verifiable, empirical data. Hypothesis: a tentative statement about empirical reality, involving a relationship between two or more variables Variable: a characteristic or property that can vary (take on different values or attributes) Independent variable: a variable that is hypothesized to cause, or lead to, variation in another variable Dependent variable: a variable that is hypothesized to vary depending on, or under the influence of, another variable Direction of association: a pattern in a relationship between two variables—the values of variables tend to change consistently in relation to change on the other variable; the direction of association can be either positive or negative Inductive research: The type of research in which general conclusions are drawn from specific data Deductive research: the type of research in which a specific expectation is deduced from a general premise and is then tested Research circle: a diagram of the elements of the research process, including theories, hypotheses, data collection, and data analysis Replications: repetitions of a study using the same research methods to answer the same research question Theory: a logically interrelated set of propositions about empirical reality Anomalous findings: unexpected patterns in data Serendipitous findings: unexpected patterns in data, which stimulate new explanations, insights, or theoretical approaches Cross-sectional research design: a study in which data are collected at only one point in time DRI Notes 5 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Longitudinal research design: a study in which data are collected that can be ordered in time; also defined as research in which data are collected at two or more points in time Validity: when statements or conclusions about empirical reality are correct Measurement validity: when a meas9ure measures what we think it measures Generalizability: when a conclusion holds true for the population, group, setting, or event that we say it does, given the conditions that we specify Causal validity (internal validity): when a conclusion that A leads to or results in B is correct Authenticity: when the understanding of a social process or social setting is one that reflects fairly the various perspectives of participants in that setting Sample generalizability: when a conclusion based on a sample, or subset, of a larger population holds true for that population Cross-population generalizability (external validity): when findings about one group, population, or setting hold true for other groups, populations, or settings Learning Objectives Name the three characteristics of a good research question Feasible - possible within the time and resources available Socially important - the topic should be important to other people and society Scientifically relevant - focuses on issues that have not been resolved by existing research Discuss the role of theory in social research Increases the value of a social research project Helps reflect on broader social patterns through existing social theories Demonstrate how to search and review the research literature  Specify the research question DRI Notes 6 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681  Identify relevant databases to search  Create a list of tentative search terms  Narrow the search  Check titles and abstract  Locate the relevant articles Describe three key social research strategies and when they are best used Explanatory research Evaluate cause-effect relationships Quantitative methods Theory → data: deductive Exploratory research Determines how people express/interact Qualitative methods Data → theory: inductive Descriptive research Provides a picture Qualitative or quantitative Makes empirical generalisations on data Name and illustrate the three different longitudinal designs Trend study Samples are drawn from the same population (different units in sample) at 2 points in time Panel study The same sample is measured at 2 points in time Cohort study Samples selected from a population who have a common starting point DRI Notes 7 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Define the standards of measurement validity, generalizability, causal validity, and authenticity Measurement validity - when a measure is valid for what it is supposed to measure; researchers cannot assume that measures are valid without evidence of measurement validity Generalizability - two aspects → sample or cross-population generalizability; low generalizability leads to low external validity DRI Notes 8 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Causal validity - the internal validity or truthfulness of an assertion that A causes B Authenticity - when a study correctly reflects the different perspectives of participants in a setting Chapter 3: Research Ethics Definitions Debriefing: a researcher’s informing subjects after an experiment about the experiment’s purposes and methods and evaluating subjects’ personal reactions to the experiment Code of ethics: general principles of competence, integrity, and responsibility as well as respect for the rights, dignity, and diversity of others, including research participants, and being socially responsible and using research to contribute to the public good Conflict of interest: when a researcher has a significant financial stake in the design or outcome of his or her own research Informed consent: must be given by persons who are competent to consent (usually over 18 years), who have consented voluntarily, who are fully informed about the research and know who is conducting the research, and who have comprehended what they have been told Deception: Used in social experiments to create more “realistic” treatments in which the true purpose of the research is not disclosed to participants, often within the confines of a laboratory Learning Objectives Describe some of the historical events involving research projects that have raised questions about the protection of human subjects Harmful experiments on Jews in Nazi concentration camps - mental and physical harm in the name of science DRI Notes 9 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Tuskegee syphilis study used personal information - the results of their syphilis tests - and failed to inform those who tested positive and did not provide treatment found from the experiment Identify the key ethical principles involved in conducting ethical research US Government: Belmont Report established 3 ethical principles: Respect for persons Treating persons as autonomous agents and protecting those with diminished authority Beneficence Minimising possible harms and maximising benefits Justice Distributing benefits and risks of research fairly List current standards for the protection of human subjects in research Research should cause no harm to subjects Participation in research should be voluntary Subjects must give their informed consent to participate in the research and researchers must disclose their identity Researchers should avoid deception, except in limited circumstances Anonymity or confidentiality must be maintained for individual research participants unless it is voluntarily and explicitly waived Researchers should consider the uses of a research project so that its benefits outweigh any foreseeable risks z Explain how an institutional review board operates and how it classifies research Federal regulations require that every institution that seeks funding for biomedical or behavioural research on human subjects have an institutional DRI Notes 10 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 review board IRB that reviews research proposals involving human subjects Research: a systematic investigation designed to develop or contribute to generalizable knowledge Human subject: a living individual about whom an investigator conducting research obtains data through intervention or interaction with the individual or jus identifiable private information IRBs apply ethical standards set by federal regulations The IRB has the authority to require changes in a research protocol or to refuse to approve a research protocol if it deems its human subjects protections inadequate To make IRBs fair, they include 5 members, with at least 1 non-scientist and a member from outside the institution; members must be from both sexes, diverse backgrounds, and multiple populations Chapter 4: Conceptualisation and Measurement Definitions Concept: a mental image that summarizes a set of similar observations, feelings, or ideas Indicator: the question or other operation used to indicate the value of cases on a variable Constant: a number that has a fixed value in a given situation; a characteristic or value that does not change Conceptualization: the process of specifying what we mean by a term; conceptualization helps translate portions of an abstract theory into specific variables that can be used in testable hypotheses Operationalization: the process of specifying the measures that will indicate the value of cases on a variable DRI Notes 11 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Level of measurement: the mathematical precision with which the values of a variable can be expressed Units of analysis: the level of social life on which a measure is focused, such as individuals, groups, towns, or nations Measurement validity: the extent to which measures indicate what they are intended to measure Learning Objectives Define and distinguish conceptualization and operationalization Operationalisation is the empirical counterpart of conceptualisation In variables: Defining → conceptualisation Defining measure → operationalisation Identify the different forms of single questions and response choices Single questions are frequently used in public opinion polls and social surveys Close ended (fixed choice) questions Open ended questions Responses need to be mutually Useful when asking questions to exclusive and exhaustive which answers cannot be predicted Easy to answer and process Harder to answer and process Obscure choices if it doesn't relate to the respondent If there is no other/don't know option, respondents may choose a random one - lower validity Give examples of the four levels of measurement  Nominal - variables whose values have no mathematical interpretation The values measured must be mutually exclusive and exhaustive - QUAL DRI Notes 12 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681  Ordinal - numbers assigned to cases which only specify the order of the cases High to low scales may use ordinal values, e.g. first, second, etc - QUAN  Interval - equal intervals among levels Numeric values specify distance between variables - QUAN  Ratio - like the interval level but also includes a 0 Numeric scale starting from zero - QUAN  Dichotomies are variables with only two values - categorical measurement Define the individual and group units of analysis and the ecological and reductionist fallacies The level of social measured Individual level analysis should be based on individual level data Group level analysis should be based on group level data Using group level data to analyse individual level processes - ecological fallacy Using individual level data to analyse group level processes - reductionist fallacy Compare the advantages and disadvantages of the four approaches to testing measurement validity Face validity: seeing if a measure 'looks' right Content validity: the measure covers the full range of the concept's meaning; measures different aspects of the same concept Criterion validity: when the scores obtained by one measure can be accurately compared with those obtained with a more direct measure of the same phenomenon Concurrent validity: when a measure yields scores that are closely related to scores on a criterion measured at the same time DRI Notes 13 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Predictive validity: the ability of a measure to predict scores on a criterion measured in the future Construct validity: validity that is established by showing that a measure is related to other measures as specified in a theory Chapter 5: Sampling and Generalizability Definitions Population: the entire set of individuals or other entities to which study findings are to be generalized Sample: a subset of a population that is used to study the population as a whole Elements: the individual members of the population whose characteristics are to be measured Sampling frame: a list of all elements or other units containing the elements in a population Sampling units: units listed at each stage of a multistage sampling design Sampling error: any difference between the characteristics of a sample and the characteristics of a population; the larger the sampling error, the less representative the sample Target population: set of elements larger than or different from the population sampled and to which the researcher would like to generalize study findings Representative sample: sample that “looks like” the population from which it was selected in all respects potentially relevant to the study; the distribution of characteristics among the elements of a representative sample is the same as the distribution of those characteristics among the total population Census: research in which information is obtained through responses from or information about all available members of an entire population Probability sampling method: a sampling method that relies on a random, or chance, selection method so that the probability of selection of population elements is known; no systematic bias DRI Notes 14 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Nonprobability sampling method: sampling methods that are not based on advance knowledge of the likelihood of selecting each element Non-respondents: people or other entities who do not participate in a study although they are selected for the sample Systematic bias: overrepresentation or underrepresentation of some population characteristics in a sample resulting from the method used to select the sample; a sample shaped by systematic sampling error is a biased samples Random sampling: sampling that relies on a random, or chance, selection method so that every element of the sampling frame has a known probability of being selected Learning Objectives Distinguish the two meanings of generalizability Cross population generalisability: studies form a study can be generalized to another, somewhat different population Sample generalizability: the findings from a sample population can be generalized to the population from which the sample was selected Identify the circumstances that make sampling unnecessary and the reasons they are rare If all individuals in a population are identical → no need for sample Rarely occurs since people are never identical When conducting a census → no need for sample Identify the relations among the desired sample, the obtained sample, the sampling frame, and sample quality Nonresponse undermines sample quality The sampling frame is based on the desired sample The desired sample is meant to provide the highest sample quality DRI Notes 15 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 The obtained sample determines the real sample quality The sampling frame is based on the desired sample Define and distinguish the major types of probability sampling method and indicate when each is preferred  Simple random sampling Choosing sampling units by lottery or random number generator Used for phone surveys - random digit dialling Process can long and time-taking for large samples Every element has an equal probability of selection  Systematic random sampling First element then every nth element is randomly selected from a list Elements do not need to be in a printed sampling frame but can be represented by folders in filing cabinets  Total population/number of cases required; if 50 is needed from 100, sampling interval is 20  Number from 1 to sampling interval (e.g. 1 - 20 is selected  First unit, then every nth unit is selected; if sampling interval is not a whole number the size is varied systematically (if interval - 30.5, 30th and 31st unit is alternatively chosen) This method does not work when in which the sequence of elements is affected by periodicity - when every nth element is affected by a different factor  Stratified random sampling Uses information about the total population to make the sampling process more efficient The population is divided into relevant strata and then elements are randomly chosen from each strata DRI Notes 16 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Ensures appropriate representation of elements across strata Proportionate stratified sampling ensures same proportion of individuals in sample as in population - no sampling error Disproportionate stratified sampling uses the strata to intentionally vary proportions of individuals in sample from population  Cluster sampling Useful when sampling frame is not available for samples taken from a population spread out over a large geographical area Multistage cluster sampling Greater sampling error: less clusters = higher sampling error Explain when nonprobability sampling methods may be preferred  Availability sampling Sampling in which elements are selected on the basis of convenience Easy to find Good for researching a particular setting Low generalizability  Quota sampling Nonprobability sampling method in which elements are selected to ensure that the sample represents certain characteristics in proportion to their prevalence in the population DRI Notes 17 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Intended to overcome the most obvious flaw of availability sampling Quotas are set to ensure that the sample represents certain characteristics in proportion to their prevalence in the population No way of knowing if the sample is representative for any other characteristics - researchers know what the population looks like relative to the characteristics relevant to their concerns  Purposive sampling Each sample element is selected for a purpose, usually because of the unique position of the sample elements  Snowball sampling Useful for hard-to-reach or hard-to-identify populations for which there is no sampling frame, but when the members are somewhat interconnected Researchers cannot be confident that a snowball sample represents the total population of interest, generalizations must be tentative Describe the concept of sampling error and explain how it is affected by the number of cases sampled, the heterogeneity of the population, and the fraction of the population included in the sample Random sample has no systematic bias but will have sampling error from chance Degree of sampling error can be mathematically calculated The size and homogeneity of a sample affect the degree of error as result of chance Larger the sample = higher representativeness More homogeneous population = higher representativeness The size of the sample affects representativeness, not the proportion of the population it is taken from DRI Notes 18 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Chapter 6: Causation and Experimental Design Definitions Causal explanation: an explanation that identifies common influences on a number of cases or events Causal effect: when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable Randomized comparative change design: pre-test post-test control group design Randomized comparative post-test design: post-test only control group design Spurious relationship: relationship between two variables that is caused by variation in a third (unconsidered) variable Extraneous variable: variable that influences both the independent and the dependent variables, creating a spurious association between them that disappears when the extraneous variable is controlled Quasi-experimental design: research design in which there is a comparison group that is very similar to the experimental group in critical ways, but subjects are not randomly assigned to the comparison and experimental groups Non-equivalent control group design: quasi-experimental research design in which experimental and comparison groups are designated before the treatment occurs but are not created by random assignment Before and after designs: experiments where there is no control group, all participants are exposed to the experimental condition Repeated-measures panel designs: the inclusion of several pre-test and post-test observations allows study of the process by which an intervention or treatment has an impact over time Ex-post facto control group design: when individuals can choose whether to be in the control or experimental group DRI Notes 19 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Time series design: the trend in the dependent variable until the date of the intervention or event whose effect is being studied is compared with the trend in the dependent variable after the intervention Between-subjects study design: different people test each condition, so that each person is only exposed to a single user interface Within-subjects study design: the same person tests all the conditions (i.e., all the user interfaces) Selection bias: source of internal (causal) invalidity that occurs when characteristics of experimental and comparison group subjects differ in any way that influences the outcome History effect: source of causal invalidity that occurs when events external to the study influence post-test scores; also called an effect of external events Differential attrition (mortality): problem that occurs in experiments when comparison groups become different because subjects are more likely to drop out of one of the groups for various reasons Contamination: source of causal invalidity that occurs when either the experimental or the comparison group is aware of the other group and is influenced in the post-test as a result Expectancies of experimental staff: source of treatment misidentification in experiments and quasi-experiments that occurs when change among experimental subjects results from the positive expectancies of the staff who are delivering the treatment rather than from the treatment itself; also called a self- fulfilling prophecy Double-blind procedure: experimental method in which neither the subjects nor the staff delivering experimental treatments know which subjects are getting the treatment and which are receiving a placebo Placebo: fake “treatment” given to a comparison group to make sure their experience is no different from that of the experimental group except for the actual treatment Hawthorne effect: A type of contamination in research designs that occurs when members of the treatment group change relative to the dependent variable DRI Notes 20 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 because their participation in the study makes them feel special Solomon four-group design: type of experimental design that combines a randomized pre-test post-test control group design with a randomized post test only design, resulting in two experimental groups and two comparison groups Solomon four-group design Learning Objectives List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection Belief that an independent variable X has produced a change in dependent variable Y, ceteris paribus → causal effect Conditions needed for determining causality: Empirical association Appropriate time order Non-spuriousness Improving understanding of a causal connection Mechanism Context Explain the meaning of the expression “correlation does not prove causation” An association between two variables might be caused by something other than an effect of the presumed independent variable on the dependent variable This is called a spurious relationship DRI Notes 21 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 List the essential components of a true experimental design Two (or more) groups → experimental and causal groups Allows for comparison between experimental and control groups Variation in the independent variable before post test Time order → variation in independent variable has to occur before the dependent variable Post test measures the outcome in both groups Random assignment to the comparison groups Ensures non-spuriousness by eliminating possible systematic bias No existence of a third variable not considered in the experiment Pre-test not required because the initial scores on the dependent variable should the same in all groups at the beginning due to random assignment Distinguish the concepts of random assignment (randomization) and random sampling Random assignment into control and experimental group(s) DRI Notes 22 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Random selection of a sample from a population Explain the advantages and disadvantages of using quasi-experimental and experimental designs Used when a true experimental design is not feasible with the desired participants in the desired setting An experiment is quasi experimental when control and experimental groups cannot be randomly assigned Define five threats to validity in research Causal (internal) validity Selection bias History effect Differential attrition Contamination Expectancies of experimental staff External validity (generalizability) Sample generalizability Cross-population Ecological validity generalizability Discuss the most distinctive ethical challenges in experimental research Deception: used to make experiments more "realistic" treatments, often in labs Selective distribution of benefits: how much researchers can influence the benefits that subjects receive as part of the treatment being studied in a field experiment Chapter 7: Survey Research Definitions DRI Notes 23 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Social desirability bias: the tendency to “agree” with a statement just to avoid seeming disagreeable, also known as agreement bias or acquiescence effect Idiosyncratic variation: variation in responses to questions that is caused by individuals’ reactions to particular words or ideas in the question Learning Objectives Identify the reasons for the popularity of survey research Versatility Efficiency Generalisability Can be used for Low cost and can be Can sample large practically all research done relatively quickly populations Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions Fence sitters: see themselves as being 'neutral' - make skew results when forced to choose between opposites Allows fence sitters to choose the right option Floaters: respondents who choose a substantive answer when they really don't know or have no opinion Many could also choose don't know because they want to move on List the different methods for improving survey questions For a survey to succeed it must minimise four types of errors: Poor Non response Inadequate Sampling error measurement sample Avoid confusing phrasing - short words and sentences Do not use double negatives and double barrelled questions Use filter questions to choose respondents who have relevant information Use a skip pattern to guide respondents to relevant questions DRI Notes 24 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Do not use words/phrases that induce biases or controversy Fixed response questions should be exhaustive Measure attitudes using Likert items using unipolar or bipolar scales Include interpretive questions to help the researcher understand what the respondent meant by their responses Maintain focus on the research objective The order in which questions are presented will influence how respondents answer the questionnaire; first question should link to the primary objective; try to avoid context effects Leave lots of white space between questions - makes it look easy to complete Translate the questionnaire for to fit the needs of the sample Outline a cover letter for a survey that contains each of the required elements Cover letter: critical to the success of a mailed survey; sets the tone for the questionnaire Credible: establish the purpose of the research and mention the conducting authority Personalised: include the name of the respondent Interesting: describe contents to interest the respondent Responsible: reassure that information will be treated confidentially, if they want the final report and include phone number List the strengths and weaknesses of each type of survey design, giving particular attention to response rates DRI Notes 25 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681  Mailed Central concern - maximising response rate (most questionnaires have 30% response rate but can be improved to 70%  Send brief letter before the survey is sent  Send questionnaire with a personalised cover letter, with if possible, monetary reward  Send postcard after 2 weeks thanking respondents and reminding non- respondents, friendly and must include phone number  Send replacement questionnaire to non respondents after 2 to 4 weeks with a new cover letter; let them know they haven't responded and stress on importance of the survey  Attempt to administer the survey using a different mode, e.g. phone survey  Self administered  Group administered Completed by individuals assembled as a group  In person Use the same interviewer and same set of questions for standardisation or use computer assisted personal interview Presence of interview can impact results  Web DRI Notes 26 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Best way to reach a very large sample relatively easily, except some parts of the population may not be reachable - coverage bias Very useful when email addresses are known Advantages for increasing measurement validity, can use various forms of media to clarify meaning Web surveys have a lower social desirability effect  Phone surveys Procedures can be standardised using computer assisted telephone interviews and interactive voice response survey technology Uses random digit dialling Landlines are rare nowadays and cell phone only interviews may only give access to a particular demographic, not representative Increase response using callbacks, monetary incentives, letters to non respondents, skilled interviewers - but response only increased for 110 to 15 DRI Notes 27 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Explain the rationale for a mixed-mode survey Combining different survey designs to improve overall participation rate Mixed mode surveys allow the strengths of one survey design to compensate for the weaknesses of another May reduce total costs - starting with the cheapest and then putting in more effort to improve it Discuss the key ethical issues in survey research DRI Notes 28 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Confidentiality and anonymity Should be anonymous if possible (if anon, hard to follow up) Information should be stored confidentially Harm to individuals should be identified and disclosed Chapter 8: Qualitative Methods Definitions Adaptive research design: research design that develops as the research progresses Reflexivity: sensitivity of and adaptation by the researcher to his or her influence in the research setting Field researcher: researcher who uses qualitative methods to conduct research in the field Reactive effects: changes in individual or group behaviour that result from being observed or otherwise studied Gatekeeper: person in a field setting who can grant researchers access to the setting Key informant: an insider who is willing and able to provide a field researcher with superior access and information, including answers to questions that arise in the course of the research Learning Objectives Describe the features of qualitative research that most distinguish it from quantitative research Collection of data that observes natural behaviour that captures social life as participants experience it, rather than in categories the researcher predetermines Exploratory research questions - inductive reasoning Focusses on previously unstudied processes and unanticipated phenomena DRI Notes 29 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 An orientation to social context and interconnections between social phenomena Focus on human subjectivity - meaning participants attach to phenomena Adaptive research design Sensitivity to the subjective role of the researcher - reflexivity Define the methods of case study research, ethnography, and netnography Ethnography Study of culture(s) shared by a group of people Naturalistic: seeks to describe and understand the natural world A researcher immerses themselves into a group for a long time to establish trust and gain first had experience of the culture Code of the street: the fabric of everyday life, the factors that influence how residents shape their everyday lives Netnography Use of ethnographic methods to study online communities Can focus on communities whose members are geographically very far Compare the advantages and disadvantages of each participant observer role Participant observation is a field research method; observes people in during their normal activities DRI Notes 30 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Covert observation: observe without participating, not disclosing that they're a researcher; may impact the actions of other in that setting; cannot take notes or ask questions, harder to play the role successfully Overt observation: complete observer, may lead to change in behaviour to create a problem of reactive effects; researchers have found that their presence is ignored after a while Overt participation: acknowledges role of researcher but also participates; can develop rapport with group members while maintaining research goals Complete (covert) participation: acts just like other members with disclosing that they're a researcher; cannot take notes or ask questions, harder to play the role successfully Discuss the major challenges at each stage of a field research project Entering the field May not gain access due to strict gatekeeper Could use personal contacts to enter the field This process lays the groundwork for collecting data, creating relationships Researcher should have a rationale as to why they should gain access to the field Developing and maintaining relationships DRI Notes 31 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Must be careful and manage relationships so they can continue to observe Use of a key informant to gain insight Sampling people and events Researcher should select a critical case that is unusually rich in information Sample should be adequate and informative, even if not representative A theoretical sample is drawn in a sequential fashion, with settings or individuals selected for study as earlier observations or interviews indicate that these settings or individuals are influential Managing personal dimensions Identifying and dealing with reflexivity Sharing similarities between group being studied may produce mutual feelings and can lead to bias in perspective Explain how field researchers take and analyse notes Take down jottings, less detailed notes describing highlights of events observed Later in the day, these notes should be made into more descriptive field notes A daily log should be used to record each day's events Describe the process of intensive interviewing and compare it to the process of interviewing in survey research Method of finding out about people's experiences, thoughts and feelings DRI Notes 32 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Uses open ended questions No fixed structure More driven by what the respondent mentions - conversation with a purpose Interviewer must plan their questions around a flexible outline Lengthy explanations and uses follow up questions Using a recorder to record the interview with little distraction Selection methods for long interviews are carefully considered - rarely random selection Respondents chosen until saturation point - when new interviews give no new information Important to establish rapport with the respondent Considering how they will react to questions Respecting their standards of social behaviour Appropriate pace of an interview Online interviews Can gain access to people regardless of location Can be synchronous (more like in person) or asynchronous (may lose interest after a while) Relative anonymity Loss of indicators like body language and rapport cannot be built in the same way DRI Notes 33 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Discuss the advantages of focus group research and identify particular challenges that focus group researchers face Qualitative and relatively unstructured 5 - 10 individuals recruited who share key characteristics with target population, to participate to gain access to insights (participants usually do not know each other) Mimics the natural process of forming and expressing opinions, with open- ended questions posed by the researcher (or group leader) Emphasise discovering unanticipated Not possible to conduct with large findings and exploring hidden representative samples meanings Consider how recruitment of the Good for developing hypotheses and sample can impact generalizability survey questions - investigating (external validity) meaning of survey results and quickly assessing the range of opinions about an issue Identify the major ethical challenges faced by qualitative researchers and discuss one qualitative research project that posed particular ethical concerns  Voluntary participation Subjects must be given a clear choice of whether they want to participate Using process consent to check whether participants want to continue Rapport between researcher and participants may mean they reveal sensitive information which the researcher may not want to be responsible for knowing  Subject well-being Harm to participants Negative feelings about publishing sensitive information learned DRI Notes 34 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681  Identity disclosure Consent may not be meaningful if researcher is covert Netnography can be controversial if researcher joins and analyses a group without informing members  Confidentiality Typically use fake names of participants in reports All identifiable information must be removed Focus groups are harder because participants could share information sensitive/private about other participants  Appropriate boundaries - cannot be controlled Maintaining distance from the subjects - not getting too involved  Researcher safety Researchers on the field may be exposed to unfamiliar or dangerous settings Chapter 9: Unobtrusive Methods Definitions Unobtrusive methods: research methods in which data are collected without the knowledge or participation of the individuals or groups that generated the data Unobtrusive measure: a measurement based on physical traces or other data that are collected without the knowledge of or participation of the individuals or groups that generated the data Secondary data analysis: the method of using pre-existing data in a different way or to answer a different research question than intended by those who collected the data Big data: massive data sets reflecting human activity that are accessible in computer readable form, available to social scientists, and manageable with today's computers DRI Notes 35 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Variable-oriented research: research that focuses attention on variables representing particular aspects of the cases studied and then examines the relations between these variables across sets of cases Learning Objectives Discuss the advantages of using unobtrusive measures of behaviour Can improve measurement validity by Most useful when looking at group avoiding the bias of self reports about behaviour, individual behaviour cannot socially desirable/undesirable be tracked using this method behaviour Allows researchers to investigate behaviour that has already occurred but which could not have been observed due to time, money or ethical reasons List four major sources of secondary data for social scientists Social science surveys and data sets Government agencies Books Experimental reports Explain the special cautions that secondary data analysts should be aware of in their research The researcher may need to change their research question to fit the secondary data they're using - the research design is not determined by the researcher but the data being used Data quality is always a concern with secondary data - the data may have been collected to reflect the interest of a certain group which may lead to DRI Notes 36 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 inconsistencies Research across national borders may mean that the methods used may not match which can lead to further confusion and inconsistencies Confidence in secondary analysis will increase if these questions can be answered  What were the agency's or researcher's goals in collecting the data?  What data were collected, and what were they intended to measure?  What specific methods were used for data collection?  What is known about the success of the data collection effort? How are missing data indicated? What kind of documentation is available? How consistent are the data with data available from other sources? Identify several sources of Big Data Social media platforms Global positioning system GPS Internet usage patterns Search engines Discuss the advantages and disadvantages of using Big Data to answer social research questions Very extensive and large volume of Potential problems in measurement or data available for use sampling - may overestimate certain trends when people search/write about them in relation to someone else Data may be used without telling those involved - privacy risks Usage of internet/social media may differ between different class/ethnic backgrounds DRI Notes 37 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Define the three basic types of historical and comparative research methods and give an example of each Qualitative and inductive Holistic and conjunctural - concerned with the context and interrelations Case oriented - focuses on cases as a between different events and whole, rather than specific aspect processes Temporal - looks at the related series Uses narrative explanations of events that unfold over time; specific to the time/place studied Problems with historical research: Documents and evidence may be lost Available evidence may be biased towards more successful/newsworthy figures Written records will be biased towards those who were prone to writing Feelings of individuals involved in past events may be impossible to reconstruct Comparative social science methods: used to make comparisons between countries May use quantitative variable oriented research Describe the process of content analysis and give one example Content analysis: the systematic, objective, quantitative analysis of message characteristics Starts with text, speech broadcasts or visual images Develops procedures for coding of the content DRI Notes 38 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 DRI Notes 39 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Discuss the major ethical challenges that arise in the use of unobtrusive methods Use of data/information from secondary sources with citation (plagiarism) All information that could lead to potential identification of persons dead/alive must be removed When using government provided information like the census, they could be inaccurate due to social or political pressures DRI Notes 40 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Big data may be used for surveillance and can be used identify individuals Chapter 10: Evaluation and Mixed-Methods Research Definitions Inputs: the resources, raw materials, clients, and staff that go into a program Program process: the complete treatment or service delivered by the program Outputs: the services delivered or new products produced by the program process Outcomes: the impact of the program process on the cases processed Feedback: information about service delivery system outputs, outcomes, or operations that can guide program input Stakeholders: Individuals and groups who have some basis of concern with the program Learning Objectives Define each of the types of evaluation research and explain when each is appropriate DRI Notes 41 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Needs assessment: tries to answer the following questions with systematic, credible evidence; assessed using social indicators Process evaluation: research to investigate the process of service delivery The term formative evaluation may be used when the evaluation findings are used to help shape and refine the program Impact analysis/evaluation: compares what happened after the program to what would have happened if the program wasn't there Efficiency analysis: identifies whether the program had more benefits than costs Describe the four main types of questions that evaluation research looks to answer Needs assessment: Is a new program needed or an old one still required Is there any need at all? Process evaluation: What actually happens in a social program? Questions regarding a program's implementation Impact analysis/evaluation: Did the program work? Did it have the intended result? Efficiency analysis: Whatever the programs benefits, does it offset the programs' costs? Write an argument supporting or opposing research to evaluate social programs Compare and contrast different mixed-methods designs Evaluation research projects often combine qualitative and quantitative methods to provide a more comprehensive picture of program operation or impact → mixed methods DRI Notes 42 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Integrated design: both methods are used concurrently and given equal importance Embedded design: both methods are used concurrently but one is given priority Staged design: both methods are used in sequence and one is given priority The primary method is written in capitals = QUAN or QUAL The secondary method is written in lowercase letters = quan or qual If both methods are given equal priority both are written in caps = QUALQUAN If one method is used before the other, the sequence is indicated with an arrow = QUAL→quan, or qual→QUAN, or QUAN QUAL If two methods are used concurrently, but one has priority, the secondary method is said to be “embedded” in the primary method = QUAL(quan) or QUAN(qual) If two methods are used concurrently, but they have equal priority, the relation between the two methods is indicated with a +: QUALQUAN. Discuss potential strengths and weaknesses of mixed-methods studies Enhancing measurement validity, Creates extra challenges for generalisability, causal validity or researchers as each method requires authenticity specific expertise Measurement validity is enhanced are quantitative and qualitative methods can be used to verify the methods and measurements Identify some of the ethical challenges in evaluation research Direct impact on research participants, and potentially their families, heightens the attention that evaluation researchers have to give to human subject concerns DRI Notes 43 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 When an ongoing entitlement program is being evaluated and experimental subjects would typically be eligible for program participation - not ethical to bar some potential participants from the programs There are many other ethical challenges in evaluation research: How can confidentiality be preserved when the data are owned by a government agency or are subject to discovery in a legal proceeding? Who decides what level of burden an evaluation project may tolerably impose on participants? Is it legitimate for research decisions to be shaped by political considerations? Must evaluation findings be shared with stakeholders rather than only with policy makers? Is the effectiveness of the proposed program improvements really uncertain? Will a randomized experiment yield more defensible evidence than the alternatives? To conduct research on children, parental consent usually is required - can dramatically reduce participation because many parents simply do not bother to respond to mailed consent forms Could use passive consent - returning form only in their lack of consent Chapter 13: Summarizing and Reporting Research Definitions Front matter: the section of an applied research report that includes an executive summary, an abstract, and a table of contents Back matter: the section of an applied research report that may include appendixes, tables, and the research instrument(s) DRI Notes 44 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 Plagiarism: presenting as one’s own the ideas or words of another person or persons for academic evaluation without proper acknowledgment Learning Objectives Identify unique problems that must be overcome in writing student papers, applied research reports, and journal articles If the intended audience does not learn about the study’s results, the research should be judged a failure Outline a report before writing - organisation or draft should be considered fixed Shorten and clarify statements Writing is revising ⭐ Every research report should include good writing; a clear statement of the research question; an integrated literature review; and presentation of key findings with related discussion, conclusions, and limitations Journal articles Journal article must focus on answering a particular research question Good journal editor makes sure that they have different types of experts Researchers continually critique and improve the field Applied research report An applied report is likely to have the broader purpose of describing a wide range of study findings and attempting to meet the diverse needs of multiple audiences for the research Typically provides much more information about a research Relies primarily on descriptive statistics rather than only those statistics useful for the specific hypothesis tests DRI Notes 45 Gedownload door Eugene ([email protected])  lOMoARcPSD|31736681 List the major sections of a research report Introduction - literature review, identifies gap in research Samples and measures Results Discussion Describe the elements that should be considered in writing research reports to ensure adherence to the ethical standard of honesty Honest accounting of research process, any changes that had to be made and why Evaluation of strengths and weaknesses Findings should be interpreted within the body of literature from that prior research Full record of research project available to answer an questions that may arise Report the process of developing the analysis Research sponsors should be acknowledged DRI Notes 46 Gedownload door Eugene ([email protected])