Social Research Methods PDF

Social Research Methods, Sixth Canadian Edition Edward Bell, Alan Bryman, and Steven Kleinknecht © 2022 Oxford University Press  Chapter 2: Research Designs Introduction Research Designs Bringing Research Orientatio...

Social Research Methods, Sixth Canadian Edition Edward Bell, Alan Bryman, and Steven Kleinknecht © 2022 Oxford University Press  Chapter 2: Research Designs Introduction Research Designs Bringing Research Orientation and Design Together © 2022 Oxford University Press  Learning Objectives 1. Discuss how and why some research designs are used to produce nomothetic explanations, while others are used to come up with idiographic explanations. 2. Explain the structure and logic of the experimental method, and describe how a knowledge of this method can aid in the understanding other research designs. 3. Identify the strengths and weaknesses of cross-sectional designs. 4. Describe the purposes of using a longitudinal design. 5. Explain how the case study approach is used in research. © 2022 Oxford University Press  Introduction (1 of 4) Research design – A framework for the collection and analysis of data Ask: – What do I want to learn? – What is the nature of research question? – What kind of explanation will I want? Typically, nomothetic and idiographic © 2022 Oxford University Press  Introduction (2 of 4) Nomothetic explanations – Involve attributions of cause-and-effect, expressed in terms of general laws and principles – Typically quantitative – Nomothetic explanations must satisfy three criteria of causation: Correlation Time order Non-spuriousness © 2022 Oxford University Press  Introduction (3 of 4) Idiographic explanations – Involve a rich description of a person or group and seek to explain the particular – Typically qualitative – Not meant to apply to persons or groups who were not part of the study Empathetic understanding © 2022 Oxford University Press  Introduction (4 of 4) Once a design is selected, select specific method such as – Questionnaire – Structured interview – Participant observation – Ethnography – Experiments © 2022 Oxford University Press  Research Designs Experimental design Cross-sectional design Longitudinal design(s) Case study design © 2022 Oxford University Press  Experimental Design (1 of 5) True experiments are common in psychology and organizational studies but rare in sociology or political science – Many variables of interest are not subject to experimental manipulation – Ethical concerns preclude performing experiments – Many phenomena of interest have long-term, complex causes that cannot be simulated in experiments – Even where applicable, experimental models do not get at the perceptions and feelings of research subjects © 2022 Oxford University Press  Experimental Design (2 of 5) Two kinds of experiments: – Field experiments are conducted in real-life surroundings – Laboratory experiments take place in artificial environments Controls research environment Easier to randomly assign research subjects; therefore, enhanced internal validity Easier to replicate However, weak external validity © 2022 Oxford University Press  Experimental Design (3 of 5) Manipulation of variables – Causality underpins different types of research designs in social research, both quantitative and qualitative – It is often expressed in the language of variables Variables: characteristics or attributes of data that vary or change (e.g., gender, age, interest in a subject, belief) © 2022 Oxford University Press  Experimental Design (4 of 5) Manipulation of variables (cont’d) – Independent variables are manipulated to see if they have an impact on dependent variables – Think: Dependent is the outcome (like headache pain) Independent is the manipulated variable (like taking Tylenol or not) © 2022 Oxford University Press  Experimental Design (5 of 5) Key concepts relevant to experiments: – Experimental or treatment group: receives a treatment or manipulation of some kind – Control group: does not get the treatment or manipulation – Random assignment: participants are placed in the experimental or control group using a random method – Pre-test: measurement of the dependent variable before the experimental manipulation – Post-test: measurement of the dependent variable after the experimental manipulation © 2022 Oxford University Press  Classic Experimental Design (1 of 3) Independent and dependent variables are identified Dependent variable is observed/measured (pre-test) in control and treatment groups and recorded at T1 (time 1) The treatment group receives the treatment while the control group is left alone © 2022 Oxford University Press  Classic Experimental Design (2 of 3) The dependent variable is observed/measured (post-test) in each of the control and treatment groups and recoded as occurring at T2 (time 2) Any changes in each group are noted © 2022 Oxford University Press  Classic Experimental Design (3 of 3) Ideally, change will only occur in the treatment group True experimental evidence would eliminate all other possible (rival) explanations for the change in the dependent variable in the treatment group Most social experience involves complex issues; thus, it is hoped that the control group and random distribution of subjects will increase internal validity © 2022 Oxford University Press  Internal Validity (1 of 3) Validity is concerned with the integrity of the conclusions generated by a piece of research Internal validity is concerned with the issue of whether causation has been established by a particular study – For example: Did the study establish that personal income level in Canada really is influenced by one’s level of education? Could income be influenced by something else? © 2022 Oxford University Press  Internal Validity (2 of 3) Cook and Campbell (1979) outline some threats to internal validity in experiments that lack random assignment and/or the presence of a control group – History: Some event occurring after the treatment was given may have influenced the dependent variable – Testing: The pre-test may have influenced the dependent variable – Instrumentation: Changes in the way a test is administered may account for pre-test and post-test differences © 2022 Oxford University Press  Internal Validity (3 of 3) Threats to internal validity in experiments – Mortality: Participants leave the experiment before it is over – Maturation: Participants change over time (e.g., get older, develop mentally and emotionally, etc.) – Selection: Post-test differences between the control and experimental groups may have been caused by pre-existing differences © 2022 Oxford University Press  Measurement Validity Measurement validity (or construct validity) involves the question, “Are you measuring what you want to measure?” – For example: Is the number of murders recorded in annual police statistics a valid indicator of murder rate? © 2022 Oxford University Press  External Validity (1 of 3) External validity has two primary concerns: 1. Are the findings applicable to situations outside the research environment? a) Naturalistic studies tend to satisfy this criterion 2. Can the findings be generalized beyond the people or cases studied? a) Studies using representative samples tend to satisfy this criterion © 2022 Oxford University Press  External Validity (2 of 3) Cook and Campbell (1979) also outline some threats to external validity in experimental research: – The representativeness of the study participants: The findings may not be generalizable to a wide variety of people who were not in the experiment – The effects of the setting: The findings may not apply to settings and environments that differ from those of the experiment © 2022 Oxford University Press  External Validity (3 of 3) Threats to external validity in experiments – History effects: The findings may not apply to other time periods, either in the past or in the future – Effect of pretesting: The findings may not apply to people who were not pretested, and few people in society are pre-tested – Reactive effects of experimental arrangements: The findings may be invalid because they were caused by subjects behaving atypically because they were in an experimental situation © 2022 Oxford University Press  Replicability Replicability – The results remain the same when others repeat all or part of a study – The procedures used to conduct the research are sound and are spelled out © 2022 Oxford University Press  The Laboratory Experiment Greater control over environment is an asset Easier to assign participants randomly to conditions Limitations: – Low external validity – Life in a test tube? © 2022 Oxford University Press  Quasi-experiments Differ from true experiments in that internal validity is harder to establish There are many different types of quasi-experiments, but a particularly interesting type are “natural experiments” – Experiment-like conditions are produced by naturally occurring phenomena or changes brought about by people not doing research – Lack clear causation – e.g., evaluation research © 2022 Oxford University Press  Cross-sectional Design (1 of 4) Cross-sectional designs involve taking observations at one point in time (no “before” and “after” comparisons) They do not include a manipulation of the independent variable (no “treatment” is given) – Examples: questionnaires, structured interviews, structured observation Two or more variables are measured in order to detect patterns of association – Remember: correlation does not equal causation © 2022 Oxford University Press  Cross-sectional Design (2 of 4) There are issues with internal validity, specifically establishing the direction of causation – For example, a researcher may find a positive association between self-esteem and income – But does self-esteem influence the level of income, or is it the other way around? Is there reciprocal causation? – Self-esteem influences income and income influences self-esteem? © 2022 Oxford University Press  Cross-sectional Design (3 of 4) There can also be problems with external validity For cross-sectional studies to have external validity, it helps if some random method is used to select participants for the study If random methods are not used, the findings may not hold for people who were not studied © 2022 Oxford University Press  Cross-sectional Design (4 of 4) One strength of the design is that it can examine the effect of variables that cannot be manipulated in experiments These include things like age, gender, ethnicity, culture, social class, etc. © 2022 Oxford University Press  Longitudinal Design(s) (1 of 3) Cases are examined at a particular time (T1), and again at a later time or times (T2, T3, etc.) These designs provide information about the time-order of changes in certain variables This helps establish the direction of causation – e.g., if an increase in income is observed at T1, and an increase in life satisfaction occurs at T2, that is evidence that the increase in life satisfaction was preceded by the increase in income, rather than the other way around © 2022 Oxford University Press  Longitudinal Design(s) (2 of 3) Two basic types: 1. Panel study: The same people, households, organizations, etc. are studied at different times 2. Cohort study: People sharing the same experience are studied at different times, but different people may be studied at each time For example, a researcher may study people born in 1990 at three different times (say in 2000, 2010, and 2020), but may use different subjects each time © 2022 Oxford University Press  Longitudinal Design(s) (3 of 3) Drawbacks: – Attrition over time – It may be difficult to determine when subsequent waves of the study should be conducted – Panel conditioning: People’s attitudes and behaviours may change as a result of participating in a panel © 2022 Oxford University Press  Case Study Design (1 of 2) Sometimes questions arise regarding the external validity of case studies—the findings for a particular case may not be applicable to other cases However, achieving external validity is not the main reason for doing a case study Case studies have other strengths—they provide in-depth descriptions of the characteristics of a particular case that cannot be achieved using other methods © 2022 Oxford University Press  Case Study Design (2 of 2) A basic case study involves an in-depth study of a single case A single case can be a person, family, organization, event, country, etc. It can involve qualitative and/or quantitative research methods © 2022 Oxford University Press  Types of Case (1 of 3) The critical case: illustrates the conditions under which a certain hypothesis holds or does not hold – For example, studying a person for whom certain counselling techniques are successful © 2022 Oxford University Press  Types of Case (2 of 3) The extreme (or unique) case: illustrates unusual cases, which help in understanding the more common ones – For example, studying the life of a person who has been married seven times helps researchers understand more common marriage patterns © 2022 Oxford University Press  Types of Case (3 of 3) The revelatory case: examines a case or context never before studied – For example, the study of a particular historical figure may be enhanced when documents are “de-classified” or enter the public domain, such as the diaries of former Prime Minister McKenzie-King © 2022 Oxford University Press TABLE 2.1 Research strategy and research design © 2022 Oxford University Press