STATA222F PDF - Social Research

STATA222F The purpose of social research ● To understand and explain social phenomena ○ You’ve observed developments and changes in society ○ You’ve become interested in, and puzzled by, what is going on; you have an intellectual puzzle ○ You look for an explanation: What, How and Why ○ You draw u...

STATA222F The purpose of social research ● To understand and explain social phenomena ○ You’ve observed developments and changes in society ○ You’ve become interested in, and puzzled by, what is going on; you have an intellectual puzzle ○ You look for an explanation: What, How and Why ○ You draw upon social sciences perspectives to ■ look for explanations ■ illuminate relevant issues ■ To draw implications The logic of social research Research question(s) → Research strategies → research design → → Research methods (Data collection or generation) The role of theory ● What does a theory do ? ○ A theory represents a perspective (or an angle) to look at things ○ Theory helps us make sense of social phenomena in particular ways ● Theory and social research ○ To examine, analyze and evaluate what theoretical perspectives are more useful in helping us explain a social phenomenon or a set of social patterns ■ i.e., patterns of what/how people act, think and feel The role of literature review ● Be familiarized with relevant theories and concepts ● Identify unresolved issues and/or research gaps ● Formulate research question(s) The literature = The source of research ideas / topics, research question and research designs (=> data source & sampling, methods of data collection, and analytical strategy) The literature provides the reference for researchers about the research work previously conducted (and published) ‘A literature review is an assessment of existing knowledge– both empirical and theoretical – relating to a research topic, issue or question’ (Becker et al. 2012, p. 99) Literature review helps you ● Identifying relevant informants (=> research population) (研究對象) for the study ● Assessing relevant research designs and methods ● Identifying relevant strategies for data analysis and interpretation Good literature review ● Sensitive: able to differentiate the quality of different sources of literature ● Critical: able to assess the strengths and weaknesses of different sources and to make appropriate use of them ● Coherent (連貫): able to construct the linkages between different sources ● Precise: able to select the relevant sources in the review ● Accurate: able to document all the bibliographic information of the sources ● Relevant: able to link the details of the sources reviewed to the current research project The advantages of web search: Convenient accessibility / Wide range of coverage (?) The advantages of using library search: ● Copyrighted materials ● Archived materials ● Wider scope of materials beyond the pre-designed algorithm (演算法) with limited relevance to a specific discipline Conceptualization and Measurement - Moving from a general idea about what you want to study to effective and well-define measurements in the real world - Conceptualization - Concepts as Constructs - Dimensions and indicators - Operating definitions - Operationalization: Measurement - Conceptualization and operationalization as an ongoing process Conceptualization (Making sense of concepts) Concepts are ‘human-created ideas that represent reality - The essential components in scientific research - e.g., Intelligence: A concept, a human-created idea that captures how smart an individual is Conceptualization refers to the process of explicitly defining (明確定義) what it is meant by a particular concept In social research, the process of coming to an agreement about what a term means is conceptualization, and the result is concept (Babbie2016, pp. 125-126) Conceptualization: Dimensions and indicators of a concept A complete conceptualization involves both specifying dimensions and identifying the various indicators for each An indicator = A sign of the presence or absence of the concept we are studying From conceptualization to operationalization (操作化) : Definitions of concepts An operational definition specifies precisely how a concept will be measured i.e., the operations performed by the researchers Conceptualization and operationalization: An ongoing, continuous process The concepts and measurements of a study are not the conclusive definitions on the social phenomena under investigation The logic of social research 1. Research question(s) 2. Research strategies a. Quantitative vs. Qualitative i. The broad orientation in social research (quantitative vs qualitative) 3. Research design Experimental Cross-sectional Case study Comparative Longitudinal The framework that guides the execution of research method(s) 4. Research methods (Data collection or generation) The skills/ technique for data collection/generation using a specific research instrument (e.g., a survey questionnaire / a semi-structured interview schedule) i. Survey (incl. questionnaire) ii. Experimental iii. Semi-structured (In-depth) interviews iv. Ethnography / Participant-observation v. Focus group interviews vi. Use of documents/Visual methods vii. Online research A research method can be associated with different kinds of research strategy and design Quantitative researchers Qualitative researchers Macro-level Micro-level / Contextual Describe a social phenomenon (Patterns of) Behaviour Meanings (or/behind behaviour) How / Why / What Explain its causes( how strong) independent variables / dependent variables Emphasize how research participants understand the social world in their own terms Infer which variable causes variation observed in the other variable Seek to understand how a social phenomenon unfolds over time in specific context(s) Commonly present data in numbers Commonly present data in words,images, narratives (stories) ‘Quantitative’ and ‘qualitative’ strategies are amenable to addressing different research questions Quantitative and Qualitative research = Two different approaches to know and to explain society They may be employed in researching the same social phenomenon but can contribute to shed different light on it Quantitative researchers seek to describe and ascertain causality Qualitative researchers seek to explore how people develop/create meanings of the social world in social processes within specific contexts Focus of quantitative research IV: Time Spent with Pets at Home DV: Level of self-reported well-being Focus of qualitative research Experience in Pet Ownership Self-understanding of Mental Well-being since 2019 How has one started owning a pet? What does ‘mental well-being’ mean to the pet owners? How do pet owners think their level of mental well-being has changed since 2019? What is it like to own a pet/pets? What does owning the pet(s) mean to the pet owners? Any memorable stories to tell? The logic behind research approaches Assumptions of Quantitative research ● In the eyes of quantitative researchers: ○ Social life can be studied in the same way as we study the natural world (i.e., the ‘natural science’ model) ○ There can only be one single ‘version’/‘formula’ that explains social reality ○ Research ‘finds’ this reality through numbers and correlations ● A deductive approach to research ○ Theories inform the formulation of hypotheses ○ Theories and relatively fixed concepts drive the research process ○ The research process involves testing hypotheses that may or may not be valid in explaining social phenomena The main steps taken in Quantitative research 1. Research question(s) 2. Theory 3. Deduction of a hypothesis/hypotheses to be tested 4. Research design 5. To devise measurements of concepts (i.e., operationalization) 6. To design research instruments (e.g., a questionnaire) 7. Selection of research sites 8. Sampling 9. To administer research instruments (e.g., a questionnaire) / Data collection 10. Data processing 11. Data analysis 12. Findings and conclusion Assumptions of Qualitative research ● In the eyes of quantitative researchers: ○ Social life cannot be studied in the same way as we study the natural world, because human beings have the capacity to think, self-reflect and to interpret the meanings of social phenomena, whereas objects of natural science (e.g., molecules, trees) do not ○ Social life cannot be explained by a single ‘formula’ (because there can be multiple interpretations of social reality) ● An inductive (歸納的) approach to research ○ Theoretical ideas emerge as an outcome of research ○ Research begins with concepts that give a general sense of what to look for, but tends not be restricted by fixed concepts ○ Findings inform ■ creation of new theories ■ qualification/revision (商榷/修正) of existing theories The main steps taken in Qualitative research In reality, such ‘pure’ form of research process is very rare ● Research question(s) ● Selection of relevant research site(s) and participants ● Data generation ● Data interpretation and analysis ● Conceptual and theoretical work ● Tighter specification of the research question(s) ● Generation of further data ● Further data interpretation and analysis ● Conceptual and theoretical work ● Findings/Conclusions The preoccupations of Quantitative research Causality ● Describing a phenomenon ● Explaining causes of variation ● Inferring about ○ Relationships between independent and dependent variables ○ The magnitudes of such relationships Generalizability ● Application to wider populations (beyond the research population) ● Emphasis on generating a representative sample Not all quantitative research adopt a representative sample (see Lecture 3) Replicability 可複製性 ● Standardized research processes ○ Emphasis on minimal ‘intrusion’ of researchers’ values and biases ● It is inevitable for research processes to be influenced by researchers’ values and biases (see Lecture 9) ○ This applies to both quantitative and qualitative research Different research questions require different research strategies, designs and methods There is NO ‘intrinsically’ ‘good’ or ‘bad’ strategy, design or method The preoccupations of Qualitative research Meanings ● ● describe or explain ❌ ✔️Just Interact with participants Context ● The same behaviour (or any social phenomena) could mean different things depending on the context ● Context is data and helps explain how things happen under a particular set of circumstances ● Detailed descriptions of the research setting ● Emphasis on research participants’ perspective, i.e., how they understand social phenomena in their own terms Processes ✔️ Understand how a social phenomenon unfolds over time in context Flexibility ● ‘Fixed’ concepts/ measurements could lead us to ignore things that are important to the research participants ● General (rather than specific) research questions at the start of a research Generalizability ● Theory and concepts are often an ‘emerging’ outcome of research ● Tendency to move back and forth between data generation, data analysis, data interpretation and theoretical development in the research process Formulating the Research Question A research question = An explicit (ADJ. 明確的) statement of what it is the researcher wants to know A research question ● Asks what, how and why ● Addresses a topic related to developments and changes in society ● Draws on theoretical perspectives and concepts to make sense of the said developments and changes What is a proper Research Question? 1. Significance and Value a. Are they relevant and worth asking? b. What is the significance of addressing this research question? 2. Theoretical Grounding a. Are they sufficiently grounded in relevant literature? 3. Originality a. Are they original in terms of i. The subject matter of interest? ii. The theoretical perspectives and concepts? iii. Research design/methods? iv. Analysis/Interpretation? v. Implications for policy/practice? 4. Are your research questions researchable? ● Are the ideas and concepts contained in the research question convertible (changeable) into researchable items ○ I.e They can be operationalized? ● Are the research procedures feasible in practical terms (i.e., time, resources, access…)? Research Design Experimental Cross-sectional Case study Comparative Longitudinal The framework that guides the execution of research method(s) A proper research is expected to measure up to (at least some of) the Quality Criteria in social research ● Measurement or Construct validity ● Internal validity ● External validity (Generalizability) ● Ecological validity ● Reliability ○ Reliability depends on Validity Measurement or Construct validity ● To what extent does a measure that is designed to capture an idea or a concept contained in the RQ really reflect and help you capture the idea/concept that it (the measure) is supposed to be denoting? E.G. Measurement of (Self-reported) Connectedness with Friends Measurement of Self-perceived Mental Well-being Mental Health Awareness Measurement of (Self-reported) Interaction with Community Internal validity ● How confident can researchers claim that the independent variable causes variation observed in the dependent variable? External validity (Generalizability) ● To what extent can findings be generalized beyond the research setting? Ecological validity ● To what extent can the findings be applicable to people's everyday natural social settings? Reliability depends on Validity Research Design 1. 2. 3. 4. 5. Experimental design Cross-sectional design Longitudinal design Case study Comparative design Experimental design (實驗) A randomized control trial: ● Participants are randomly assigned to experimental / treatment vs. control groups ○ Random assignment = Blind assignment ○ (i.e., not affected by human judgement) with every participant having a equal chance to be assigned to either the experimental/treatment group or the control group) Experimental manipulation (i.e., the treatment) is carried out with the experimental / treatment group The independent variable is manipulated This allows researchers to establish that the only difference observed between the groups is responsible for variation observed in the dependent variable e.g., to establish that the special diet is responsible for variation in weight change The Randomized Control Trial (RCT) permits ● The comparison of the impact of an intervention ( 介 入 ) (e.g., special diet designed for weight loss, or a policy designed for addressing a social problem) with what would have happened if there had been no intervention, OR ● The comparison of the impacts of different kinds of intervention Experimental designs are most popular in ● Medical / Health-related research ● Social psychology, organizational studies, and social policy studies Strengths - Experimental design Strong internal validity ● The manipulation of the independent variable allows the control or elimination of the possible effects of rival explanations of a causal finding ● Researchers can be confident of the robustness of the causal finding Limitations - Experimental design Social scientists cannot be manipulated Many independent variables that interest e.g., Gender; Socio-economic status (SES) or Social class; Age, etc. Not too popular in social research E.g. Cannot call participants to change gender. Such a design can be considered as ethically problematic. Problems in generalizability (External validity) ● Not generalizable to those who are not pre-tested ● In real life, people are rarely pre-tested; the pre- test could have influenced how participants respond to the post-test Low ecological validity ● • Participants act with an awareness that they are in an experiment (i.e., likely to act differently from how they act in real life) Problematic Ethics ● It could cause participants harm (e.g., by making them go on diet or spend more time on mobile gaming than usual) ● Deception (lying) may be involved Quasi-experimental design（類實驗） ● It shares some characteristics of experimental designs，BUT participants are not randomly assigned to experimental vs. control groups ● Most popular in Evaluation research Evaluation of the effectiveness of clinical intervention E.g A drug, a therapy, etc. Evaluation of the effectiveness of social programmes E.g Allowance for low-income carers for older adults, training for professionals about discrimination in the workplace, etc. Strengths and Limitations - Quasi-experimental design: Ascertaining (make sure) causality ● We can assess whether and how an intervention (the independent variable) causes variation in the dependent variable Stronger ecological validity (Can it be applied to natural social life ?) ● When compared to experimental design, there is no experimental manipulation and no artificial intervention into ‘real’, natural social life Weaker internal validity (see earlier discussion) ● Variation in the dependent variable (e.g., academic achievement) cannot be claimed to be exclusively produced by variation in the independent variable (e.g., gaming restrictions) ● No random assignment ○ UNLIKE experimental designs, research participants are not randomly assigned Do qualitative researchers use experimental design? (Less likely) The logic of qualitative research is at odds with ● The deductive (scientific) logic and the preoccupation with causality (rather than meanings, processes and contexts) of experimental design ● The reliance of experimental design on experimental manipulation and the use of an artificial (rather than naturally- occurring) setting ● The potentially problematic ethics of experimental design (see later lectures) Cross-sectional design Data on more than one case is collected at a single point of time The data collected relates to (at least) two variables i.e. Relationship between average hours spent per day on gaming and duration of lockdown in home city among different age groups in Country A (as of October, 2021) Strengths - Cross-sectional design Advantages over experimental design Variables that interest social researchers often cannot be manipulated in experimental design e.g., age, gender It is more feasible to study the causal effects of these variables in a cross-sectional design Strengths Often standardized and clearly specified research procedures (=> higher replicability; see Lecture 1 and later discussions) A good chance of producing generalizable findings (if a random sample is used => Lecture 4) Limitations - Cross-sectional design Lower internal validity when compared to experimental design ● No participants are pre-tested, exposed to treatment and post-tested ● Direction of causal influence is ambiguous ○ Which comes first? Which causes which? ● Low confidence in drawing causal inferences Relatively low ecological validity (Can it be applied to natural social life ?) ● Participants are likely not acting in the same way as in ‘real’, natural settings when they respond to an ‘artificial’ research instrument e.g., when they try to complete a survey questionnaire in a shop Comparative design ● ● ● ● ● Studying at least two contrasting cases (e.g., organizations, communities, countries) using identical methods Data from each case is usually collected within a cross-sectional design Quantitative or qualitative strategies (or both) can be adopted Cases can be selected based on their supposed contrasting features Comparison among two or more meaningfully contrasting cases => Better understanding of social phenomena Strengths and limitations : Comparative design ● One should be reminded, however, that the apparently contrasting features of the selected cases may not exclusively explain variation observed in the findings ○ Thus, some caution is necessary when one make inferences about the findings ○ Cases can also be selected based on their similarity ● Any variations observed in the findings are likely to be derived (衍生出) from what researchers reveal as important during the research process, rather than ‘pre-existing’ differences of the cases ● Raising awareness of the need to appreciate research findings as culturally-specific ● Nonetheless, it is hard to ensure that data can be directly comparable (in terms of, e.g., how the sample is derived, how questions are asked) ● There could be issues of translation and potential problems of insensitivity to specific contexts Longitudinal design ● ● Longitudinal design = A combination of various cross-sectional research at two (or more) points of time Data is generated for at least two waves on the same people, organizations, cases, etc. ○ e.g Changes in patterns of time use on video gaming in different age groups of City X population between June 2019 and June 2022 Comparison with cross-sectional design When compared to cross-sectional design, ● Longitudinal design is time-consuming and costly, hence less often used; BUT allows some insight into the time ordering of variables, and thus better facilitates the making of causal inferences when compared to cross-sectional design Two types of longitudinal design Panel study ● A sample is randomly selected on at least two or more occasions for data collection ○ e.g Changes in patterns of video gaming in population X: Comparison between Cohort 1 (Born in 1975) and Cohort 2 (Born in 1995) ● A panel study can distinguish between age effects (the impact of ageing process on individuals) and cohort effects (effects due to being born at a similar time) Cohort study ● An entire cohort of people of a random sample is selected ○ e.g Changes in patterns of video gaming in population X (All born in 1995) ● The sample is expected to share a certain characteristic, e.g., being born in the same week, or sharing a certain experience of, e.g., the financial tsunami, the Covid-19, etc. ● A cohort study can help ascertain(make sure) age effects, but it does not permit inferences about cohort effects Strengths and Limitations - Longitudinal design Strengths - Both panel and cohort studies are concerned with illuminating social change and with improving the understanding of causal influences over time Limitations - Sample attrition (because of e.g., deaths, withdrawal) ● Those who have left the 2nd, 3rd ... wave sample could be different in some important respects with those who remain, and so the remaining sample population do not form a representative sample for the 2nd, 3rd ... wave data generation Panel conditioning effect: Continued participation affects how respondents behave in successive waves of data generation => Adverse effects on data quality Is longitudinal design popular among both quantitative and qualitative researchers? (YES) Ethnographers are likely to be staying in a location for a relatively lengthy period for data generation ● what they do can be considered as a form of qualitative longitudinal research ● ● Repeated qualitative interviews, which can be considered as a form of qualitative longitudinal research, have become more common Social change and shifts in life-course in one’s thoughts and feelings can be captured Case study ● ● ● A detailed and intensive analysis of a single case Concerned with complexity and the unique features of the case A case could be a setting (e.g., an organization (such as a school), a community, a country) ○ However, the (physical) location may or may not be part of the object of analysis ● Both quantitative and qualitative strategies can be employed ● A case study may exhibit elements of a cross-sectional or longitudinal design ● In the latter case, multiple research methods may be employed over a relatively lengthy period What is the unit of analysis? Individual human beings - Most common unit of analysis Groups, e.g., Families; Peers；Teachers;Nurses；Tourists; Activists; Street gangs Social relations, e.g.,Parent-child relations；Teacher-student relations Social interactions, e.g.,Classroom teaching and learning；Participation in weddings；Drug Dealing Organizations, e.g.,Colleges；Multinational companies；Government departments Social and cultural artefacts, e.g.,Any product of social (and cultural) beings Welfare policy; Measures about extreme weather Film What is the case? The critical case A case is chosen for study on the grounds that it will allow a better understanding of the circumstances that critically affects whether a hypothesis will or will not hold The extreme or unique case The typical or exemplifying case It seeks to capture circumstances and conditions of a commonplace situation and provides a suitable context for certain research questions to be addressed The revelatory case It allows the opportunity to observe and analyze a phenomenon previously inaccessible to scientific investigation Strengths and Limitations - Case study External validity (Generalizability) ● One cannot easily claim/ treat typical cases as ‘representative’ of (e.g., communities, organizations and practices) of a wider universe Analytic Generalization ● X statistical generalization ● Generation of theory ● Qualification/Revision of theory ● When findings are drawn from comparable cases researched by others, generalizations could possibly be made Research strategy and research design Research design Research strategy Quantitative Qualitative Experimental Most researchers using an experimental design employ quantitative comparisons between experimental and control groups with regard to the dependent variable. No typical form. However, Bryman (1988a: 151–2) notes a study in which qualitative data on schoolchildren were collected within a quasiexperimental research design Cross-sectional Survey research or structured observation on a sample at a single point in time. Content analysis on a sample of documents. Qualitative interviews or focus groups at a single point in time. Qualitative content analysis of a set of documents relating to a single period. Longitudinal Survey research on a sample on more than one occasion, as in panel and cohort studies. Content analysis of documents relating to different time periods. Ethnographic research over a long period, qualitative interviewing on more than one occasion, or qualitative content analysis of documents relating to different time periods. Such research warrants being dubbed longitudinal when there is a concern to map change. Case study Survey research on a single case with a view to revealing important features about its nature. The intensive study by ethnography or qualitative interviewing of a single case, which may be an organization, life, family, or community. Comparative Survey research in which there is a direct comparison between two or more cases, as in cross-cultural research. Ethnographic or qualitative interview research on two or more cases. Sampling Research Population = The population suitable for addressing the research question(s) HKMU students = The research population Each HKMU student = a ‘unit’ of the population Sampling Frame = The listing of all Units The population from which sample is selected A segment of the population selected for investigation HKMU students = The research population e.g., student records or class rosters (with information on names, schools, programmes, years of study, etc.) The sample = A segment of the population selected for investigation Representative sample and Sampling error Representative sample - A sample that reflects the research population accurately i.e., it reflects the research population in terms of relevant characteristics, - gender, ethnicity; - study programmes; - levels of engagement in part-time work;etc. A representative sample is a ‘microcosm’ of the research population Sampling error - Sampling error happens when a problematic sampling strategy is employed => A sample different from the research population in terms of relevant characteristics Sample size and Sampling error NO absolute answer about what constitutes a ‘good’ sample size Decision about sample size is NOT straightforward ● Depends on the research question; are the researchers focusing: ○ on the big picture patterns, trends, causal relations? ○ on what things mean, how things work under particular circumstances (context)? ● Decisions are a compromise between ○ constraints of time/ cost ○ and requirements of the research question Note that it is the absolute size, rather than the relative size, that counts Caution: Sampling error decreases as the sample size increases*, *Provided that sample is randomly selected BUT A large sample CANNOT guarantee precision; it CANNOT be 100% ‘error-free’ - When a research population is very heterogeneous, a large sample is needed to reflect the heterogeneity of the population e.g., If the research population is not ‘HKMU students’ but ‘university students in Hong Kong’, ideally the sample size should be much larger than 450 units. Non-sampling error : 1) An inadequate sampling frame ; 2) Non-response Inadequate sampling frame ● A sampling frame is the listing of all units in the research population from which sample is selected ; e.g. If research population = HKMU students STATA222F class rosters = An inadequate sampling frame ● An inadequate sampling frame: Listing of units cannot represent the research adequately (in terms of relevant characteristics) Non-response - happens when people selected for the sample refuse to participate or cannot be contacted, or for some reason cannot supply the required research data Sampling bias - This happens when some members of the research population stand little or no chance of being selected as sample => Distorts sample representativeness i.e., the resulted sample does not accurately reflect the research population (in terms of relevant characteristics) The resulted sample is likely to be biased towards certain groups of the research population Illustration of sampling bias Research question: Researchers examine the effects of part-time work on alcohol consumption among HKMU FTUG students ; Target sample = 450 (out of 9,000, i.e., 1 in 20) HKMU students Strategy 1 Strategy 2 450 students are randomly selected out of class rosters provided by Registry ● ALL schools, programmes and years of study are covered The randomly selected 450 students are then contacted by email/phone, one-byone, to participate in the study Stay in OU Café at 12pm - 5pm from Monday to Friday, chat up people and select whoever volunteers to participate in the study Random sampling ensures that every member of the research populationstands equal chance of being selected This heavily relies upon the availability of students at said place/time Inadequate sampling frame (i.e, students who visit OU Café at 12-5pm Mondays- Fridays): NO guarantee that students of different schools, programmes and year of study can stand equal chance of being selected/ recruited at said time/place - Very hard to ascertain the representativeness of the sample - It is likely to be biased towards certain groups of the research population (e.g., those who like coffee, could afford OU cafe ́ price, etc.) However, non-response is likely to happen - Non-sampling error - The eventual sample that provided usable data is unlikely to be representative Random selection is blind Decisions about which student to approach/recruit at OU Café is unavoidably influenced by judgement (i.e., human bias) about how ‘friendly’, ‘cooperative’, etc. the potential research participants are Strategy 2 is more prone to sampling bias However, Strategy 1 is still likely to be prone to non-response hence non-sampling error Probability sampling (Higher sample representativeness) 1. Simple random sampling/ Systematic random sampling 2. Stratified random sampling 3. Multi-stage cluster sampling Non-probability sampling (Lower sample representativeness) 1. Quota sampling 2. Snowball sampling 3. Convenience sampling Decisions about sampling highly depends on whether a researcher needs generalizable findings (which requires a representative sample) to address the research question(s) NOT ALL research questions require strong generalizability of findings, and therefore NOT ALL research requires a statistically representative sample Probability sampling - A representative sample is more likely to be generated - A sampling error can be minimized (but NOT eliminated) A simple random sample as an equal probability to be included in the sample - You will then select units of the population directly from the sampling frame Note that the list of units of population in the sampling frame must first be randomly ordered A stratified(arranged in layers) random sample - A sample proportionally representative of those characteristics of the research population relevant to the research question ● In generating a stratified sample, the research population is divided by ONE criterion (e.g., school enrolled) OR by a set of criteria (e.g., school enrolled, year of study, gender, etc.) ○ The stratifying criterion (criteria) reflects (reflect) population characteristic(s) relevant to the research question ● Stratified sampling is possible only when relevant research population characteristics (e.g., school enrolled; or year of study, gender, etc.) can be identified as the stratifying criteria ● IF the sampling frame (e.g., the class rosters) does NOT give you information about the stratifying criteria, stratified sampling is not possible Multi-stage cluster sampling - A research question might require a territory-wide or national sample e.g., RQ = To what extent does part time work impact upon alcohol consumption among full-time undergraduate (FTUG) students in Hong Kong ? - Reaching sampling units will be costly and time-consuming Determining whether a probability sample is required / Evaluating probability sampling 👍 Strengths and limitations (Probability sampling) A sampling error can be minimized, probability sampling does not and cannot eliminate sampling error - It is important for researchers to demonstrate an awareness of the limitations of a probability sample, rather than to claim that the sample is ‘error free’ Non-response and sampling bias, Not all of these potential participants could provide usable data - It is difficult to derive a truly representative sample and to eliminate bias in sampling - However, bias can be minimized through sampling decisions 🤝 Non-probability sampling : Convenience sampling / Snowball sampling / Quota sampling Convenience sampling - The sample is generated by virtue of its easy accessibility to the researcher Advantages and limitations ● Generalization is impossible because there is no way to ascertain the kind of population that the sample is representative ● Response rate is relatively high ● Suitable for pilot studies intended to test a research instrument before more systematic sampling is conducted Snowball sampling - Initial contact with a small group relevant to the research question, then uses the aforesaid contacts to establish contact with other relevant groups Advantages and limitations ● Often the only way to do sampling when it is difficult or impossible to have a sampling frame ○ e.g., when your target research population are visitors to HKMU – this population is shifting ● Generalization is impossible, because the sample is not representative Quota sampling - reflect the research population in terms of the relative proportion of people in specified categories ● Quota sampling is NOT stratified sampling, Selection is NOT blind (x random) ○ because human judgment affects the selection process ● The researcher determines the categories and the number of people (i.e., the quota) to be recruited in each category ○ People who fit (the characteristics required by) these categories are then selected; this process will continue until all quotas have been filled Advantages ● Cheaper, quicker than probability sampling ● Suitable for : Testing / developing new measures / research instruments / Exploratory research ● Rare in academic research, but very popular in commercial research and opinion polls Limitations ● A quota sample is not representative ● Unlike stratified random sampling, selection in quota sampling is not carried out randomly ○ human judgement ● A quota sample can still be considered as more favorable than a probability sample when the latter generates a low response rate ● Non-probability sampling in online research ● A non-probability sample is almost always generated in online research ● The size of the research population is unknown ○ Thus, the sampling frame is likely to be unavailable, if not very expensive to access ● A sampling frame can be generated by posting information or invitation to participate via relevant forums/mailing lists ● BUT No way to ascertain the response rate or sample representativeness when research population is unknown ● Internet users = A biased sample → Better Educated / Wealthier / Younger Probability sampling is common in quantitative research. HOWEVER: NOT ALL quantitative research employ probability sampling Non-probability sampling is common in, but NOT exclusive to qualitative research A truly representative sample is very difficult to be obtained, even when probability sampling is employed A problematic sampling frame and non-response could give rise to non-sampling error / introduce sampling bias (=> less generalizable findings) Research question(s) → Research strategies → research design→ Research methods Survey research: Self-administered questionnaire / Structured interview Survey research comprises(​構 ​ 成) a cross-sectional design ● in relation to which data are collected predominantly(mainly) by questionnaire or by structured interview ● on a sample of cases drawn from a wider population and at a single point in time ● in order to collect a body of quantitative or quantifiable data in connection with a number of variables, which are then examined to detect patterns of association. Self-administered questionnaire vs Structured interview A self-administered questionnaire (Supervised/ Postal/ Internet) ● entails(necessary) a succession of usually closed-ended questions to be completed by research respondents on their own ● can be administered(manage) under supervised conditions (e.g., inside a classroom), ■ by post, or online (e.g., through email or social media) Structured interview (Face-to-face/ Telephone) ● entails face-to- face meetings or phone conversations between the interviewer(s) and the interviewee(s) ● Interviews are carried out according to an interview schedule covering a succession of closed- and open-ended questions ○ e.g., face-to-face interviews carried out by census officers Designing the survey instrument Types of questions ● Open- or Closed- ended questions ● Questions about ■ Facts (e.g., personal information) ■ Attitudes, beliefs, normative standards, and values, Knowledge ● Vignette questions (see the following discussion) Closed-ended questions ● Any question whereby research participants are provided with options from which a response is chosen ○ Predetermined(fixed) options (i.e., reflecting the researcher’s prior judgments about what counts as appropriate responses) (i.e. Strongly Disagree←→Strongly Agree) ● Questions may be phrased as a statement which requires a response Limitations ● Difficulty to come up with fixed alternatives that do not overlap and are exhaustive without producing a long list of possible answers ○ Arbitrary(求其) selection or missing data ○ Absence of spontaneity ○ Forced-choice answers and discrepancies in interpretation of questions between the ■ Lower validity of data(Unusable data) Open-ended questions - Allow respondents to reply in their own terms ● Allow the room for generating ‘unusual’ response and/or exploring new empirical issues or theoretical ideas ● Require greater effort from respondents; ● Generate answers that take a long time to code (categorize for the purpose of analysis) ○ Prone to low standardization in the coding process (see Lecture 10) Vignette questions - Suitable for examining normative(establishing) standards Ask respondents how they would respond when confronted with a particular set of circumstances Advantages: ● By anchoring a choice in a situation, it encourages a more reflective reply ● For sensitive topics, vignette questions provide a less ‘threatening’ context for respondents; ○ when thinking about ‘imagined people’, respondents are less likely to feel that they are being judged when offering a response Designing and asking questions, Rule… 1. Ask questions relevant to your research question 2. Ask questions which help you measure / capture what you intend to measure / capture a. to ensure measurement validity and reliability of data b. to ensure validity of findings 3. Offer appropriate options of answers 4. No ambiguous terms, lengthy questions, complex sentences 5. No double-barrelled(double-direct) question 6. Avoid asking a question that contains two questions 7. Avoid leading question that appears to lead the respondent to a particular direction 8. Avoid very general questions 9. Avoid questions including negatives 10. Avoid technical terms (or jargons) 11. Avoid asking questions which demand certain knowledge 12. Make sure that you provide balanced answers a. Think carefully whether you want to introduce, ‘neutral’; these options often encourage satisficing behaviour