HS 3315 Research Methodology Lecture Notes PDF

Wk6 HS 3315 Research Methodology Sem 1, 2024-2025 Instructor Hamzah Mohd. Salleh (Prof. Dr.) [email protected]  ‫ع ِلِّ ْمنِي َمب يَ ْنفَعُنِي َو ِز ْدنِي ِع ْل امب‬...

Wk6 HS 3315 Research Methodology Sem 1, 2024-2025 Instructor Hamzah Mohd. Salleh (Prof. Dr.) [email protected]  ‫ع ِلِّ ْمنِي َمب يَ ْنفَعُنِي َو ِز ْدنِي ِع ْل امب‬ َ ‫علَّ ْمتَنِي َو‬ َ ‫اللَّ ُه َّم ا ْنفَ ْعنِي ِب َمب‬ “O Allah! Grant me benefit in what you have taught me, and teach me useful knowledge and increase me in knowledge.” [Sunan at-Tirmizi] ُ ‫ش ُع َو ِم ْن نَ ْف ٍس ََل ت َ ْشبَ ُع َو ِم ْن َدع َْوةٍ ََل يُ ْست َ ََج‬ ‫بب لَ َهب‬ ٍ ‫عوذُ ِب َك ِم ْن ِع ْل ٍم ََل يَ ْنفَ ُع َو ِم ْن قَ ْل‬ َ ‫ب ََل يَ ْخ‬ ُ َ ‫اللَّ ُه َّم ِإنِِّي أ‬ “O Allah, I seek refuge in You from knowledge which does not benefit, from a heart that does not entertain the fear (of Allah), from a soul that is not satisfied and the supplication that is not answered.” [Sahih Muslim] Recall Main Components of Research Methodology include: 1. Problem Statement 2. Literature Review: Before conducting research, it is crucial to review existing literature on the subject to identify gaps, build on prior work, and establish the research’s significance. 3. Research Design: This outlines the overall plan of the study and describes how data will be collected and analysed. Common research designs include experimental, observational, correlational, and qualitative designs, among others. 4. Data Collection Methods: Researchers use various methods to gather data, such as surveys, interviews, observations, experiments, case studies, and archival research. The choice of method & technique/instrument depends on the research questions and the nature of the data required. 5. Sampling: In most cases, it is impractical or impossible to study an entire population. Hence, researchers select a subset of the population called a sample. Proper sampling techniques are vital to ensure the sample is representative of the entire population. 6. Data Analysis: After data collection, researchers use statistical or qualitative analysis techniques to interpret the data and draw conclusions. This analysis depends on the research design and data type. 7. Ethical Considerations: Research involving human subjects must adhere to ethical guidelines to ensure the well-being, privacy, and informed consent of participants. Ethical considerations also extend to studies involving animals and sensitive data. 8. Validity and Reliability: Research should aim for validity (the accuracy of findings) and reliability (the consistency of results). Validity is enhanced through rigorous design and measurement techniques, while reliability is achieved through consistent procedures. 9. Limitations: Researchers should acknowledge the limitations of their study, such as sample size, constraints, or potential biases. Transparency about limitations helps in interpreting the study’s findings accurately. ❑ Research methodology is essential in ensuring that scientific investigations are conducted in a systematic and rigorous manner, producing reliable and credible results. ❑ By following established research methods, researchers can make meaningful contributions to their fields and advance knowledge. POPULATION AND SAMPLING  POPULATION ❑ Definition:- Refers to all members of a particular group. ▪ The group of interest to the researcher. ▪ The group to which the results will be applied. Other definitions The larger group from which individuals are selected to participate in a study. The total number of persons inhabiting a country, city, or any district or area.  SAMPLE ❑ Any part of a population of individuals on whom information is obtained A group on which the population is obtained Other definition The representatives selected for a study whose characteristics exemplify the larger group from which they were selected.  POPULATION AND SAMPLE EXAMPLE:- A researcher is interested in studying the effect of a new English curriculum implementation in primary school in W city (with 4 districts) ▪ Population – All the English teachers of Primary school in W city (370) ▪ Sample – 20 selected English teacher from each district. (20 x 4 = 160)  POPULATION AND SAMPLE NOTE: ❑In the same group, they can be a sample and a population in different context. EXAMPLE: All the English teachers in W city constitute the population of English teachers in Sabah, yet they also constitute a sample of all English teacher in East Malaysia.  POPULATION AND SAMPLE ❑Sometimes the population from which the sample is drawn may not be the same as the population about which we actually want information. ▪ Large gap but not overlap ▪ Sometimes they may be entirely separate Example:- Study rats in order to get a better understanding of human health Study records from people born in 2008 in order to make predictions about people born in 2009. POPULATION AND SAMPLE POPULATION INFERENCE SAMPLE  Inclusion and Exclusion Criteria | Examples & Definition Inclusion and exclusion criteria determine which members of the target population can or can’t participate in a research study. Collectively, they’re known as ELIGIBILITY CRITERIA. For a clinical study, examples of common inclusion and exclusion criteria are: ❑ Demographic characteristics: age, gender identity, ethnicity ❑ Study-specific variables: type and stage of disease, previous treatment history, presence of chronic conditions, ability to attend follow-up study appointments, technological requirements (e.g., internet access) ❑ Control variables: Fitness level, tobacco use, medications used Failure to properly define inclusion and exclusion criteria can undermine your confidence that causal relationships exist between treatment and control groups, affecting the internal validity of your study and the generalisability (external validity) of your findings. What are inclusion criteria? Inclusion criteria comprise the characteristics or attributes that prospective research participants must have in order to be included in the study. Common inclusion criteria can be demographic, clinical, or geographic in nature. Example: Inclusion criteria You are running a clinical trial for a new treatment for individuals with chronic heart failure. The following inclusion criteria apply: ✓ 18 to 80 years of age People who meet ✓ Diagnosis of chronic heart failure at least 6 months before trial the inclusion criteria are then ELIGIBLE ✓ On stable doses of heart failure therapies TO PARTICIPATE ✓ Willing to return for required follow-up (posttest) visits in the study. What are exclusion criteria? Exclusion criteria comprise characteristics used to identify potential research participants who should not be included in a study. ▪ These can also include those that lead to participants withdrawing from a research study after being initially included. ▪ In other words, individuals who meet the inclusion criteria may also possess additional characteristics that can interfere with the outcome of the study. For this reason, they must be excluded. Typical exclusion criteria can be: ❑ Ethical considerations, such as being a minor or being unable to give informed consent ❑ Practical considerations, such as not being able to read If potential participants possess any additional characteristics that can affect the results, such as another medical condition or a pregnancy, these are also often grounds for exclusion. Example: Exclusion criteria In the clinical trial for individuals with chronic heart failure, the following exclusion criteria apply: People who meet one ❑ The patient requires valve or other cardiac surgery or more of the ❑ The patient is unable to carry out any physical activity without discomfort exclusion criteria ❑ The patient had a stroke within three months prior to enrollment must be disqualified. This means that they ❑ The patient refuses to give informed consent can’t participate in the ❑ The patient is a candidate for coronary bypass surgery or something similar study even if they meet the inclusion criteria. Why are inclusion and exclusion criteria important? ❑ Defining inclusion and exclusion criteria is important in any type of research that examines characteristics of a specific subset of a population. ▪ This helps researchers identify the study population in a consistent, reliable, and objective manner. ▪ As a result, study participants are more likely to have the attributes that will make it possible to robustly answer the research question. ❑ In clinical trials, establishing inclusion and exclusion criteria minimises the likelihood of harming participants (e.g., excluding pregnant women) and safeguards vulnerable individuals from exploitation (e.g., excluding individuals who are unable to comprehend what the research entails.) Ethical considerations like these are critical in human-based research.  SAMPLING ❑ The process of selecting a number of individuals for a study in such a way that the individuals represent the larger group from which they were selected. another view: ❑ The process of selecting units (e.g., people, organisations) from a population of interest so that by studying the sample we may fairly generalise our results back to the population from which they were chosen.  OBJECTIVES OF SAMPLING ❑To make inferences about the larger population from the smaller sample. another view: ❑To gather data about the population in order to make an inference that can be generalised to the population.  ADVANTAGES OF SAMPLING ❑ Saves money, time and energy ❑ Provides information that is almost as accurate as that obtained from a complete census ▪ Extensively used to obtain some of the census information. ❑ Has much smaller “non-response”, hence much easier. ❑ Essential to obtaining the data when the measurement process physically damages or destroys the sampling unit under investigation. ❑ The only means available for obtaining the needed information when the population appears to be infinite or is inaccessible ❑ Provides a valid measure of reliability for the sample estimates ❑ Possible to obtain more detailed information from each unit of the sample  DISADVANTAGES OF SAMPLING ❑ Mostly can be biased and in some cases can choose people/units inappropriate for the circumstances  BEWARE of potential MISTAKES ❑ Threaten to render a study’s findings invalid ▪ Sampling Error ▪ Sampling Bias The chance and random variation in Nonrandom differences, generally the fault of variables that occurs when any the researcher sample is selected from the ▪ Cause the sample is over-represent individuals or population groups within the population To avoid sampling error, a census of ▪ Lead to invalid findings the entire population must be taken To control for sampling error, Sources of sampling bias include the use of researchers use various sampling volunteers and available groups methods  STEPS IN SAMPLING 1. Defining the population (N) 2. Determine sample size (n) 3. Control for bias and error 4. Select the sample  Define the Population (N) Identify the group of interest and its characteristics to which the findings of the study will be generalised ❑ Choose the “target” population ▪ The ideal selection: actual population ❑The “accessible” or “available” population must be used ▪ The realistic selection  Target Versus Accessible Populations TARGET ACCESSIBLE 1. Rarely available to generalise 1. Able to generalise 2. Researcher’s ideal choice 2. Researcher’s realistic choice 3. Example: 3. Example: All Year 1 and Year 2 pupils All Year 1 and Year 2 in Sek. Jenis in Sabah Kebangsaan in Kota Kinabalu, Sandakan and Tawau. Remember… ❑ Narrow population - Save time, effort and money ▪ Generalisability is limited ❑ The population and sample must be specific enough ▪ provide readers a clear understanding of the applicability of our study to their particular situation and their understanding of that same population. ❑ Common weaknesses of published research report ▪ fail to define in detail ❑ Actual sample may be different from original ▪ Subject refuse to participate, drop out, data lost, etc.  Determine the Sample Sizes ❑ The size of the sample influences both the representativeness of the sample and the statistical analysis of the data ▪ Larger samples are more likely to detect a difference between different groups ▪ Smaller samples are more likely not to be representative  Rules to Determine Sample Sizes ❑The larger the population size, the smaller the percentage of the population required to get a representative sample ▪ For smaller samples (N < 100), there is little point in sampling. Survey the entire population ▪ If the population size is around 500, 50% should be sampled. ▪ If the population size is around 1500, 20% should be sampled. ▪ Beyond a certain point (N = 5000), the population size is almost irrelevant and a sample size of 400 may be adequate.  Rules to Determine Sample Sizes Minimum numbers of subject needed STUDIES MINIMUM NUMBERS Descriptive 100 Correlational 50 Experimental and Causal-comparative 30 PER GROUP Qualitative 1-20  Control for Bias and Error ❑Be aware of the sources of sampling bias, and identify how to avoid it ❑Decide whether the bias is so severe that the results of the study will be seriously affected ❑In the final report, document researcher’s awareness of bias, rationale for proceeding, and potential effects  Select the Sample ❑ A process by which the researcher attempts to ensure that the sample is representative of the population from which it is to be selected ❑ Requires identifying the sampling method that will be used ❑ Two types of sampling Random sampling Non-random sampling  RANDOM SAMPLING ❑ Allows a procedure governed by chance to select the sample; this controls for sampling bias ▪ Every members of the population had equal chances to be selected ▪ A group of individual represent the entire population An accurate view of the larger group EXAMPLE: ▪ 100 students’ names were place into a box, mixed them thoroughly and then draws out 25 students’ name ❑ In order to have a random selection method, one must set up some process or procedure that assures that the different units in your population have equal probabilities of being chosen ▪ Computers can generate random numbers as the basis for random selection.  RANDOM SAMPLING (RS) METHODS Simple RS Stratified RS Cluster RS Two Stage RS  4 key steps in Simple RS Simple RS 1 2 3 4 The process of selecting a sample that Stratified RS Cluster RS Two Stage RS allows individual in the defined population to have an equal and independent chance of being selected for the sample. Step 1: Define the population Start by deciding on the population that you want to study. It is important to ensure that you have access to every individual member of the population, so that you can collect data from all those who are selected for the sample.  4 key steps in Simple RS Simple RS 1 2 3 4 Stratified RS Cluster RS Two Stage RS Step 2: Decide on the sample size Next, you need to decide how large your sample size will be. Although larger samples provide more statistical certainty, they also cost more and require far more work. There are several potential ways to decide upon the size of your sample, but one of the simplest involves using a formula with your desired confidence interval and confidence level, estimated size of the population you are working with, and the standard deviation of whatever you want to measure in your population. The most common confidence interval and levels used are 0.05 and 0.95, respectively. Since you may not know the standard deviation of the population you are studying, you should choose a number high enough to account for a variety of possibilities (such as 0.5).  4 key steps in Simple RS Simple RS 1 2 3 4 Stratified RS Cluster RS Two Stage RS Step 3: Randomly select your sample This can be done in one of two ways: the lottery or random number method. In the lottery method, you choose the sample at random by “drawing from a hat” or by using a computer programme that will simulate the same action. In the random number method, you assign every individual a number. By using a random number generator or random number tables, you then randomly pick a subset of the population. You can also use the random number function (RAND) in Microsoft Excel to generate random numbers.  4 key steps in Simple RS Simple RS 1 2 3 4 Stratified RS Cluster RS Two Stage RS Step 4: Collect data from your sample Finally, you should collect data from your sample. To ensure the validity of your findings, you need to make sure every individual selected actually participates in your study. If some drop out or do not participate for reasons associated with the question that you’re studying, this could bias your findings. For example, if young participants are systematically less likely to participate in your study, your findings might not be valid due to the underrepresentation of this group.  Simple RS ❑ ADVANTAGES Large-likely to produce a representative sample Easy to conduct Strategy requires minimum knowledge of the population to be sampled ❑ DISADVANTAGES Not easy to do Need names of all population members May over-represent or under-estimate sample members There is difficulty in reaching all selected in the sample  Stratified RS ❑ In Stratified RS, researchers divide a population into homogeneous subpopulations called strata (singular, stratum) based on specific characteristics (e.g., race, gender identity, location, etc.). ▪ Every member of the population studied should be in exactly one stratum. ❑ Each stratum is then sampled using another probability sampling method, (such as cluster sampling or simple random sampling), allowing researchers to estimate statistical measures for each sub-population. Researchers rely on stratified sampling when a population’s characteristics are diverse and they want to ensure that every characteristic is properly represented in the sample. This helps with the generalisability and validity of the study, as well as avoiding research biases (like undercoverage bias). Research EXAMPLE: You are interested in how having a PhD affects the wage gap between gender identities among graduates of a certain university. Because only a small proportion of this university’s graduates have obtained a PhD degree, using a simple random sample would likely give you a sample size too small to properly compare the differences between men and women, with a PhD degree versus those without one. Therefore, you decide to use a stratified sample, relying on a list provided by the university of all its graduates within the last ten years. When to use stratified sampling? ❑ To use stratified sampling, you need to be able to divide your population into mutually exclusive and exhaustive subgroups. ▪ That means every member of the population can be clearly classified into exactly one subgroup. ❑ Stratified sampling is the best choice among the probability sampling methods when you believe that subgroups will have different mean values for the variable(s) you’re studying. Steps in Stratified RS 1. Identify and define the population 2. Identify the variable and subgroups (strata) for which you want to guarantee appropriate, equal representation. 3. Classify all members of the population as members of one identified subgroup. (see next slide) 4. Determine the desired sample size from each stratum. 5. Randomly select, (e.g. using a table of random numbers) an “appropriate” number of individuals from each of the subgroups, appropriate meaning an equal number of individuals EXAMPLE: Separating the population into strata You compile a list of every graduate’s name, gender identity, and the degree that they obtained. Using this list, you stratify on two characteristics: gender identity, with two strata (male and female), and degree, with three strata (bachelor’s, master’s, and doctorate). Combining these characteristics, you have six groups in total. Each graduate must be assigned to exactly one group. Characteristic Strata Groups Gender Identity Male Male bachelor’s graduates Female Female bachelor’s graduates Male master’s graduates Degree Bachelor’s Female master’s graduates Master’s Male doctoral graduates Doctorate Female doctoral graduates Proportionate vs disproportionate stratum SAMPLE SIZE: First, you need to decide whether you want your sample to be proportionate or disproportionate. ❑ In proportionate sampling, the sample size of each stratum is equal to the subgroup’s proportion in the population as a whole. ▪ Subgroups that are less represented in the greater population (for example, rural populations, which make up a lower portion of the population in most countries) will also be less represented in the sample. ❑ In disproportionate sampling, the sample sizes of each strata are disproportionate to their representation in the population as a whole. ▪ You might choose this method if you wish to study a particularly underrepresented subgroup whose sample size would otherwise be too low to allow you to draw any statistical conclusions. SAMPLE SIZE Next, you can decide on your total sample size. This should be large enough to ensure you can draw statistical conclusions about each subgroup. ❑ If you know your desired margin of error and confidence level as well as estimated size and standard deviation of the population you are working with, you can use a sample size calculator to estimate the necessary numbers. EXAMPLE: Sample size Because you need to ensure your sample size of doctoral graduates is large enough, you decide to use disproportionate sampling. Even though doctoral students make up a small proportion of the overall student population, your sample is about ⅓ bachelor’s graduates, ⅓ master’s graduates, and ⅓ doctoral graduates. Sample size calculator https://www.qualtrics.com/uk/experience- management/research/determine-sample-size/ Stratified RS ❑ADVANTAGES More precise sample Can be used for both proportions and stratification sampling Sample represents the desired strata ❑DISADVANTAGES Need names of all population members There is difficulty in reaching all selected in the sample Researcher must have names of all populations Cluster RS ❑ The process of randomly selecting intact groups, not individuals, within the defined population sharing similar characteristics EXAMPLE: Population of 10,000 teachers, 10 school were as a sample. All the teachers in 10 schools are sample. ❑ More effective with larger numbers of cluster ❑ Similar to simple random sampling ▪ Sampling unit is a group and not the individuals. Cluster RS Research EXAMPLE: You are interested in the average reading level of all the seventh-graders in Bandar Seri Begawan. It would be very difficult to obtain a list of all seventh-graders and collect data from a random sample spread across the city. However, you can easily obtain a list of all schools and collect data from a subset of these. You thus decide to use the cluster sampling method. Steps in Cluster RS 1. Identify and define the population 2. Identify and define a logical cluster. 3. List all clusters (or obtain a list) that make up the population of clusters. 4. Determine the desired sample size. 5. Determine the number of clusters needed by dividing the sample size by the estimated size of a cluster. 6. Randomly select the needed number of clusters (e.g. using a table of random numbers) 7. Include in your study all population members in each selected cluster. Sample ❑ You assign a number to each school and use a random number generator to select a random sample. ❑ You choose the number of clusters based on how large you want your sample size to be. This in turn is based on the estimated size of the entire seventh-grade population, your desired confidence interval and confidence level, and your best guess of the standard deviation (a measure of how spread apart the values in a population are) of the reading levels of the seventh-graders. ❑ You then use a sample size calculator to estimate the required sample size. Cluster RS ❑ADVANTAGES Efficient Researcher doesn’t need names of all population members Reduces travel to site Useful for educational research ❑DISADVANTAGES Fewer sampling points make it less like that the sample is representative  Two Stage RS ❑ Combination of Cluster Random Sampling and Individual Random Sampling EXAMPLE: Population of 3000 individuals in 100 classes ▪ Selecting 25 class ▪ Randomly select 4 students from each class ❑ Less time-consuming NONRANDOM SAMPLING ❑ Nonprobability, purposive sampling ▪ Does not have random sampling at any state of the sample selection; this can increase probability of sampling bias ▪ Each member did not have equal chance of being selected EXAMPLE Each person who enters the bookstore at lunch time will be given a questionnaire. (anonymous) After two weeks, 200 completed questionnaires were obtained  NONRANDOM SAMPLING METHODS Systematic Convenience Purposive Sampling Sampling Sampling Systematic Sampling When to use systematic sampling Systematic sampling is a method that imitates many of the randomisation benefits of simple random sampling, but is slightly easier to conduct. You can use systematic sampling with a list of the entire population, like you would in simple random sampling. However, unlike with simple random sampling, you can also use this method when you’re unable to access a list of your population in advance. Systematic Sampling ❑ The process of selecting individuals within the defined population from a list by taking every nth name. EXAMPLE, with Random Start ▪ Population: 5000 names ▪ Selecting every tenth name on the list till it reach 500 sample 1. Sampling Interval ❑ Two important terms: Distance in a list between each of the individuals selected for sample ▪ Sampling interval Population size ▪ Sampling ratio Desired sample size 2. Sampling Ratio Proportion of individuals in the population that is selected for the sample Sample size Population size Steps in Systematic Sampling 1. Identify and define the population. 2. Determine the desired sample size. 3. Obtain a list of the population. 4. Determine what n is equal to by dividing the size of the population by the desired sample size (i.e. calculate sampling interval k) 5. Select the sample and collect data ▪ Start at some random place in the population list. (e.g. close your eyes and point your finger to a name. ▪ Starting at that point, take every nth name on the list until the desired sample size is reached. If the end of the list is reached before the desired sample is reached, go back to the top of the list. Systematic Sampling Listing the population in advance ▪ Ensure that your list contains the entire population and is not in a periodic or cyclic order. ▪ Ideally, it should be in a random or random-like (such as alphabetical) order, which will allow you to imitate the randomisation benefits of simple random sampling. EXAMPLE: Listing the population In your department store study, your customers make up your target population. To create your sample ahead of time, you would need to create a list of every customer who visited your store in the last week. However, creating such a list would be difficult, if not entirely impossible. You could choose to use receipts to create your list, but this would exclude any non-buying customers, which would most likely bias your results. Systematic Sampling Selecting your sample on the spot If you cannot access a list in advance, but you are able to physically observe the population, you can also use systematic sampling to select subjects at the moment of data collection. In this case, ensure that the timing and location of your sampling procedure covers the full population to avoid bias in the results. EXAMPLE: Sampling on the spot ❑ As you cannot get a complete list of your store’s customers, you instead choose to sample every kth customer as they exit the store. This allows you to include both those who buy items and those who do not. ❑ You must ensure that you are sampling throughout the entire week to ensure a representative sample, because different types of customers enter at different times and days: Teenagers usually shop after school and on the weekends, while working professionals might shop later in the evening and stay-at-home parents during the day. Systematic Sampling Sample size and sampling interval Although you do not know exactly how many people will visit your store ahead of time, you can estimate the total population by using an average of the prior few weeks’ foot traffic. You estimate that around 7500 people visit your store each week, and based on this estimate you calculate an ideal sample size of 366. Your sampling interval k thus equals 7500/366 = 20.49, which you round to 20. Systematic Sampling EXAMPLE: Data collection You choose an employee to stand by the door and survey every 20th customer who leaves. It is important that as many as possible of those chosen for the sample decide to participate; otherwise, your results may not properly reflect the opinions of the overall population. For instance, those with particularly good or bad opinions of the store may be more willing to participate than the general customer population, thus biasing the results of your survey. Systematic Sampling ❑ ADVANTAGE ▪ Sample selection is simple ❑ DISADVANTAGE ▪ All members of the population do not have an equal chance of being selected. ▪ The nth person may be related to a periodical order in the population list, producing un- representativeness in the sample ▪ Periodicity- a markedly biased sample can result If the arrangement of individuals on the list is in some sort of pattern accidentally coincidence with the sampling interval. Systematic Sampling When planning Ensure no cyclical pattern Not bias the sample  Convenience Sampling ❑ When to use convenience sampling? Convenience sampling is often used in qualitative and medical research studies. ▪ In medical research, convenience sampling often involves selecting clinical cases or participants that are available around a particular location (such as a hospital) or a medical records database. ▪ In qualitative research, convenience sampling is often used in social sciences and education where it’s convenient to use pre-existing groups, such as students. ❑ Convenience sampling could be a good fit for your research if: ▪ You want to get an idea of people’s attitudes and opinions ▪ You want to run a test pilot for your survey ▪ You want to generate hypotheses that can be tested in greater depth in future research Convenience Sampling ❑ The process of including whoever happens to be available at the time ▪ called “accidental” or “haphazard” sampling EXAMPLE: Restaurant manager select 50 samples by choosing the first 50 students who walk in front of his store. ❖ Cannot be considered as sample, should be avoided ❑ Should be replicated to decrease the likelihood that the results obtained were simply one time occurrence. Convenience sampling examples EXAMPLE REMARKS Online convenience ▪ You are researching how parents use a popular online parenting forum. You want to find sampling out if parents are likely to participate in discussions online or just “lurk,” as well as what kind of information they are seeking there. ▪ Since it’s an online community, there is no membership list to use as a sampling frame. This is a good scenario for using convenience sampling. You decide to draw a convenience sample of 100 users. ▪ You create a pop-up ad that invites users to complete your online survey, which the administrators agree to place on the website.To entice users to participate, a prize draw is mentioned in the ad. Convenience ▪ Suppose you are researching why people visit Seri Kenagan Beach, a popular sampling based on destination in your district. To gather insights, you stand in a parking area and approach people at random, asking them if they would be interested in participating in a five- location minute anonymous survey on their preferred recreational activities. ▪ To maximise the number of responses, you also create flyers with a scannable QR code and a shortened URL link. You place them at the Welcome Centre and other locations along the beach. Convenience sampling examples EXAMPLE REMARKS Crowdsourced ▪ You are conducting research into attitudes toward depression. You are interested in the convenience sampling difference between collectivistic and individualistic cultures. As an early-career researcher, you do not have an extensive international network. You can use a crowdsourcing platform (e.g. Amazon Mechanical Turk, MTurk). ▪ Crowdsourcing platform covers a wide range of demographic populations both in the locally and internationally. This enables you to access a more diverse pool of respondents in exchange for monetary compensation. Convenience ▪ You are doing a survey to investigate work satisfaction at a large camping gear sampling of a pre- company in your town. The manager has given you permission to conduct your research but cannot give you a list of all employees due to privacy regulations. existing group ▪ As you do not have a sampling frame, you cannot use probability sampling. Instead, you decide to use convenience sampling. You stand next to the coffee machine and approach random employees, asking them to fill in your quick survey. How to reduce bias in convenience sampling? LIMITATIONS of convenience sampling:. ▪ Since the researcher draws the sample based on convenience and not equal probability, convenience samples never result in a statistically balanced selection of the population. This leads to sampling bias. ▪ Very often, participants are offered monetary or other incentives to complete a survey. If a reward is their only motivation, they may give inaccurate or false answers. This leads to response bias, social desirability bias, and self-selection bias. ▪ Researchers are subjective in how they choose their participants (e.g., by stopping the passersby who appear friendliest). This leads to observer bias. Ways TO REDUCE BIAS in research: ❑ Describe in detail how you recruited your participants in the methodology section of your research paper to make your research reproducible and replicable. ❑ Diversify your data collection by recruiting as many participants or cases as possible and use a sample size calculator to determine the appropriate sample size. ❑ Distribute your surveys at different days and times, and use different methods for recruiting participants ❑ Use appropriate descriptive analysis methods, rather than statistical analyses designed for probability samples Overall, avoid overstating your research findings. Remember that findings based on a convenience sample only apply to the selected cases or participant group. By definition, they cannot be generalised to the target population.  Convenience Sampling ADVANTAGES Depending on your research design, there are advantages to using convenience sampling. ❑ Convenience sampling is usually low-cost and easy, with subjects readily available. ❑ In the absence of a sampling frame, convenience sampling allows researchers to gather data that would not have been possible otherwise. ❑ If you are conducting exploratory research, convenience sampling can help you gather data that can be used to generate a strong hypothesis or research question. DISADVANTAGES Convenience sampling has its disadvantages as well, and it’s not a good fit for every study. ❑ Since the sample is not chosen through random selection, it is impossible that your sample will be fully representative of the population being studied. This undermines your ability to make generalisations from your sample to the population of interest. ❑ Getting responses only from the participants who are easiest to contact and recruit leaves out many respondents. This affects the accuracy of your data and runs the risk that important cases are not detected, leading to undercoverage bias. ❑ Convenience sampling relies on the subjective judgment of the researcher and the subjective motivations of the participants. This leads to a high risk of observer bias. Purposive Sampling What is it? | Definition & Examples ❑ Purposive sampling refers to a group of non-probability sampling techniques in which units are selected because they have characteristics that you need in your sample. ▪ i.e, units are selected “on purpose” in purposive sampling. involves a process whereby the researcher selects a sample based on experience or knowledge of the group to be sampled ▪ also called judgmental sampling, this sampling method relies on the researcher’s judgment when identifying and selecting the individuals, cases, or events that can provide the best information to achieve the study’s objectives EXAMPLES: 1) Teacher choose 2 students from each level of intelligent to find about how does her class feel about the role play in the classroom 2) Researcher only interview those he thinks possess the needed information. ❑ Purposive sampling is common in qualitative research and mixed methods research. ▪ it is particularly useful if you need to find information-rich cases or make the most out of limited resources, but is at high risk for research biases like observer bias.  Purposive Sampling When to use purposive sampling? ❑ best used when you want to focus in depth on relatively small samples ▪ perhaps you would like to access a particular subset of the population that shares certain characteristics, or you are researching issues likely to have unique cases ❑ main goal of purposive sampling is to identify the cases, individuals, or communities best suited to helping you answer your research question ▪ for this reason, purposive sampling works best when you have a lot of background information about your research topic ▪ the more information you have, the higher the quality of your sample. PURPOSIVELY SAMPLING examples EXAMPLE REMARKS Maximum Variation ▪ is used to capture the widest range of perspectives possible. Sampling, also known ▪ EXAMPLE: Suppose you are researching the challenges of mental health services programs in your state. Using maximum variation sampling, you select programs in as heterogeneous urban and rural areas in different parts of the state, in order to capture maximum sampling variation in location. In this way, you can document unique or diverse variations that have emerged in different locations. Homogeneous ▪ Continuing your research on mental health services programs in your state, you are Sampling, aims to now interested in illuminating the experiences of different ethnicities through group interviewing. reduce variation, ▪ Using homogeneous sampling, you select Latinx directors of mental health services simplifying the analysis agencies, interviewing them about the challenges of implementing evidence-based and describing a treatments for mental health problems. particular subgroup in depth PURPOSIVELY SAMPLING examples EXAMPLE REMARKS Typical Case Sampling is used ▪ You are researching the reactions of 9th grade students to a job placement when you want to highlight what is program. To develop a typical case sample, you select participants with similar considered a normal or average socioeconomic backgrounds from five different cities. instance of a phenomenon to those ▪ You collect the students’ experiences via surveys or interviews and create a who are unfamiliar with it. Participants profile of a “typical” 9th grader who followed a job placement program. This are generally chosen based on their can offer useful insights to employers who want to offer job placements to likelihood of behaving like everyone students in the future. else sharing the same characteristics or experiences. Keep in mind that the goal of typical case sampling is to illustrate a phenomenon, not to make generalized statements about the experiences of all participants. PURPOSIVELY SAMPLING examples EXAMPLE REMARKS Critical Case Sampling is used ▪ You are researching how to involve local communities in local government when a single or very small number of decision-making processes, but you are not sure whether the communities will cases can be used to explain other understand the regulations. similar cases. Researchers ▪ If you first ask local government officials and they do not understand them, determine whether a case is critical then probably no one will. Alternatively, if you ask random passersby, and they by using this maxim: “if it happens do understand them, then it’s safe to assume most people will. here, it will happen anywhere.” In ▪ In this way, your critical cases could either be those with relevant expertise or other words, a case is critical if what those who have no relevant expertise. is true for one case is likely to be true for all other cases. PURPOSIVELY SAMPLING examples EXAMPLE REMARKS Expert Sampling is used when your ▪ You are investigating the barriers to reduced meat consumption among research requires individuals with a high consumers in the US. In addition to focus groups with consumers, you level of knowledge about a particular decide to contact a number of experts and interview them. subject. Your experts are thus selected ▪ In the context of your research, food scientists are the experts who can based on a demonstrable skill set, or level provide valuable insights into the root of the problem, as well as any of experience possessed. successes, failures, or future trends to watch.  Purposive Sampling ADVANTAGES ❑ Although it is not possible to make statistical inferences from the sample to the population, purposive sampling techniques can provide researchers with the data to make other types of generalisation from the sample being studied. Remember that these generalisations must be logical, analytical, or theoretical in nature to be valid. ❑ Purposive sampling techniques work well in qualitative research designs that involve multiple phases, where each phase builds on the previous one. Purposive sampling provides a wide range of techniques for the researcher to draw on and can be used to investigate whether a phenomenon is worth investigating further. DISADVANTAGES ❑ As with other non-probability sampling techniques, purposive sampling is prone to research bias. Because the selection of the sample units depends on the researcher’s subjective judgment (may be in error), results have a high risk of bias, particularly observer bias. ❑ If you are not aware of the variations in attitudes, opinions, or manifestations of the phenomenon of interest in your target population, identifying and selecting the units that can give you the best information is extremely difficult.  GENERALISING FROM A SAMPLE ❑ Generalise (Generalisation) Apply the findings of a particular study to people/ setting that go beyond the particular people / settings used in study Considering nature and environmental condition ❑ Determine the External Validity The extent to which the result of a study can be generalised from a sample to a population  GENERALISING FROM A SAMPLE ❑ Population Generalisability ▪ The extent to which the results of a study can be generalised to the intended population ▪ Result of investigation need to be applicable as wide as possible ▪ Representatives – relevant (sample must include all representative) Contributing factor to any result obtain ▪ Not applicable when The result is for particular group at particular time  GENERALISING FROM A SAMPLE ❑ Sample - As thoroughly as possible To let others judge it well Representatives of the target population ❑ Replication Repeat the study using different groups in different situation Get same result, additional confidence in generalising finding  GENERALISING FROM A SAMPLE ❑ Ecological Generalisability The extent to which the results of a study can be generalised to conditions or settings other than those that prevailed in a particular study Ensure the nature of environmental conditions same in all important respects in any new situation INTERNAL and EXTERNAL Validity Internal validity examines whether the study design, conduct, and analysis answer the research questions without bias. (in other words, internal validity is the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables.) External validity examines whether the study findings can be generalised to other contexts. (in other words, external validity is the extent to which your results can be generalised to other contexts.)  End of Lec6 Notes