Appraising Sampling and Data Collection in Quantitative Studies PDF

Chapter 9 Appraising Sampling and Data Collection in Quantitative Studies Basic Sampling Concepts #1 ❖Population (“P” in PICO questions) o The entire group of interest based on eligibility criteria ❖Sampling o Selection of a portion of the population (a sample) to represent the entire...

Chapter 9 Appraising Sampling and Data Collection in Quantitative Studies Basic Sampling Concepts #1 ❖Population (“P” in PICO questions) o The entire group of interest based on eligibility criteria ❖Sampling o Selection of a portion of the population (a sample) to represent the entire population ❖Eligibility criteria o The characteristics that define the population ▪ Inclusion criteria ▪ Exclusion criteria Copyright © 2022 Wolters Kluwer. All Rights Reserved. Basic Sampling Concepts #2 ❖Sampling bias: overrepresenting or underrepresenting population segment in terms of key characteristics ❖Strata: subpopulations of a population (e.g., male/female) ❖Target population: the entire population of interest ❖Accessible population o The portion of the target population that is accessible to the researcher, from which a sample is drawn Copyright © 2022 Wolters Kluwer. All Rights Reserved. Sampling Goal in Quantitative Research ❖Representative sample o A sample whose key characteristics closely approximate those of the population—a sampling goal in quantitative research ❖More easily achieved with: o Probability sampling o Homogeneous populations o Larger samples achieved through power analysis Copyright © 2022 Wolters Kluwer. All Rights Reserved. Sampling Designs in Quantitative Studies ❖Nonprobability sampling o Does not involve selection of elements at random; is rarely representative of the population ❖Probability sampling o Involves random selection of elements: each element has an equal, independent chance of being selected. o Allows researchers to estimate the magnitude of sampling error (difference between population values and sample values) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Types of Nonprobability Sampling— Quantitative Research ❖Convenience sampling: selecting the most conveniently available people as participants ❖Quota sampling: identifying population strata and figuring out how many people are needed from each stratum ❖Consecutive sampling: recruiting all people from an accessible population over a specific time interval ❖Purposive sampling: handpicking sample members Copyright © 2022 Wolters Kluwer. All Rights Reserved. Comparison of Sampling Types Copyright © 2022 Wolters Kluwer. All Rights Reserved. Question #1 Which type of sampling is most vulnerable to bias? a. Convenience sampling b. Consecutive c. Quota sampling d. Purposive sampling Copyright © 2022 Wolters Kluwer. All Rights Reserved. Answer to Question #1 a. Convenience sampling Rationale: Although it is the most widely used approach for quantitative researchers, convenience sampling is the most vulnerable to sampling biases. The problem with convenience sampling is that people who are readily available might be atypical of the population. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Types of Probability Sampling ❖Simple random sampling o Researchers establish a sampling frame—a list of population elements. ❖Stratified random sampling o The population is first divided into two or more strata, from which elements are randomly selected. ❖Systematic sampling o Involves the selection of every kth case from a list, such as every 10th person on a patient list Copyright © 2022 Wolters Kluwer. All Rights Reserved. Question #2 Tell whether the following statement is True or False. Stratified random sampling is associated with a larger sampling error, but it is more efficient. a. True b. False Copyright © 2022 Wolters Kluwer. All Rights Reserved. Answer to Question #2 b. False Rationale: Stratified random sampling enhances representativeness; cluster sampling is associated with a larger sampling error but is considered more efficient. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Sample Size ❖The number of study participants in the final sample o Sample size adequacy is a key determinant of sample quality in quantitative research. o Sample size needs can and should be estimated through power analysis. o The risk of “getting it wrong” (statistical conclusion validity) increases when samples are too small. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Critically Appraising Sampling Plans: Considerations #1 ❖The type of sampling approach used (e.g., convenience, consecutive, random) ❖The population and eligibility criteria for sample selection ❖The sample size, with a rationale ❖A description of the sample’s main characteristics (e.g., age, gender, clinical status, and so on) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Critically Appraising Sampling Plans: Considerations #2 Copyright © 2022 Wolters Kluwer. All Rights Reserved. Data Collection in Quantitative Research ❖Basic decision is the use of the following: o New data, collected specifically for research purposes, or o Existing data ▪ Records (e.g., patient charts) ▪ Historical data ▪ Existing data set (secondary analysis) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Major Types of Data Collection Methods ❖Self-reports o Patient-reported outcome (PRO) ❖Observation ❖Biophysiological measures o Biomarkers Copyright © 2022 Wolters Kluwer. All Rights Reserved. Overview of Data Collection and Sources ❖Structure ❖Quantifiability ❖Objectivity Copyright © 2022 Wolters Kluwer. All Rights Reserved. Structured Self-Reports ❖Data are collected with a formal instrument. o Interview schedule ▪ Questions are prespecified but asked orally. ▪ Either face-to-face or by telephone o Questionnaire ▪ Questions prespecified in written form, to be self-administered by respondents Copyright © 2022 Wolters Kluwer. All Rights Reserved. Types of Questions in a Structured Instrument #1 ❖Closed-ended (fixed alternative) questions o For example, “Within the past 6 months, were you ever a member of a fitness center or gym?” (yes/no) ❖Open-ended questions o For example, “Why did you decide to join a fitness center or gym?” Copyright © 2022 Wolters Kluwer. All Rights Reserved. Types of Questions in a Structured Instrument #2 Copyright © 2022 Wolters Kluwer. All Rights Reserved. Advantages of Questionnaires (Compared With Interviews) ❖Questionnaires are less costly and are advantageous for geographically dispersed samples. ❖Questionnaires offer the possibility of anonymity, which may be crucial in obtaining information about certain opinions or traits. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Advantages of Interviews (Compared With Questionnaires) ❖Higher response rates ❖Appropriate for more diverse audiences o Some people cannot fill out a questionnaire. ❖Opportunities to clarify questions or to determine comprehension ❖Opportunity to collect supplementary data through observation Copyright © 2022 Wolters Kluwer. All Rights Reserved. Question #3 What is a major advantage of using a questionnaire? a. Higher response rates b. Diversity of audience is not a problem. c. Lower cost d. Questions can be clarified if needed. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Answer to Question #3 c. Lower cost Rationale: Because each subject does not need to be seen face-to-face, questionnaires are associated with lower costs than interviews. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Composite Psychosocial Scales ❖Scale—a device that assigns a numeric score to people along a continuum o Used to make fine quantitative discriminations among people with different attitudes, perceptions, traits ❖Likert scales—summated rating scales ❖Summated rating scales (composite scales) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Likert Scales #1 ❖Consist of several declarative statements (items) expressing viewpoints ❖Responses are on an agree/disagree continuum (usually five or seven response options). ❖Responses to items are summed to compute a total scale score. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Likert Scales #2 Copyright © 2022 Wolters Kluwer. All Rights Reserved. Response Set Biases ❖Biases reflecting the tendency of some people to respond to items in characteristic ways, independently of item content ❖Examples o Social desirability response set bias o Extreme response set o Acquiescence response set (yea-sayers) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Evaluation of Self-Reports ❖Strong on directness ❖Allows access to information otherwise not available to researchers ❖But can we be sure participants actually feel or act the way they say they do? Copyright © 2022 Wolters Kluwer. All Rights Reserved. Observation ❖Structured observation of prespecified behaviors o Involves the use of formal instruments and protocols that dictate what to observe, how long to observe it, and how to record the data ❖Focus of observation ❖Clear-cut assessment criteria ❖Method of recording observations Copyright © 2022 Wolters Kluwer. All Rights Reserved. Structured Observations ❖Category systems  checklists o Formal systems for systematically recording the incidence or frequency of prespecified behaviors or events o Systems vary in their exhaustiveness. ▪ Exhaustive system: All behaviors of a specific type recorded, and each behavior are assigned to one mutually exclusive category. ▪ Nonexhaustive system: Specific behaviors, but not all behaviors, are recorded. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Rating Scales ❖Ratings are on a descriptive continuum, typically bipolar. ❖Ratings can occur: o At specific intervals o Upon the occurrence of certain events o After an observational session (global ratings) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Observational Sampling ❖Time sampling—sampling of time intervals for observation o Examples ▪ Random sampling of intervals of a given length ▪ Systematic sampling of intervals of a given length ❖Event sampling—observation of integral events; requires researchers to either know when events will occur or wait for their occurrence. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Evaluation of Observational Methods ❖Excellent method for capturing many clinical phenomena and behaviors ❖Potential problem of reactivity when people are aware that they are being observed ❖Risk of observational biases—factors that can interfere with objective observation o Observational biases probably cannot be eliminated, but they can be minimized through careful observer training and assessment. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Biophysiological Measures ❖In vivo measurements o Performed directly within or on living organisms (e.g., blood pressure measures) ❖In vitro measurements o Performed outside the organism’s body (e.g., urinalysis) Copyright © 2022 Wolters Kluwer. All Rights Reserved. Question #4 Which procedure would be classified as an in vivo biophysiological measure? a. Tissue biopsy b. Blood glucose level c. Bacterial culture d. Body temperature Copyright © 2022 Wolters Kluwer. All Rights Reserved. Answer to Question #4 d. Body temperature Rationale: In vivo measures are those performed directly within or on living organisms, such as blood pressure, body temperature, and vital capacity measurements. Tissue biopsy, blood glucose level, and bacterial culture are examples of in vitro measures. Copyright © 2022 Wolters Kluwer. All Rights Reserved. Evaluation of Biophysiological Measures ❖Strong on accuracy, objectivity, validity, and precision ❖May be cost-effective for nurse researchers ❖But caution may be required for their use, and advanced skills may be needed for interpretation Copyright © 2022 Wolters Kluwer. All Rights Reserved. Factors Affecting Data Quality in Quantitative Research ❖Procedures used to collect the data ❖Circumstances under which data were gathered ❖Adequacy of instruments or scales used to measure constructs o Psychometric assessment evaluates the measure’s measurement properties. o Reliability: extent to which scores are free from measurement error Copyright © 2022 Wolters Kluwer. All Rights Reserved. Data Quality in Quantitative Research: Validity #1 ❖Face validity: whether the instrument looks like it is measuring the target construct ❖Content validity: the extent to which the instrument’s content adequately captures the construct ❖Criterion validity: the extent to which the scores on a measure are a good reflection of a “gold standard” ❖Construct validity: the degree to which evidence about a measure’s scores in relation to other variables supports the inference that the construct has been well represented Copyright © 2022 Wolters Kluwer. All Rights Reserved. Data Quality in Quantitative Research: Validity #2 Copyright © 2022 Wolters Kluwer. All Rights Reserved.