N311 Week 7 Part I Sampling & Data Collection PDF

SAMPLING & DATA COLLECTION IN QUANT RESEARCH N 311 Dr. S. Prendergast PhD NP  CONCEPTS Population, strata, sample, sample size Sampling: non-probability, probability sampling bias, sampling error Data: Patient reported outcomes (self-report), observational measur...

SAMPLING & DATA COLLECTION IN QUANT RESEARCH N 311 Dr. S. Prendergast PhD NP  CONCEPTS Population, strata, sample, sample size Sampling: non-probability, probability sampling bias, sampling error Data: Patient reported outcomes (self-report), observational measures, biophysical measures Quality: Reliability, validity, measurement bias, recall bias, response bias (esp. p. 172)  Differentiate between a population and sample Distinguish between non-probability and probability sampling Identify and describe several types of LE AR NI N G sampling designs O BJ E CT IV ES Describe methods for collecting data (self-report, observation, biophysical) Readings: Woo Ch. 10  SAMPLING IN QUANTITATIVE RESEARCH  KEY TERMS  Sampling: the process of selecting a portion of the population to represent the entire population  Population: the entire aggregation of cases that meet specific criteria  Sample: a subset of the population  Target population: the entire population that the researcher is interested in  Accessible population: cases from the target population that are accessible to the researcher as a pool of subjects  Eligibility (inclusion) Criteria: researchers specify the characteristics that delimit the study population  Representative sample: a sample whose characteristics closely approximate those of the population  Sampling bias: systematic overrepresentation or underrepresentation of some segment of the population in terms of a characteristic relevant to the research questions  Strata: a mutually exclusive segments of a population based on a specific characteristic  Elements: entities that make up the samples and populations  SAMPLING DESIGNS 1. Nonprobability = nonrandom sampling (4 types; p. 168) 2. Probability = random sampling (3 main types; p. 169-170)  SAMPLING DESIGN: NON-PROBABILITY  Convenience sampling: selecting the most conveniently available people as participants (the weakest but most commonly used).  Quota sampling: identifying strata of a population, then determining how many participants are needed from each stratum. Participants are a convenience sample from each stratum  Consecutive: recruiting all people from an accessible population over a specific time or for a specified sample size  Purposive sampling: researchers purposely hand pick the cases to be included in the sample  SAMPLING DESIGN: PROBABILITY  Simple random sampling: most basic probability sampling design NOTE: There is no guarantee of a representative sample, but random selection guarantees that differences between the sample and the population are purely a function of chance.  Stratified random sampling: the population is divided into homogeneous strata from which elements are randomly selected.  Systematic Sampling: selection of every __th case from a list or group.  SAMPLE SIZE Sample size: the number of subjects/study participants in a sample Quantitative researchers are generally advised to use the largest sample possible Larger sample size = maximized representativeness Larger sample size = smaller sampling error Power analysis: strategy to estimate how large a sample should be to test a hypothesis  Small sample size = suspicion of bias Weak sample strategy = suspicion of bias CRITIQUE OF Response rate: the number of people SAMPLING participating in a study relative to the PLANS number of people sampled Nonresponse (response) bias: differences between participants and those who declined to participate Refer to box 10.1 on page 172 for guidelines  SUMMARY Nonprobability Probability Convenience Simple Random  Main consideration in assessing a sample in quantitative studies Quota Stratified Random is its representativeness to reduce bias Purposive Systematic  Sample size is important to a study’s statistical conclusion validity Consecutive DATA COLLECTION In Quantitative Research  DATA COLLECTION Data collection is the process of gathering data that will be measured, observed or recorded New vs. Existing Data (eg, existing records, clinical data gathered for non-research purposes) Types of data include; self report, observation & biophysiologic measures  SELF REPORTS Self reports are participants’ responses to questions posed by the researcher Techniques – Question Form, Instruments: Interviews vs Questionnaires, Scales  SELF REPORTS Question Strengths Weaknesses Form Open- Easier to Difficult to ended develop administer or Question Form: Allows for in- analyze depth response Time Closed-ended (Fixed consuming Alternative): response alternatives (answers) are Closed- Easy to Difficult to pre-specified by researcher ended administer and develop analyze Limiting Open-ended: participants Efficient (time) respond in their own words Increased participation  SELF REPORTS Instrument: Formal written document used to collect data Interview schedule: questions asked face to face or by telephone Questionnaire: respondents complete the instrument themselves SELF REPORTS Instrument Strengths Limitations Interviews Response rates Expensive are higher Time Additional Consuming Interviews vs. information can Anonymity is Questionnaires be gathered compromised through observation Questionnaires Cheaper Low response Time efficient rates Can provide No anonymity observational data  SELF REPORTS Scales: Summated rating scales (Likert Scale), Visual Analog Scale Challenges With Scales - response set biases – Social Desirability Response, Extreme Response, Acquiescence Response  SELF REPORTS Strengths Weaknesses Direct Questions Validity and Evaluation: Strength and Weaknesses and Answers Accuracy of Critiquing Data about answers Is self reporting the best option for behaviours you can this research? not observe, Appropriate degree of structure? personal belief and Did the research use the best mode feelings for collecting data? Do the questions in the instrument cover the whole problem under investigation? If there is a scale, is it appropriate? Does the scale capture the variable?  OBSERVATIONAL METHOD Observational method: the structured and unstructured technique of collecting data about behaviours and events Methods of structured observation: category system, a checklist, a rating scale Pp. 178-179 Observational sampling O B S E RVAT I O N A L ME THO D Strengths Weaknesses Works well within the Ethical difficulties Evaluation: Nursing Setting - Observed may change Nurses already observe their behaviour when behaviours and actions observed (reactivity) of clients and colleague Bias Unobtrusive Provide depth and variety Humans are instruments  OBSERVATIONAL METHOD Critique: - Is this the proper approach for this data collection? Is there an alternative method? - Were the observers concealed? If no, what effect did this have on the observed? - Did the observer interact with the observed? Did this have an effect on the data collected? - Was the setting appropriate to the behaviour being studied? - How was the data recorded? Was this appropriate? - What steps were taken to minimize observer bias?  BIO-PHYSIOLOGIC MEASURES AS DV OR OUTCOME VARIABLE In Vitro: Data gathered by extracting some In Vivo: Directly within or biophysiologic material on living organisms (BP, and testing it in a body T) laboratory (hormone level, bacterial counts etc)  BIO-PHYSIOLOGIC MEASURES Evaluation Advantages: Measures are accurate, precise and objective. The instruments provide valid measure of the targeted variables Disadvantages: The measures can be invasive Critique Is this approach appropriate? Is there an alternative? Was the proper instrument used? Is there a better alternative? Does the research have the right skills to interpret the measure?  DATA COLLECTION METHODS Summary 3 main types of data collection: Self Reports, Observations, Biophysiologic Measures These methods can not be interchanged; they are specific to the type of data There are benefits and limitations to each one Examining the data collection methods is a critical part of evaluating and critiquing research articles DATA QUALITY IN QUANT RESEARC H  MEASUREMENT OF VARIABLES The assignment of numbers to represent the amount of an attribute present in an object or person, using Example: specific rules Job satisfaction Advantages: Removes guesswork A Provides precise information Less vague than words or B Which is more precise?  ERRORS OF MEASUREMENT Obtained Score = True score + Error Obtained score: An actual data value for a participant (e.g., anxiety scale score) True score: The score that would be obtained with an infallible measure Error: The error of measurement, caused by factors that distort measurement (see next slide for examples)  FAC TORS THAT C ON TRI BU TE TO ERRO RS O F MEA SUREM ENT Situational contaminants (e.g., lighting) Transitory personal factors (e.g., fatigue) Response-set biases Administration variations  KEY CRITERIA FOR EVALUATING QUANTITATIVE MEASURES Reliability Validity  RELIABILITY The consistency (absence of variation) and accuracy with which an instrument measures the target attribute (some form of replication is used to assess reliability)  T H R EE AS P E CT S O F R E L IA B I LI T Y C A N B E EVAL UAT E D Test-retest reliability, or Stability* Inter-rater reliability (e.g., OSCE exam) Internal consistency *Do not confuse a psychometric assessment of instruments (scales, questionnaires) and the use of these instruments for data collection as part of research design (eg, pretest-posttest design)  RELIABILITY ASSESSMENTS Reliability assessments involve computing a reliability coefficient Reliability coefficients can range from.00 to 1.00 Coefficients below.70 are considered unsatisfactory Coefficients of.80 or higher are desirable Internal consistency is the most widely used reliability approach, specifically Cronbach’s α  The degree to which an instrument measures VALIDITY what it is supposed to measure  FOUR ASPECTS OF VALIDITY Face Content Criterion Construct validity validity validity validity  FACE VALIDITY Refers to whether the instrument Based on judgment, no objective looks as though it is measuring the criteria for assessment appropriate construct If it looks like a duck…  The degree to which an instrument has an appropriate sample of items for the construct being measured (example: the NCLEX RN exam) CONTENT VALIDITY Evaluated by expert evaluation, often via a quantitative measure: the content validity index  CRITERION VALIDITY The degree to which the instrument is related to an external criterion (Is a measure a good reflection of a ‘gold standard’?) Validity coefficient is calculated by correlating scores on the instrument and the criterion Validity coefficient r should be.70 or higher  CONSTRUCT VALIDITY Concerned with the questions: What is this instrument really measuring? Does it adequately measure the construct of interest? Ex.: does an Empathy Scale measure true empathy or something else like mood?  SOME METHODS OF ASSESSING CONSTRUCT VALIDITY Known-groups technique Relationships based on theoretical predictions (e.g., test a Courage Measure on a group of people who have shown great courage in their life) Factor analysis a statistical procedure CRITIQUING RESEARCH REPORT questions in Box 10.2 on p. 183