HE 201 Research Methods Past Paper October 2024 (PDF)

Questionnaire Development, HE 201: Research Methods Surveys & in the Health Sciences October 23, 2024 Interviews Dr. Ketan Shankardass Associate Professor, Department of Health Sc...

Questionnaire Development, HE 201: Research Methods Surveys & in the Health Sciences October 23, 2024 Interviews Dr. Ketan Shankardass Associate Professor, Department of Health Sciences Type 1 and Type 2 error ◦ Type 1 () error ◦ occurs when a study population yields a statistically-significant test result even though a significant difference or association does not actually exist in the source population – When  is set at 5%, about 1 in 20 statistical tests will result in a type 1 error Type 2 () error ◦ occurs when a statistical test of data from a study population finds no significant result even though a significant difference or association actually exists in the source population ◦ The best way to minimize the likelihood of type 2 errors is to have a large sample size ◦ When  is set at 20%, 1 in 5 statistical tests will result in a type 2 error ◦ The inverse is what we refer to as a study’s “power” to detect a significant difference, if it exists. https://sample-size.net/sample- size-means/ Today’s objectives: To introduce the basic To review introductory methods of principles of survey and questionnaire interview design development, including how to properly write a clear survey question Chapters 21, 22   A questionnaire (or survey instrument) is a Questionnaire series of questions Design used as a tool for Overview systematically gathering data from study participants Less this… More like this… (Jacobsen, 2021)  The first set of questionnaire items The first step in designing a typically are ones that enable the questionnaire is to list the topics researcher to confirm that that the survey instrument must participants meet the eligibility cover criteria for the study Several questions may be required Theoretical frameworks can inform to accurately assign participants to the diversity of questions that may key exposure and disease be relevant to include in the survey categories instrument Questionnaire Content iarc.fr  The first set of questionnaire items The first step in designing a typically are ones that enable the questionnaire is to list the topics researcher to confirm that that the survey instrument must participants meet the eligibility cover criteria for the study Several questions may be required Theoretical frameworks can inform to accurately assign participants to the diversity of questions that may key exposure and disease be relevant to include in the survey categories instrument Questionnaire Content Practice: Write out topics related to the following… Is there an association between television watching and Alzheimer’s Disease in Canadians over the age of 65? Question areas   The next step is to decide which types of questions are most appropriate:  Closed-ended questions (close- Closed- ended questions) allow a limited number of possible responses: ended vs.  Date/time  Numeric Open-  Categorical:  ordinal (ranked) or ended  nominal (unordered)  Paired comparisons questions  Rank-ordering  Open-ended, or free response, questions allow participants to explain their answers at length (unlimited number of responses).  Do you currently have a regular primary health care provider, that is, someone you can go to for routine medical check-ups or for specific health concerns? A regular Close-ended primary health care provider can question include, but is not limited to, a (example) family doctor, a nurse practitioner, a walk-in clinic, or interdisciplinary health centre. ❑ Yes ❑ No  Tell us about your experiences trying to access a regular primary health care provider Open-ended _____________________________________ question _____________________________________ (example) _____________________________________ _____________________________________ _____________________________________ Additional examples of types of questions Likert Responses  Anonymity is the inability of a When anonymity is important, avoid For many types of studies, there is no participant’s identity to be discerned from asking questions that could allow the need to collect names, contact his or her responses to a survey instrument participant’s identity to be determined information, or other identifiable or records in a database based on their responses. information. Anonymity (Jacobsen, 2021)   Many questionnaires start with easy or at least general questions before moving to more difficult or sensitive questions  It is often best to group similar questions with similar response types, so that they are asked consecutively – Sometimes it is better to mix up such questions to Order of prevent habituation, an error that occurs when Questions participants become so accustomed to giving a particular response (like “agree… agree… agree…”) that they continue to reply with the same response even when that does not match their true perspectives  Survey developers must carefully consider how previous questions could taint the answers to later ones  Both paper-based and electronic data entry forms require careful attention to layout and spacing  Paper-based and electronic survey pages should be Layout and carefully checked for grammatical errors, misspellings, missing questions, gaps in logic, Formatting unclear instructions, formatting errors, and other organizational issues  The selected typefaces and fonts must be readable and sufficiently large   Computer-based data entry programs allow a researcher to force participants to provide an answer to selected questions, but required fields Layout and force a person who does not want to provide an Formatting answer to one required question to quit the survey and leave all subsequent items unanswered (Cont.)  Skip logic codes can automatically hide irrelevant questions from participants based on their responses to filter questions Methods for Collecting Data  An interview is the  A self-administered process of a survey uses a researcher verbally questionnaire form that asking a participant participants complete by themselves, using either a questions and paper-and-pencil version recording that or an online version of the person’s responses survey instrument   The goal is to maximize the participation rate among sampled individuals so as to yield a study population that is reasonably representative of the source population.  Provide multiple opportunities/invitations to participate, (e.g., follow-up mailings, e- mails).  Make participation as easy as possible Recruiting  Consider appropriate incentives.  Incentives such as small gifts may be an Methods effective means of encouraging participation among those invited to be in a study  The recruitment methods are often paired with the data collection method (such as using online methods to recruit for an Internet-based survey). Examples of Methods for Contacting Members of the Sample Population Data Recording Methods  Paper-based surveys must later be scanned or typed into computers; this can be expensive and time-consuming, but they also allow a large number of people to take the survey at one time and place.  Computer-assisted surveys input survey responses directly into a computer and can have built-in checks and “skips,” but not all populations are comfortable with computers.  Provide clear instructions about the acceptable types of answers for each Types of question. Responses Add a response for “Not applicable” when relevant.  1. Example:  Are you aware of the recent Principles of amalgamation of the OCPH and writing good ESTHU’s into the new SWHU? survey questions ❑Yes (Dillman, 2007) ❑No  1. Choose simple over specialized words  Better: Principles of  Are you aware of the recent merger of writing good Oxford County Public Health and the Elgin- survey questions St. Thomas Health Unit into the new Southwest Public Health Unit? ❑Yes ❑No  2. Example:  How many people were living or staying at this residence on Saturday, March 3rd, 2018? To make sure each person in Canada is counted only once, it very important to: Principles of  Include everyone who lives here whether writing good related to you or not, and anyone staying temporarily who has no permanent place to survey questions live;  But not include anyone away at university or college, away in the Armed Forces, in a nursing home, hospice, mental hospital, correctional facility or other institution. ____________________ # people  2. Choose as few words as possible to pose a question  Better:  How many people were living or staying at your home or residence on Saturday, March 3rd, 2018? Principles of  Include everyone who lives here writing good whether related to you or not, and anyone staying temporarily who survey questions has no permanent place to live;  Do not include anyone away at university or college, away in the Armed Forces, in a nursing home, hospice, mental hospital, correctional facility or other institution. ____________________ # people  3. Example:  Do you agree or disagree with the following statement: Wait times for surgeries and receiving care in emergency departments should be shorter. Principles of ❑Strongly disagree writing good ❑Disagree survey questions ❑Neutral ❑Agree ❑Strongly agree  3. Avoid double- barreled questions  Better:  Do you agree or disagree with the following statements:  Wait times for Principles of  Wait times for emergency writing good surgeries should be departments shorter. should be shorter. survey questions ❑Strongly disagree ❑Strongly ❑Disagree disagree ❑Neutral ❑Disagree ❑Agree ❑Neutral ❑Strongly agree ❑Agree ❑Strongly agree  4. Example: Lack of a primary care provider in Ontario is not perceived to be a problem for most Principles of individuals. writing good ❑Strongly disagree survey questions ❑Disagree ❑Neutral ❑Agree ❑Strongly agree  4. Avoid double-negatives: Better: Lack of a primary care provider in Principles of Ontario is perceived to be a problem for most Ontarians. writing good ❑Strongly disagree survey questions ❑Disagree ❑Neutral ❑Agree ❑Strongly agree Principles of 5. Example: writing good Rank the following 30 factors survey questions in order of importance  5. Avoid complicated Principles of instructions writing good survey questions Better: Rank the following 5 factors in order of importance  6. Example: How do you feel about people who choose the bad habit of smoking in Principles of public? writing good ❑ Agree with it survey questions ❑ Somewhat agree with it ❑ Neutral, no opinion ❑ Disagree with it ❑ Somewhat disagree with it  6. Avoid leading questions: Better: In general, how do you feel about Principles of cigarette smoking in public spaces (e.g. parks, sidewalks)? writing good ❑ Agree with it survey questions ❑ Somewhat agree with it ❑ Neutral, no opinion ❑ Disagree with it ❑ Somewhat disagree with it  7. Example: How many times have you had a surgical procedure? Principles of ❑ 1 – 3 times writing good ❑ 4 – 6 times survey questions ❑ More than 6 times  7. Avoid making assumptions about the participant Example: Principles of Better: writing good Have you ever had a surgical survey questions procedure? ❑ Yes ❑ No How many separate times have you had a surgical procedure? ___________ number of times  8. Example: What is your annual household Principles of income? writing good ❑ Less than $25,000 survey questions ❑ $25,000 - $49,999 ❑ $50,000 - $74,999 ❑ $75,000 or more  8. Avoid objectionable questions Example: Better (CCHS):  Although many health expenses are covered by health insurance, there is still a Principles of relationship between health and income. writing good Please be assured that, like all other survey questions information you have provided, these answers will be kept strictly confidential. What is your annual household income? ❑ Less than $25,000 ❑ $25,000 - $49,999 ❑ $50,000 - $74,999 ❑ $75,000 or more  9. Example: Which one of the following age Principles of groups do you belong to? writing good ❑ 16-25 survey questions ❑ 25-40 ❑ 40-50 ❑ 50-60 ❑ 60+  9. Ensure response options are mutually exclusive Better : Principles of Which one of the following age writing good groups do you belong to? survey questions ❑ 16-24 ❑ 25-39 ❑ 40-49 ❑ 50-59 ❑ 60+  Several widely used and validated tests are available to researchers Commercial Research Some of these tools are Tools free of charge, but most are commercial products that require payment for use www.mindgarden.com   Back translation (or double translation): one person translates the questionnaire from the original language to a new language, then a second person then translates the survey instrument in the new language back into the original language Translation  A comparison of the original version of the survey with the back-translated version will reveal where the second- language translation does not match the intended meaning of the original version  Important to get the language right but also the conceptual and colloquial meaning of the original question  A pilot test (or pretest) is a small-scale preliminary study conducted to evaluate the Pilot feasibility of a full-scale Testing research project The survey instrument is revised based on these observations   Collecting qualitative data & qualitative data Next Class analysis  Chapters 23, 32  Moni Sadri-Gerrior, PhD (Candidate)