Questionnaire Design and Analysis PDF

EXPERIMENTAL DESIGN, SAMPLING TECHNIQUE, ANALYSIS FOR QUESTIONNAIRE STUDY WHAT IS A QUESTIONNAIRE? HOW TO DESIGN A QUESTIONNAIRE? HOW TO SAMPLE POPULATION? HOW TO ANALYSIS A QUESTIONNAIRE DATA? EXPERIMENTAL QUESTIONNAIRE...

EXPERIMENTAL DESIGN, SAMPLING TECHNIQUE, ANALYSIS FOR QUESTIONNAIRE STUDY WHAT IS A QUESTIONNAIRE? HOW TO DESIGN A QUESTIONNAIRE? HOW TO SAMPLE POPULATION? HOW TO ANALYSIS A QUESTIONNAIRE DATA? EXPERIMENTAL QUESTIONNAIRE STUDY DESIGN, SAMPLING TECHNIQUE, ANALYSIS IN RESEARCH RESEARCH: (Final Year Project) methods to FIELD BASED LABORATORY gather data STUDY STUDY INTRODUCTION: Steps in deciding your research topic Decide which Forestry Field you want to study (Scope) 1 Identify what Problem/s you want to address (Statement of Problem) and justify why do you need to research it (Justification) 2 Define your specific objective/s and develop your hypothesis 3 Select your sample population, design your experiment, plan your sampling, collect and analyze data (Methodology) 4 Data Interpretation (Results) 5 Discuss your findings, are they answering your research questions? (Discussion) 6 Conclusion and Recommendation 7 An example of research scopes and nature of study QUESTIONNAIRE LABORATORY FIELD BASED Forest degradation Soil analysis Observation Forest law and Insect, pest and Tree growth certification diseases Wildlife Tourist satisfaction Osmotic potential Forest restoration Wood chemical Recreation activity Agroforestry practices analysis Marketing of plywood GIS and remote Social and community sensing An example of research scopes and nature of study QUESTIONNAIRE LABORATORY FIELD BASED Forest degradation Soil analysis Observation Forest law and Insect, pest and Tree growth certification diseases Wildlife Tourist satisfaction Osmotic potential Forest restoration Agroforestry practices Wood chemical Recreation activity Marketing of plywood analysis Social and community GIS and remote sensing WHAT IS A QUESTIONNARE? A questionnaire is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents (Wikipedia, 2019). A written set of questions that are given to people in order to collect facts or opinions about something (Merriam- Webster, 2019). A set of printed or written questions with a choice of answers, devised for the purposes of a survey or statistical study (Oxford Advanced Learner’s Dictionary, 2019).  HOW TO DESIGN QUESTIONNARE? Questionnaire Design Process Determine survey objectives, Determine the Establish Determine the data questions response questionnaire flow Evaluate the resources and collection method questionnaire constrains format and layout Obtain approval of Implement the Pretest and revise Prepare final copy all relevant parties survey Source: Daniella McLaughlin, 2019 Questionnaire Design Process (Cont.) Step 1: Determine survey objectives, resources, and constraints - Survey objectives: outline of the decision-making information sought through the questionnaire. Step 2: Determine the data-collection method – Survey data can be gathered, in variety of ways (e.g. Internet – Google Forms, SurveyMonkey, Typeform, Jotform, telephone, mail, or self- administration) and the survey method impacts questionnaire design. Top online survey tools and apps Questionnaire Design Process (Cont.) Step 3: Determine the question response format. Three major types of questions are used in marketing research: A. Open-ended questions: questions to which the respondent replies in her/his own words. B. Closed-ended questions: questions that require the respondent to choose from a list of answers C. Scaled-response questions: closed- ended questions in which the response choices are designed to capture the intensity of the respondent’s feeling. Types of Questionnaire Open ended questions Close ended questions Questions that should be Questions that should be answered with long responses answered with short responses Answers are often descriptive Answers are often short and and explanatory factual Questions begin with words like Questions begin with words like how, why, explain, describe, etc. is, would, do, what, etc. Essay questions Multiple choice questions Take a long time to answer Can usually be answered quickly Open ended questions 1. What do you like best about our product? 2. What will you like to see improve on our services? Closed ended questions 1. Do you like our product? Yes, or no? 2. Are you satisfy with our services? Yes, or no? Scale ended questions 1. On scale of 1 to 5, with 1 very bad and 5 very good, what was your perception on our product? 2. On scale of 1 to 7, with 1 not satisfy and 5 very satisfy, please rate every services listed here. Simple questionnaire with both close and open-ended questions. Questionnaire Design Process (Cont.) Step 4: Decide on the question wording. Four guidelines about the wording of questions are useful to bear in mind: 1. Make sure the wording is clear 2. Avoid biasing the respondent 3. Consider the respondent’s ability to answer the questions 4. Consider the respondent’s willingness to answer the question Questionnaire Design Process Step 5: Establish questionnaire flow and layout. Guidelines concerning questionnaire flow: – Use screening questions to identify qualified respondents – Begin with a question that gets the respondent’s interest – Ask general questions first – Ask questions that require “work” in the middle – Position sensitive, threatening, and demographic questions at the end – Put instructions in capital letters – Use a proper introduction and closing Questionnaire Design Process Step 5: Establish questionnaire flow and layout – A proper introduction and closing Example: Introduction/opening Hello, my name is …………., and I’m calling from (university). Today/Tonight we are calling to gather opinions regarding (general subject), and are not selling anything. This study will take approximately (length) and may be monitored (and recorded) for quality purposes. We would appreciate your time. May I include your opinions? Closing Thank you for your time and cooperation. I hope this experience was a pleasant one. Please remember that your opinion counts! Have a good day/evening. Questionnaire Design Process Step 6: Evaluate the 1. For each question, is the question questionnaire. necessary? Issues should be considered: 2. Is the questionnaire too long? 3. Will the questions provide the information needed to accomplish the research objectives? Questionnaire Design Process Step 7 Step 8 Step 9 Step 10 Obtain approval of Pretest with the Prepare final Implement the all relevant parties target respondents questionnaire copy survey – Example: for a and revise – Include precise new product/ instructions: where services to interview, target questionnaire respondents, and should get the when to approval from the respondents' test relevant items (if any) authorities Questionnaire Design Process Costs, Profitability, and Questionnaires The role of the questionnaire in survey research costs can be a decisive one. If a research firm overestimates data-collection costs, chances are that it will lose the project to another supplier. Most data-collection costs are associated not with conducting the actual interview, but with finding a qualified respondent (failed attempts, cooperation problems, screener determines respondent not eligible, or respondent terminated during interview).  Example: Questionnaire on Forest Degradation SAMPLING TECHNIQUES Example: Questionnaire on Forest Degradation Sample populations This survey is aimed at collecting All countries in the world? information on how forest Countries in Asia? degradation is defined and assessed in various countries and organisations Only tropical countries? as an input to an International Study Southeastern countries? on Forest Degradation as part of the Global Forest Resources Assessment 2010. The study is part of a joint initiative of the Collaborative Partnership on Forests (CPF) on harmonising forest definitions and on streamlining forest-related reporting. SAMPLING TECHNIQUES IN QUESTIONNAIRE Sampling can be explained as a specific principle used to select members of population to be included in the study. It has been rightly noted that “because many populations of interest are too large to work with directly, techniques of statistical sampling have been devised to obtain samples taken from larger populations.” In other words, due to the large size of target population, researchers have no choice but to study a number of cases of elements within the population to represent the population and to reach conclusions about the population (see Figure 1 below). Population, sample and individual cases Sampling Process in Data Collection 1. Defining target 2. Choosing 3. Determining population. sampling frame. sampling size. 4. Selecting a 5. Applying the chosen sampling sampling method. method in practice.  Step 1. Defining target population. Target population represent specific segment within wider population Sampling that are best positioned to serve as Process in Data a primary data source for the Collection research. For example, for a dissertation entitled ‘Impact of social networking sites on time management practices amongst university students in Malaysia” target population would consist of individuals residing in Malaysia.  Step 2. Choosing sampling frame. Sampling frame can be explained as Sampling a list of people within the target population who can contribute to Process in Data the research. For a sample Collection dissertation named above, sampling frame would be an extensive list of Malaysian university students.  Step 3. Determining sampling size. This is the number of individuals from the sampling frame who will participate in the primary data collection process. The following observations need to be taken into account when determining sample size: a) The magnitude of sampling error can be diminished by Sampling increasing the sample size. b) There are greater sample size requirements in survey- Process in Data based studies than in experimental studies. c) Large initial sample size has to be provisioned for mailed Collection questionnaires, because the percentage of responses can be as low as 20 to 30 per cent. d) The most important factors in determining the sample size include subject availability and cost factors For example, for the same research of ‘Impact of social networking sites on time management practices amongst university students in Malaysia’ sample size could be determined to include 200 respondents.  Step 4. Selecting a sampling method. This relates to a specific method according to which 200 university students in Malaysia are Sampling going to be selected to participate in Process in Data research named above. Collection Step 5. Applying the chosen sampling method in practice. Category of Sampling Techniques Probability sampling involves Non-probability sampling random selection, allowing you to involves non-random selection make strong statistical inferences based on convenience or other about the whole group. criteria, allowing you to easily collect data. Types of Sampling Design Sampling methods are broadly divided into two categories: probability and non-probability. In probability sampling every member of population has a known chance of participating in the study. Probability sampling methods include simple, stratified systematic, multistage, and cluster sampling methods. In non-probability sampling, on the other hand, sampling group members are selected on non-random manner, therefore not each population member has a chance to participate in the study. Non- probability sampling methods include purposive, quota, convenience and snowball sampling methods. Probability versus Non-Probability Sampling  Definition/ Technique Advantages Disadvantages Explanation Sample group members are High level of sampling Highly effective if all subjects Random selected in a random error when sample size is participate in data collection manner small Effective representation of all Knowledge of strata subgroups Representation of specific membership is required Stratified Precise estimates in cases of subgroup or strata Complex to apply in homogeneity or heterogeneity practical levels within strata Time efficient Including every Nth member High sampling bias if Systematic of population in the study periodicity exists Cost efficient  Definition/ Technique Advantages Disadvantages Explanation Complex to conduct High level of Sampling conducted on several Impacted by limitations of Multistage flexibility at stages cluster and stratified sampling various levels methods Group-level information needs to Clusters of participants Time efficient be known representing population are Cluster Usually, higher sampling identified as sample group Cost efficient errors compared to members alternative sampling methods Time efficiency Sample group members are Unscientific approach Judgement selected on the basis of judgement Samples are not of researcher highly Personal bias representative  Definition/ Technique Advantages Disadvantages Explanation High level of subjectivity Sample group members are High level of reliability than Quota selected on the basis of random sampling Difficult to estimate sampling specific criteria Usually cost-effective error High levels of simplicity and Obtaining participants Highest level of sampling error ease Convenience conveniently with no requirements whatsoever Selection bias Usefulness in pilot studies Over-representation of a Sample group members particular network Possibility to recruit hidden Snowball nominate additional members Reluctance of sample group population to participate in the study members to nominate additional members  Types of questions (open ended/close ended/scale response) determine types of analysis used Step 1: Assign code to your HOW TO ANALYZE questionnaire. For example; QUESTIONNAIRE? Male=1, Female=2 Step 2: Transfer information into spreadsheet or statistical package example: Statistical package for Social Science (SPSS) HOW TO ANALYZE QUESTIONNAIRE? Step 3: Strategy for analysis, link to your research questions Example: Category type questions: frequency of a response using percentage, using bar or pie charts, Continuous type questions: measures of central tendency (average or mean, median and mode); dispersion or distribution (such as range, standard deviation) Bivariate analysis, looking at questions on how they interact or are different Crosstabulation, association between two category type questions, for example, between gender and age, that can be presented in a simple table and bar chart HOW TO ANALYZE QUESTIONNAIRE? Step 3: Strategy for analysis, link to your research questions Example: Category type questions: frequency of a response using percentage, using bar or pie charts, Continuous type questions: measures of central tendency (average or mean, median and mode); dispersion or distribution (such as range, standard deviation) Bivariate analysis, looking at questions on how they interact or are different Crosstabulation, association between two category type questions, for example, between gender and age, that can be presented in a simple table and bar chart HOW TO ANALYZE QUESTIONNAIRE? Step 3: Strategy for analysis, link to your research questions. Example: Category type questions: frequency of a response using percentage, using bar or pie charts Descriptive Data Bivariate analysis, looking at questions on how they interact or are different Crosstabulation: association between two category type questions, for example, between gender and age, that can be presented in a simple table and bar chart Descriptive Data Bivariate analysis:, category question and a continuous questions, comparison of means, compare them in a clustered bar chart Descriptive Data Scatterplot: if we have two continuous type questions and to see their relationship, relationship shown in a graph showing negative or positive relationship Statistical Data Inferential statistics to test the significance of tests, basis for making predictions, achieved by chance, set a probability level of 95% Sample of a recent questionare