Lecture 5: Secondary Analysis / Official Statistics - Mixed Methods

Lecture 5: Secondary analysis / Official statistics Mixed methods TOMAS MAINIL 2023-24 Agenda 1. Secondary analysis and official statistics a) Secondary analysis a) Secondary analysis b) Advantages & disadvantages b) Official statistics...

Lecture 5: Secondary analysis / Official statistics Mixed methods TOMAS MAINIL 2023-24 Agenda 1. Secondary analysis and official statistics a) Secondary analysis a) Secondary analysis b) Advantages & disadvantages b) Official statistics a) Official statistics b) Reliability and validity c) Advantages and disadvantages 2. Mixed methods a) Qualitative vs. Quantitative (?) b) Mixed methods c) Argument against mixed methods d) Arguments in favour of mixed methods e) Indicators of quality in mixed methods research Secondary analysis  Primary data is data we collect ourselves.  Secondary data is data that is collected by others.  Secondary analysis is done on secondary data.  In other words, someone else gathered the data – for their own purposes – and then we analyse it for our own purposes.  Those purpose can be related to a very different (type of) research Secondary analysis  Example: Survey Data Netherlands  Stores quantitative data from previous studies  Archives allow you to browse and search for data you need for your research.  http://www.surveydata.nl/ Advantages and disadvantages  Advantages  Saves money and time  Often offers high quality data  Gives an opportunity for longitudinal analysis (analysis of a phenomena over long periods of time)  Gives an opportunity for cross-cultural studies  Allows more time for data analysis  Enables the application of recent theory to old data  Gets more value from the original data  Disadvantages  The data can be very large and complex.  The quality of the data should never be taken for granted.  Variables important to your analysis might be missing. Official statistics  Collected by agencies of the state, in the course of their business  CBS  Eurostat (provide the EU with statistics at European level that enable comparisons between countries and regions). Reliability and validity  Reliability  Definitions, categories and allocated resources change over time  Can reflect priorities of agencies/organizations  Validity  Variation may be caused by factors not studied by official reports  The ecological fallacy  Error of assuming that inferences about individuals or organizations can be made from findings relating to aggregate data.  For example, official statistics might demonstrate a positive relationship between the size of a firm and the number of labour disputes involving strike action.  However, in reality, a particular large firm may show quite low levels of strike activity, while a particular small firm might show a high level. Advantages and disadvantages  Disadvantages  They only reveal ‘top of the iceberg’  The ‘dark figure’ of unrecorded events e.g. unemployed people who do not claim benefits are not officially listed as unemployed  The process used for data collection needs interpretation  Uncertain measurement validity  Due to criticisms of official statistics and their uses, they are largely ignored by business and management research.  Advantages  Some official statistics – like population census data – are accurate by any set of criteria  To reject them because they contain errors is silly, since all measurement in business research is error-prone  The data is gathered unobtrusively, i.e., free from ‘reactive’ effects. Agenda 1. Secondary analysis and official statistics a) Secondary analysis a) Secondary analysis b) Advantages & disadvantages b) Official statistics a) Official statistics b) Reliability and validity c) Advantages and disadvantages 2. Mixed methods a) Qualitative vs. Quantitative (?) b) Mixed methods c) Argument against mixed methods d) Arguments in favour of mixed methods e) Indicators of quality in mixed methods research Qualitative vs. quantitative (?)  Distinction between quantitative and qualitative useful, but …  Research methods can be combined and complement one another.  Research methods more ‘free-floating’ than presumed. Four problems with quantitative / qualitative contrast:  Behaviour vs. meaning  Theory tested in research vs. emergent from data  Numbers vs. words  Artificial vs. natural Qualitative vs. quantitative (?)  Behaviour (quantitative) versus meaning (qualitative), but …  Quantitative research often involves the study of meanings in the form of attitude scales, etc.  Qualitative researchers interpret people’s behaviour regarding norms, values, culture, etc.  Quantitative and qualitative researchers interested both in what people do and think … but investigate these areas in different ways. Qualitative vs. quantitative (?)  Theory tested in research versus emergent from data;  Quantitative research is not solely concerned with the testing of ideas that have previously been formulated (theories / hypotheses).  This fails to recognize the creative work that goes into the analysis of quantitative data and into the interpretation of findings.  From the data new ideas and theories can emerge. Qualitative vs. quantitative (?)  Numbers vs. words  Qualitative researchers sometimes undertake limited quantification of their data:  To help uncover the generality of the phenomena being described.  Because this provides greater precision into estimates of frequency than using terms like ‘many’, ‘often’, and ‘some’. I hate people that say “Age is only a number” … Age is clearly a word Qualitative vs. quantitative (?)  Artificial vs. natural  Quantitative research seen to give an artificial account of how the social world operates  Qualitative research is often viewed as more naturalistic.  However, when qualitative research is based on interviews, the depiction ‘natural’ might be less applicable  This is because interviews and focus groups still have to be arranged and interviewees have to be taken away from their normal activities.  https://thispersondoesnotexist.com/ Argument against mixed methods  The embedded methods / paradigm argument  Qualitative and quantitative research are separate, incommensurable paradigms  Mixed methods are not feasible and not desirable  Even when combined they are incompatible Hotel Mama  After a reading an article about the link between stress experiences by employees and their productivity the owner of “Hotel’ Mama orders a mixed-methods investigation.  Hotel Mama wants to find out to what degree its employees experience stress and its causes. For this they use:  An online self-completed questionnaire among all of their 100 employees.  Semi-structured interviews with 10 employees. Arguments in favour of mixed methods  Arguments in favour of mixed methods  Triangulation  Results of one method/research strategy cross-checked against the results of another  Structured interviews, survey, semi-structured interviews all provided different perspectives  For Hotel Mama, the survey might show that the employees experience stress. The semi-structured interviews could show that the stress is caused by work, home and personality related factors, and how that works out for individuals.  But what to do if results are inconsistent? … What is the truth? Arguments in favour of mixed methods  Qualitative research facilitates quantitative research  By providing hypotheses: an unstructured, open-ended approach is helpful in generating hunches for testing through experimentation or survey.  By aiding measurement: focus groups and semi-structured interviews can provide the measurements of concepts to be tested in quantitative research, leading to much more appropriate specification of questions.  For Hotel Mama the semi-structured interviews might produce insights into potential work-related causes of stress, which can then be tested in the survey. Arguments in favour of mixed methods  Quantitative research facilitates qualitative research  By providing a basis for representative sampling in qualitative research:  E.g. samples of people or companies, in particular sets of characteristics, for in-depth interviewing or case study.  For Hotel Mama the survey might give insights into the differences between stress between people working in different departments, to which the sample of interviewees can be adjusted. Arguments in favour of mixed methods  Filling in the gaps  When neither research strategy can provide all the answers  e.g. particular methods do not provide access to required information / groups of people  Quantitative and qualitative methods both compensate for the other’s weaknesses  For Hotel Mama the survey can be used to determine the stress level for all employees, while the semi-structured interviews provide an in-depth and personal perspective.  Static and processual features  Quantitative research can uncover regularities  Qualitative research reveals social processes  For Hotel Mama the semi-structured interviews can provide insights into processes that have led to changes in the stress employees experience. Arguments in favour of mixed methods  Researchers’ and participants’ perspectives  Quantitative methods test researcher’s theories.  Qualitative methods make participants’ meanings the center of attention.  The same level of work-related pressure can be experienced differently by different people. Some people enjoy a certain degree of stress. Stress, and the meaning people give to that, is a subjective concept. In the semi-structured interviews that can explored in depth and on a personal level.  Some research studies require both perspectives. Arguments in favour of mixed methods  The problem of generality  Some quantification of findings from qualitative research can help to uncover the generality of phenomena (Silverman, 1984,1985).  Quantification can counter criticisms of anecdotalism in qualitative studies.  For Hotel Mama just having 10 interviews might lead to critique on the research, e.g. because of the (lack of) representativeness of the people interviewed, Arguments in favour of mixed methods  Qualitative research may facilitate interpretation  Quantitative researchers constantly face the problem of explaining the relationships between variables and demonstrate causality  Qualitative research can discover the presence and role of intervening variables  Hotel Mama: Relationship between age and stress: higher age, more stress experienced  Intervening variable: Type of job (higher age, job with more responsibility, more stress) Indicators of quality in mixed methods research  Is the research well designed and conducted?  Are the methods appropriate to the research questions?  Is there an explicit rationale for the mixture?  Are the separate components integrated?  Is there a detailed account of the entire research process?  Are resources spread too thinly, or unevenly?  Are the researchers more skilled in one strategy than another? Lecture 6: Quantitative data analysis: An introduction DR. WIM STRIJBOSCH Agenda 1. Data analysis 2. Types of variables 3. Univariate analysis 4. Measures of central tendency 5. Measures of dispersion 6. Bivariate analysis 7. Pearson’s r 8. Other measures 9. Multivariate analysis 10. Statistical significance 11. Testing statistical significance Data analysis  Think about data analysis at an early stage in the research process, since you can’t apply any technique to any variable.  Decisions about methods and sample size affect the kinds of analysis you can do. Types of variables  Nominal  qualitatively different categories - cannot objectively be ranked (e.g. gender, political party, eye color)  Ordinal  categories can be ranked, but with unequal distances between them (e.g. education level)  Interval  regular distances between all categories in range, no fixed zero point (e.g. temperature)  Ratio  regular distances between all categories in range, fixed zero point (e.g. weight)  Important: In business research interval and ratio are not being distinguished. Types of variables  Other often used distinctions:  Categorical: limited number of different categories (e.g. political party, eye colour, etc.) (= nominal)  Dichotomous: only two categories (e.g. younger than 20, 20 or older) Types of variables Types of variables  Has children, Yes or No?  DICHOTOMOUS  Eye color  NOMINAL  Education level (MBO, HBO, University)  ORDINAL  Income (in Euro’s)  RATIO  Age  RATIO  Job satisfaction (5 groups: strongly agree, agree, neutral, don’t agree, strongly disagree)  ORDINAL  Job satisfaction (average score on a scale from 0 to 10)  RATIO Univariate analysis  Univariate analysis is analysis of one variable at a time n %  Frequency tables GENDER Male 433 28.4  Number of people or cases in each category Female 1089 71.6 Total 1522 100 AGE  Often expressed as percentages of sample 18-20 years 408 26.8 21-24 years 637 41.9  ratio data needs to be grouped 25-29 years 191 12.5 30-39 years 121 8 40-49 years 83 5.5 50 years or older 82 5.4 Total 1522 100 PROFESSION full-time employed 327 21.5 part-time employed 79 5.2 unemployed 41 2.7 student with income 165 10.8 student without income 910 59.8 Total 1522 100 EDUCATION High school degree 686 45.1 Bachelor degree 315 20.7 Master degree 461 30.3 post-Master degree 60 3.9 Total 1522 100 Univariate analysis  Diagrams  pie chart (nominal or ordinal variables)  bar chart (nominal or ordinal variables)  histogram (interval/ratio variables) Measures of central tendency  One figure that is typical for a distribution of values:  Mean (average)  sum all values in distribution, then divide by total number of values  vulnerable to outliers  Median  middle point within entire range of values (if there is an even number of values, the median is calculated by taking the means of two middle numbers of the distribution)  not distorted by outliers  Mode  most frequently occurring value Measures of central tendency  Ages:  2, 5, 15, 17, 10, 15, 13  Mean / Average  (2 + 5 + 15 + 17 + 10 + 15 + 13) / 7 = 11  Median  2, 5, 10, 13, 15, 15, 17  Mode  15 Measures of dispersion  Dispersion means the amount of variation in a sample.  The range is the difference between the minimum and maximum values in a sample.  2, 5, 15, 17, 10, 15, 13  Range = 17 – 2 = 15  The standard deviation is the average amount of variation around the mean, reducing the impact of extreme values (outliers).  2, 5, 15, 17, 10, 15, 13 2−11 + 5−11 + 15−11 + 17−11 + 10−11 + 15−11 +(13−11)  Not; 7 2−11 + 5−11 + 15−11 + 17−11 + 10−11 + 15−11 +(13−11)  7 =0 (2−11)2 + 5−11 2 + 15−11 2 + 17−11 2 + 10−11 2 + 15−11 2 + 13−11 2 190  Standard deviation = 7 = 7 = 27.1 = 5.2 Measures of dispersion  Boxplot: provides an indication of both central tendency (the median) and dispersion (the range), indicates the outliers. Bivariate analysis  Analysis of two variables at a time  Searches for correlations  Can not establish causality, but can sometimes infer the direction of a causal relationship if one variable is obviously independent of the other Bivariate analysis – Contingency Tables  Contingency table: Explores relationships between two nominal or ordinal variables  connects the frequencies of two variables All respondents Member Non-  helps you identify patterns of association member N n % n %  Can be used to calculate the X2-value Gender Male 433 429 99% 4 1% Female 1089 500 46% 589 51% Total 1522 929 593 Age 18-20 years 408 370 91% 38 9% 21-24 years 637 596 94% 41 6% 25-29 years 91 150 79% 41 21% 30-39 years 121 50 41% 71 59% 40-49 years 83 20 24% 63 76% 50 years or older 82 10 12% 72 88% Total 1522 1429 93 Bivariate analysis – Scatter Plot / Pearson’s R  Scatter plot: Explores relationship between two ratio variable Bivariate analysis – Scatter Plot / Pearson’s R  Pearson’s R: Explores the strength and direction of the relationship between the ratio variables, in the sample  Relationships must be linear for the method to work, so plot a scatter diagram first Bivariate analysis – Pearson’s R  The Pearson’s (or correlation) coefficient shows the strength and direction of the relationship between two ratio variables, in the sample  Coefficient (r) lies between -1 and +1  The closer to 1, the stronger the relationship (= correlation)  Positive or negative indicates the direction of the relationship Multivariate analysis  Three or more variables  A third variable might be moderating the relationship between two variables  e.g. correlation between age and stress could be moderated by type of job Statistical significance  A test of statistical significance allows the analyst to estimate to what extent the finding of a statistical relationship in the data (e.g., the more cigarettes you smoke, the shorter you live) is based on pure chance or whether there is a true relationship between these variables  Tests can be prone to error (see e.g., an erroneous pregnancy test)  Type-I versus Type-II error  The probability that a test is based on chance alone is given by the p-value  Generally acceptable if it is below 5% Statistical significance  The lady tasting tea experiment  A lady claims that when drinking a cup of tea, she can taste whether tea or milk has been poured in first  There is a 50% probability that she gets it right on the basis of luck alone  However, if you increase the number of cups, this probability becomes lower  0,5 x 0,5 x 0,5 x 0,5 x 0,5 x 0,5 = 0,016  The lady manages to get all 6 cups right  The probability of getting that exact outcome is very very low, so then there must be something else at stake than chance alone… Statistical significance  How do we determine if a relationship between two variables, which we find in the sample, signifies an actual relationship or just a lucky shot?  Step 1: Is there a relationship between two variables, in the sample?  Men practice more sports than women  Lower educated people practice less sports than higher educated people  E.g. based on contingency table (X2) or scatter plot (Pearson’s r)  Step 2: Is this relationship shown by the data based on chance alone, or is there an actual relationship?  This is the conclusion that is most important Statistical significance  SPSS calculates the level of significance for you  A chance of 1% is shown as p =.01  A chance of 5% is shown as p =.05  A chance of 10% is shown as p =.10  SPSS calculates the level of significance based on statistical test, and test values  X2-test: X2-value  Correlation: Pearson’s R  Basically, the p-value tells you to what extent you would find a certain test statistic if it would all be based on chance alone SBM QUANTITATIVE RESEARCH METHODS Dr. Wim Strijsbosch & Dr. Tomas Mainil BUas, 2024-25 Lecture 1: Introduction to quantitative research About Wim Strijbosch About Tomas Mainil  Master in Sociology (University of Antwerp)  Master in quantitative analysis in the Social Sciences (KULeuven)  Phd in Social Sciences (Medical Tourism and Transnational health care)  Post-Graduate studies in big data analytics and data science (KULeuven)  Worked for:  University of Antwerp  Ghent University  NHTV / BUas (Researcher, thesis supervisor, lecturer)  Now: member of the research group Tourism Impacts on Society (RTIS)  Interest fiels: health impacts of tourism  Academic and applied research Agenda 1. Schedule 2. Course material 3. Assignments 4. Evaluation 5. Introduction to Quantitative Research 6. Course Assignment 1 7. Preparation workshop 16-11 8. Contact information  Schedule Week # Day Lecture Preparation 45 (TM & WS) 1 4-11-24 Introduction Nature of quantitative research Course Assignment 1 46 (WS) 2 11-11-24 Sampling Cp.7, 8 47 (TM) 3 18-11-24 Structured interviewing Cp. 9, 10 Self-completed questionnaires 48 (WS) 4 25-11-24 Asking questions Cp. 11,12,13 Structured observation Content analysis Course assignment 2 49 (TM) 5 2-12-24 Secondary analysis & official Cp. 14, 26, 27 statistics Mixed methods research 50 (WS) 6 9-12-24 Quantitative data analysis Cp. 15  Schedule Week # Day Workshop Preparation 46 (TM) 1 14-11-24 Practice course assignment 1 Explained during Lecture 1 47 (WS) 2 21-11-24 Sampling Explained during Lecture 2 48 (TM) 3 28-12-24 Structured interviewing Explained during Lecture 3 49 (WS) 4 5-12-24 Structured observation Explained during Lecture 4 50 (TM) 5 12-12-24 Mixed Methods Explained during Lecture 5 51 (TM & WS) 6 19-12-24 Question and answers Explained during Lecture 6 Course material  Bryman, A., & Bell, E. (2015). Business Research Methods. Oxford University Press  Cp. 7 – 15, 26, 27  Power point slides lectures / workshops (BS)  Other material on BS Assignment  Course Assignment 1  Explanation today (4-11-24)  Hand in via Brightspace, ultimately on 25-11-24  Course Assignment 2  Explanation during lecture 3 (20-11-24)  Hand in via Brightspace, ultimately on 20-12-24 Evaluation  SBM Quantitative Research Methods: 4 ECTS  80% Attendance of Lectures / Workshop is required to take part in the Exam (see handbook SBM)  Final grade 50% Exam  10% Course assignment 1  40% Course assignment 2  Compensation for insufficient mark ( hypothesis -> … -> analysis -> conclusion (rejecting or accepting hypothesis)  Testing possible explanations for visitors’ expenditure  … but measurements can sometimes lead to inductive theorising  Analysis -> concepts & theory  Completely new explanations for visitors’ expenditure. Course Assignment 1  Find a scientific article (published in an international peer-reviewed journal) about a quantitative study.  Discuss the study by answering following questions:  Can you recognize the 11 “main steps of quantitative research”? For all the steps that you can recognize: Very briefly discuss how each step has been implemented in the study (in one of two sentences per step).  Discuss in more depth how step 4 of the “main steps of quantitative research” has been carried out: Which concepts does the article discuss and how are they measured (on which indicators)?  (How) have the reliability and validity of these measures/indicators been established?  What statements are being made (if any) and what conclusions can you draw about the causality of the relationship between the concepts, about generalization, and about the replicability of the study? Course Assignment 1  Find a scientific article (published in an international peer-reviewed journal) about a quantitative study.  Discuss the study by answering following questions:  When applicable: Which sampling method has been applied? Why has this method been chosen? What steps have been taken to avoid sampling and non-sampling errors? Which risks on sampling and non-sampling error remain?  What are, overall and based on your own opinion, the strong and weak points of the article and the research it describes?  What could the researchers have been done better in methodological terms, from your perspective? What would you recommend to do (different) in a follow up study. Course Assignment 1  Expected product  ONE Word or PDF-document containing: A title page that included: Name, date, course assignment number, and an APA style reference to the article (and to other sources you might have used) 1 page with the abstract. You can copy that directly from the article. A discussion, answering the questions presented above. This should be maximum 3 pages, but please don’t feel obliged to use more words than necessary.  Upload on Brightspace before or on the 25th of November 2024 Preparation Workshop 14-11  Groups of 4-5 people  Together: Select one scientific article (from an international peer- reviewed journal) discussing a quantitative study  Read the article  Bonus point on assignment 1 to be earned by the group that does the best job of analyzing the article, during the workshop.  All groups will receive feedback Contact information  Dr. Wim Strijbosch  E-mail: [email protected]  Workdays: Monday, Tuesday, Wednesday morning, Thursday, Friday  Dr. Tomas Mainil  Email: [email protected]  Workdays: Monday till Friday Lecture 2: Sampling DR. WIM STRIJBOSCH Agenda 1. Introduction / basic terms 2. Sampling error 3. Types of probability sampling a) Qualities of a probability sample b) Sample size 4. Types of non-probability sampling 5. Limits to generalization  Amstelburg Introduction / Basic terms  Sampling; ‘Selecting individuals for survey research’  Population: the universe of units from which the sample is to be selected  Sample: the segment of population that is selected for investigation  Why selecting individuals: Very often it is not possible (time, money) to have the whole population participate in the research, you need to select:  ‘Visitors to Amstelburg in the city center, near the main attractions, on the 14th of October 2022, …’  Relevant for survey (but not exclusively!) Introduction / Basic terms  How to sample?  Desire to generalize (to the population)  Aim for a representative sample: a sample that reflects the population accurately  Prevent sample bias: distortion in the representativeness of the sample  Two main categories of sampling methods  Probability sample: sample selected using random selection (every member of the population has an equal change of being selected)  Non-probability sample: sample selected not using random selection method Introduction / Basic terms  Errors: The sample is ‘biased’ / some groups under- and others overrepresented  Non-probability sample (does not allow for inferential statistics!)  Probability sample (allows for inferential statistics!)  Sampling error:  Difference between the population from which it is selected, even though a probability sample has been selected (by chance you have not selected a representative sample)  Non-sampling error:  Difference between the population and the sample that arises from  Inadequate sampling  Non-response: Members of sample are unable or refuse to take part  Poor data collection  Poor question wording  Poor interviewing  Poor data processing  Etc.  Amstelburg Sampling error Population: Day visitors Overnight visitors  Amstelburg Sampling error Population: Day visitors Overnight visitors No sampling error  Amstelburg Sampling error Population: Day visitors Overnight visitors Some sampling error  Amstelburg Sampling error Population: Day visitors Overnight visitors A lot of sampling error Types of probability samples  Simple random sample  Systematic sample  Stratified random sample  Multi-stage cluster sample  Amstelburg Simple random sample Amsteldance  Each unit has an equal probability of selection  Probability: n/N; where n = sample size and N = population size  Procedure  List all units and number them consecutively  Use random numbers to select units  Application to visitors to Amstelburg?  Registration of visitors via ‘Amstelburg visitor card’  90% of visitors buy the visitors card and  Each one has bought a numbered ticket and had to give contact data (e-mail address).  Most of visitors that bought the card (90%) have given permission to contact them via e-mail.  Application to event ‘Amsteldance’?  All visitors have bought a numbered ticket and had to give contact data (e-mail address).  In the administration tickets numbers and e-mail addresses are matched.  Most of the visitors (90%) have given the event organization permission to contact them via e-mail.  Amsteldance Systematic sample  Select units directly from sampling frame  From a random starting point, choose every nth unit (e.g. every 4th ‘Event ticket’)  Make sure sampling frame has no inherent ordering – if it has, rearrange it to remove bias  Amsteldance Stratified random sample  Categorise population into ‘strata’:  Visitors to ‘Amsteldance’: 60% Female – 40% Male  One bowl with all tickets bought by female visitors  One bowl with all tickets bought by male visitors  Randomly draw tickets from both bowls until you have reached the desired sample size, but make sure the 60% is selected from the ‘female’ bowl and 40% from the ‘male’ bowl.  Sample = proportionately representative of each stratum  Amsteldance Multi-stage sample A B C D E F G H  Divide population Sample of events  Population: Visitor to music events in the Netherlands A D  Possible multi-stage sampling strategy  Randomly select one of more event (apply stratification?) F H  Randomly select visitors for each selected event A  Amstelburg Sample size  Sample size  The larger the sample, the more precise and representative it is likely to be (as sample size increases, sampling error decreases).  Population size & Sample size Nr. Of visitors to Amstelburg 1000 10.000 100.000 1.000.000 10.000.000 Sample 278 370 383 384 384  “Every cook knows that it only takes a single sip from a well-stirred soup to determine the taste” Sample size  Factors affecting sample size  Time and cost  After a certain point, increasing sample size produces less noticeable gains in precision.  Very large samples are decreasingly cost-efficient.  Non-response  Response rate = percentage of sample who agree to participate and provide usable data.  Heterogeneity of the population  The more varied the population is, the larger the sample will have to be  Kind of analysis to be carried out  Some techniques require large samples (e.g. contingency table; inferential statistics)  Amsteldance Types of non-probability sampling  Convenience sampling  The most easily accessible individuals.  Useful when piloting a research instrument.  In business research and students’ theses these are actually more common that probability samples.  Snowball sampling  Researcher makes initial contact with a small group.  These respondents introduce others in their network.  Quota sampling  Categorize population into strata  Non-random sampling of strata  Visitors to ‘Amsteldance’: 60% Female – 40% Male  Approaching visitors during the event: 60% Female – 40% Male  Interviewers select people to fit their ‘quota’; the sample may be biased towards those who appear friendly and accessible (e.g. in the street), leading to under-representation of less accessible groups.  Often used in market research and opinion polls / Relatively cheap, quick and easy to manage. Limits to generalization  Findings can only be generalized to the population from which the sample was selected: be wary of over-generalizing Workshop (next week)  Read the Case Study ‘Sampling in survey research’ (Brightspace – Week 3)  Answer the following questions:  What kind of sampling procedure was applied in the above-mentioned example?  Why did the researchers stratify the population?  What advantages were afforded by sampling first by region and sub- region?  Why do the authors believe that the data deriving from this sample are representative?  Write down the answers and take these to the workshop.  Lecture 3: Structured interviewing & Self-completed questionnaires SBM 1B 2024-25 DR. TOMAS MAINIL Agenda 1. Structured interviewing a) Types of interviews b) Structured interview c) Example: TIDE d) Interview contexts e) Conducting interviews f) Other approaches to structured interviewing g) Problems 2. Self-completed questionnaires a) Self-completion questionnaires b) Example: DOC c) Advantages and disadvantages d) Designing tips Types of interviews  Interview: “Elicit from the interviewee or respondent […] all manner of information: Interviewees’ own behaviour or that of others, attitudes, norms, beliefs, and values’  Structured interview (standardized interview)  Each interviewee gets the same questions, in (exactly) the same way, in the same order, based on closed questions  Semi-structured interview  Sequence of questions can very, questions somewhat more general than in structured interview, spontaneous questions …  Unstructured interview  Also called intensive interview, list of topics or issues e.g. interview guide. Structured interview  Each interviewee gets the same questions, in (exactly) the same way, in the same order, based on closed questions  Standardized interview schedule  Standardized recording of interviews  Type of questions: Closed questions  Limited choice of possible answers  Eliminates coding errors / Easier to process Structured interview  Advantages:  Variations in people’s replies are due to ‘true’ or ‘real’ variation and not due to interview context  Reduces variation due to error; intra-interviewer variability & inter- interviewer variability  Intra-interviewer variability: Interviewer not consistent in the way he/she asks questions or record answers  Inter-interviewer variability: Interviewers not consistent witch each other in the way they ask questions or record answers  Intra- and inter-interviewer variability can coexist, compounding the error further  TIDE Example: Relationship between tourism and retail (TIDE)  Interviews with 130 employees /  Retail subdivides into 8 subcategories owners of retail shops  Supermarkets  Non-probability sampling method:  Specialised food shops  Three groups of municipalities:  Gas stations  Few tourists: Hansweert (Zeeland),  Shops in consumer electronics Rilland (Zeeland), Wateringen (Zuid- Holland), Breda (Brabant)  Shops in household appliances  Middle category: Ede (Gelderland),  Shops in sport, camping and Barneveld (Gelderland), Reuver recreational articles (Limburg)  Other shops  Many tourists: Domburg (Zeeland), Sluis  Markets (Zeeland), Valkenburg (Limburg), Zandvoort (Noord-Holland), Goes  130 entrepreneurs interviewed, (Zeeland) keeping in mind that all 12 municipalities and subcategories are ‘appropriately covered’ (no stratification).  TIDE Example: Relationship between tourism and retail  Questions: ...  What percentage of your customers, across the whole year, are tourists?  What percentage of your turnover, across the whole year, is caused by tourists?  If there were no more tourists coming to your shop, what percentage of your employees would loose their job?  Do you undertake activities to attract tourist?  If yes, which ones (select from list)  If no, why not? (select reasons from list)  Your company is part of which retail sector? (select from list)  TIDE Interview context  One interviewer, one interviewer (TIDE: Interviews carried out by one BUas student, based on a standardized interview schedule)  More than one interviewee  Group interviews; focus groups (mostly qualitative research method)  More than one interviewer  Unusual situation in business research  No or almost no added value (at least for a structured interview!)  Cost inefficient  TIDE Interview context  Face-to-face interview (TIDE)  Telephone/online interview Conducting structured interviews  Knowing your way around the interview schedule  Training is important to reduce variability in the asking questions (potential source of error)  Introducing the research  Rationale of research (spoken or written)  Not a sales talk but an interview  TIDE Conducting structured interviews  Achieve ‘rapport’  Willingness to participate / Putting people at ease vs. Influencing interviewees / Taking too long  Asking exactly the same questions  Reducing the variability  Recording answers in exactly the same manner  Reducing the variability  Clear instructions  Usage of filter questions (TIDE: questions about activities undertaken to attract tourists)  Question order  Keeping to exactly the same order or questions  Grouping questions (to easily skip topics if needed)  First questions related to the topic  First general then specific questions  Questions such as gender and age (social background) & potentially embarrassing questions in the end Conducting structured interviews  Probing  appears when interviewee does not understand the question well or does not provide sufficient answer  Difficult not to influence the respondent (variation)  Prompting  Interviewer suggest a possible answer to a question  All respondents must receive same prompt Other approaches to structural interviewing  Projective methods, pictorial and photo- elicitation: not so common in business research  Critical incident method: ask respondents to describe critical incidents in their lives  Verbal protocol approach: respondents think aloud while performing a task…  TIDE Problems  Characteristics of interviewers (gender, age, ethnicity, class) can evoke socially desirable responses  Response sets  People may respond in consistent but irrelevant ways  By acquiescence: e.g. agreeing or disagreeing to all questions (TIDE: 50%, 40%, 30%)  For reasons of social desirability: interviewees reflect on the way their answers might be perceived  The problem of meaning  Interviewer and respondent may not attribute the same meanings to concepts. The meaning of questions is not pre-given but rather constructed in the interview  ‘What do you consider as a tourist?’ / ‘How do you recognize a tourist?’ Agenda 1. Structured interviewing a) Types of interviews b) Structured interview c) Example: TIDE d) Interview contexts e) Conducting interviews f) Other approaches to structural interviewing g) Problems 2. Self-completed questionnaires a) Self-completion questionnaires b) Example: DOC c) Advantages and disadvantages d) Designing tips Self-completed questionnaires  Also called a self-administered questionnaire  No interviewer present (in principle)  Respondent writes answers on form  Returned to researcher  Can be distributed in person or by (e)mail Self-completed questionnaires  In comparison to interviews, self-completion questionnaires tend to:  have fewer open questions, since closed ones tend to be easier to answer;  have easy-to-follow designs to minimize the risk that the respondent will fail to follow filter questions or will inadvertently omit a question;  be shorter, to reduce the risk of ‘respondent fatigue’, since it is much easier for a respondent who becomes tired of answering questions in a long questionnaire to consign it to a waste paper bin than it is for a subject being interviewed to terminate the interview.  DOC Example: Designer Outlet Center  Construction of a new Designer Outlet Center  Expected impact of the DOC, by the inhabitants of Remscheid  Selection of respondents  Approached on the street  Online via Facebook, discussion groups, etc.  Via local radio station  Self-completed questionnaire on the street (I-pad with questionnaire) + Online self-completed questionnaire  DOC Example: Designer Outlet Center  How well do you feel informed about the process of the DOC Project?  Scale from 1 to 5 (‘very bad’ to ‘very good’)  How were you informed about the DOC Project?  Choose from list, multiple answers possible  Do you currently feel impacted by the DOC project? (If yes, how?)  Do you think that the DOC Project will impact you in the future? (If yes, how?)  Do you have a personal of professional relation to the DOC project? (yes / no)  In which professional field are you working in? (tourism, retail, other)  Which impacts do you expect through the development of the DOC on the city of Remscheid?  List of potential impacts, and then respondent had to choose between ‘no impact’, ‘small impact’, ‘moderate impact’, ‘big impact’ and ‘very big impact’  Economic Growth, job creation, increasing popularity of Remscheid, improved infrastructural development, etc.  Increasing traffic density, increasing pollution, increasing noise, job loss, etc.  Add additional impacts  DOC Advantages and disadvantages  Advantages:  Cheaper and quicker to administer (to widely dispersed populations) than face-to-face interviews (DOC: Online)  Absence of interviewer effect  No interviewer variability  Convenient for respondents  Disadvantages  Interviewees cannot be probed or prompted, and they cannot ask questions (DOC: Street-interviews …?)  Researchers can only ask salient questions  Few open-ended or complex questions (DOC: Maybe too many open questions …?)  Respondent can see the whole questionnaire before  Cannot ensure that the ‘right’ person answers  Cannot collect additional data  Respondent fatigue if there are too many questions  Excludes people with limited literacy skills  Greater risk of missing data  Lower response rates Reflections about the content of WS last week (prep Assignment 1)  Regarding the PPT - Be aware that a PPT is not the same as a CANVA presentation : can have effect on ease of presentation - Be aware that reading out loud what is on your laptop doesn’t add to the presentation skills  Regarding the analysis - Diversity between different groups in providing in-depth knowledge, understanding and analysis - Make sure that you align what you what bring as content with the limited time frame you have to present as a group. Information for this week:  You will receive written group feedback on the prep assignment 1 on Tuesday  You will have a next WS on Sampling hosted by Wim Strijbosch  Next week: Lecture 4: you will receive the guidelines for the 2th assignment on survey design  Finally: any questions about assignment 1(deadline 25.11.2024) Lecture 4: Structured observation Content analysis Course Assignment 2 Tailored Design Method Agenda 1. Structured observation a) Problems with surveys b) Types of observations c) Structured observation d) Schedule e) Sampling f) Reliability and validity g) Reactive effects h) Other forms of structured observation i) Advantages and disadvantages 2. Content analysis a) Content analysis b) Research questions c) Sampling d) Coding e) Advantages and disadvantages 3. Course assignment 2 4. Tailored Design Method (if there is time) Problems with surveys  Respondents interpret different meanings from questions  Omission of key terms when reading questions  Reliance on people’s memories of their behaviour  Social desirability effect  Threatening questions - invalid answers  Interviewer bias  Gap between stated and actual behavior  Alternative: Observation Types of observations  Structured/systematic observation vs. Unstructured observation  Participant observation vs Non-participant observation  Participant: Qualitative, mostly unstructured  Non-participant: Structured or unstructured  Simple observation vs contrived observation  Simple: A form of observation where the situation is left as it is (the observer has no influence over the situation being observed)  Contrived: A form of participant observation where the situation is changed (the observer actively alters the situation and observes the effects) Structured observation  Method of systematically observing people’s behaviour  Observation schedule or coding scheme  Behaviour is systematically recorded  Generates information on different aspects of behaviour that can be treated as variables  Kinderdijk Kinderdijk Tours tours  ‘Kinderdijk’ tours  Quality of their tour guides  Goal of ‘Kinderdijk’ tours: Make sure that visitors have a memorable experience, with a tour guide that facilitates interaction – instead of just telling facts and stories  Structured observation of the interaction between a tour guide and his tour group (to see if the goal is reached)  Observation / At random times  Tour guide: Telling stories, listening, asking questions, answering questions, no interaction  Group members: % Interacting with tour guide (listening, talking, etc.), % other activities, % outside of hearing distance  Kinderdijk Schedule tours  Must have a clear focus and be easy to use  Must be easy to operate  Specifies categories of behaviour to be observed and how to allocate behaviour to a category  Categories must be inclusive (cover all options) and mutually exclusive  Clear guidelines needed so that observers can distinguish between behaviour categories  A pilot study is useful to iron out any problems Time Telling stories Listening Asking Answering No interaction % Interacting %Other % Outside questions questions actives hearing distance 12:00 X 40% 20% 40% 12:05 x 40% 60% 12:10 x 80% 20% 12:15 x 90% 10% Sampling  Probability sample  Sampling people  random sample of individuals to observe  Sampling time periods  observe same individual(s) at different, randomly selected times  Non-probability samples  cannot use a ‘sampling people’ probability sample if there is no sampling frame (e.g. people walking along a street) Sampling  Ad libitum sampling  Record whatever is happening at the time  Focal sampling (most common form)  Observe a specific individual for a set period of time  Scan sampling  Record the behaviour of everyone in the group at regular time intervals  Behaviour sampling  Observe a whole group (over a longer period of time) to see who was involved in a particular behaviour  Kinderdijk Reliability and validity tours  Reliability  inter-observer consistency  Degree of agreement over the coding of items by two people  intra-observer consistency  Degree of consistency of the application of the observation schedule over time  Difficult to achieve because of the effects of e.g. observer fatigue, lapses in attention  Validity  does the schedule measure underlying concept? (Kinderdijk: Memorable experience? interaction?)  implementation of schedule  When the measure is unreliable, it cannot be valid  presence of observer - reactive effect  Change in the behaviour as a result of being observed  Kinderdijk Reactive effect tours  Research subject knows s/he is participating in research, thus invalidating the data.  The guinea pig effect (awareness of being tested)  Role selection (participants adopt a particular kind of role)  Measurement as a change agent (presence of an observer – things are different)  Response sets (mostly relates to survey research when the respondent replies to a set of questions in a consistent but clearly inappropriate manner).  Reactive effects are likely to occur in any research in which participants know they are the focus of investigation. Advantages & disadvantages  Advantages  It is more accurate than interviews and questionnaires  The researcher can see what people really do, not what they say they do  It is a useful accompaniment to other methods  i.e. study of behaviour, attitudes, and social context  Disadvantages  Neglects the meanings and intentions behind behaviour (motives are inferred but not investigated)  Generates fragmented data - difficult to see the wider picture Agenda 1. Structured observation a) Problems with surveys b) Types of observations c) Structured observation d) Schedule e) Sampling f) Reliability and validity g) Reactive effects h) Other forms of structured observation i) Advantages and disadvantages 2. Content analysis a) Content analysis b) Research questions c) Sampling d) Coding e) Advantages and disadvantages 3. Course assignment 2 4. Tailored Design Method (if there is time) Content analysis  Approach to analysis of documents and texts  Quantifies content in terms of predetermined categories  Systematic and reliable  A quantitative research strategy (NOT a literature review!)  Objective, systematic identification of specified characteristics of messages (Holsti, 1969) Content analysis Content analysis Research questions  Must be clearly specified before analysis  Decide which dimensions of texts to quantify:  Source: Published in which media?  Image: Inclusion of images? What kind of image?  What: What does the article say? What is their opinion / disposition? (categorize)  How much: How many times does it mention a certain word? How many pages devoted to the topic? Title Source Image Positive / #BUas #Breda Negative University tone NHTV changes its name to BUas Nu.nl Building Positive 2 1 NHTV continues as Breda Univeristy AD Logo Negative 0 2 Dutch Design Week  Eindhoven  9 days, October  Design: Food design, graphical design, industrial design, product design, spatial design, service design, fashion design.  110 locations throughout the city  Expositions  Visiting ateliers  Workshops  Seminars / Conferences / Debates  Music & Parties  Measuring the coverage of the DDW in the media Sampling  Two dimensions are possible for delimitating the ‘population’:  The media  Dates of publication Sampling-media  Progressive narrowing down  Which types of text?  Printed or visual data? Documents? Mass media?  If mass media, which kind?  TV, radio, newspapers, magazines?  More than one type?  For each type of text, which examples?  e.g. tabloid or broadsheet newspapers? Sampling-dates  Starting dates may be predetermined by an historical event (like the Black Monday, 9/11)  More open if the study is an ongoing, general phenomenon  End dates can be a matter of judgement Dutch Design Week  Types of media  National and international broadcasts  National and international newspapers  One month before the events until one month after the event  Probability sampling? Coding  What is to be counted?  Words: Frequency of words or phrases (e.g. ‘DDW’ or ‘Dutch Design Week’) Subjects and themes  Dispositions: Opinion, values, bias, ideology, etc.  Images: particularly relevant to tourism and marketing studies Coding No. Source … Date … Length (nr. of words) … Page number … Type of design Food, graphical, industrial, product discussed spatial, service, fashion. Type of activity Expositions, Workshops, Seminars / discussed Conferences / Debates, Music & Parties Evaluation of DDW Positive / negative / neutral … Coding manual No.  Coding manual Source …  A set of instructions for coders Date …  Lists all possible categories for each Length (nr. of … dimension words)  Shows which codes/numbers refer to Page number … which category Type of design Food, graphical, industrial, discussed product spatial, service,  Gives guidance on how to decide on a fashion. code Type of activity Expositions, Workshops,  Explains what to do if more than one discussed Seminars / Conferences / code applies Debates, Music & Parties Evaluation of Positive, negative, neutral DDW … Coding  Avoiding potential pitfalls  Ensure that coding scheme has:  separate dimensions  mutually exclusive categories  exhaustive categories  clear instructions to coders  a clearly specified unit of analysis  Pilot the study to make sure of consistency between:  coders (inter-coder reliability) and  consistency over time for each coder (intra-coder reliability) Advantages and disadvantages  Advantages  Transparency  Ease of longitudinal analysis  Unobtrusiveness  Flexibility  Ease of access  Disavantages  Questions of authenticity, credibility and representativeness of documents  Interpretation by coders  Inability to answer ‘why?’ questions  Has an a-theoretical approach Agenda 1. Structured observation a) Problems with surveys b) Types of observations c) Structured observation d) Schedule e) Sampling f) Reliability and validity g) Reactive effects h) Other forms of structured observation i) Advantages and disadvantages 2. Content analysis a) Content analysis b) Research questions c) Sampling d) Coding e) Advantages and disadvantages 3. Course assignment 2 4. Tailored Design Method (if there is time) Agenda 1. Structured observation a) Problems with surveys b) Types of observations c) Structured observation d) Schedule e) Sampling f) Reliability and validity g) Reactive effects h) Other forms of structured observation i) Advantages and disadvantages 2. Content analysis a) Content analysis b) Research questions c) Sampling d) Coding e) Advantages and disadvantages 3. Course assignment 2 4. Tailored Design Method (if there is time) Assignment II 1. Pick a topic that you find interesting. 2. Pick five demographic characteristics that you find interesting and include them in your research. 3. Design your draft questionnaire. 4. Pilot your questionnaire 5. Revise accordingly and design the final questionnaire. Assignment II Ad 2) Pick a topic that you find interesting  For example:  Motivations for engaging in party tourism  Impact of viewers’ characteristics on the effectiveness of a TV commercial  Impact of students’ characteristics on study success at the BUas Assignment II Ad 2) Pick five demographic characteristics that you find interesting and include them in your research.  Formulate 5 research questions with these demographic characteristics  For example:  Party tourism motivations are influenced by age  Party tourism motivations are influenced by gender  Party tourism motivations are influences by … (insert other demographic characteristic)  Etc. Assignment II Ad 3) Design your draft questionnaire, based on your research questions.  look for existing questions and scales in the literature and besides that develop some questions yourself to cover your whole area of interest Assignment II Ad 4 & 5) Pilot your questionnaire (min. 5 respondents) / Revise accordingly and design the final questionnaire  Provide a description of your pilot and the changes that you made based on the pilot. Describe both how the pilot took place (how it was set up, the instructions that the participants got, etc.), what you learned from that, and how this affected your survey. Assignment II – Expected product  ONE document, with the following paragraphs/components:  Introduction (max ½ page)  Explain your reasons for investigating this topic  Methodology section (max. 3 pages), that explains:  The variables and measures / indicators used  The five research questions  The purpose and formulation of each question in the questionnaire  Draft questionnaire (No max. length)  Set-up of the pilot study and results of your pilot study (max. 2 pages)  Final questionnaire (No max. length)  Literature list (No max. length)  Mind the language, structure, formatting, and referencing. Assignment II – Expected product  Quality criteria:  Clear and convincing explanation of the motivation for choosing this topic, in the introduction.  Clear and convincing explanations for the variables and measures/indicators, the research questions, and the purpose and formation of the questions in the questionnaire, in the methodology section.  Usage of sufficient and appropriate literature, throughout the document  Following the guidelines for survey design, as discussed in this course (book / lectures / workshops)  Detail and clarity of the explanations of the pilot study, which should be sufficient to enable an academic reader to replicate it.  Making sensible use of the pilot findings to revise the questionnaire, providing clear argumentations for the changes made.  Correct APA style references.  English understandable.  Appropriate appearance, structure, and length. Assignment II - Deadline  Hand in via Brightspace, ultimately on 20-12-24 Agenda 1. Structured observation a) Problems with surveys b) Types of observations c) Structured observation d) Schedule e) Sampling f) Reliability and validity g) Reactive effects h) Other forms of structured observation i) Advantages and disadvantages 2. Content analysis a) Content analysis b) Research questions c) Sampling d) Coding e) Advantages and disadvantages 3. Course assignment 2 4. Tailored Design Method (if there is time) The tailored design method  Dr. Dillman is professor at the Washington State University who has invested a career in fine tuning techniques to improve mail and phone surveys The tailored design method  “Tailored design involves using multiple motivational features in compatible and mutually supportive ways to encourage high quantity and quality of response to the surveyor’s request. It is developed from a social exchange perspective on human behavior, which suggests that respondent behavior is motivated by the return that behavior is expected to bring, and in fact, usually does bring, from others. It assumes that the likelihood of responding to a self- administered questionnaire, and doing so accurately, is greater when the respondent trusts that the expected rewards will outweigh the anticipated costs of responding.”  (Dillman, et al, 2009: 16) The tailored design method  Social Exchange Theory  The actions of individuals are motivated by the return these actions are expected to bring from others  People attempt to keep costs below the rewards they expect to receive  Tailored Design Method  Minimize overall survey error – coverage, sampling, measurement and non-response  Customize survey procedures for each particular survey situation  Topic?  Respondents?  Sponsor?  Budget?  Time? The tailored design method - goals  Establish trust  Increase participation benefits  Decrease participation costs The tailored design method - goals  Establish trust  Financial incentives  Money is a symbol of trust  Link the study to a reliable company/ organisation  Credibility  Build upon existing relationships  E.g. … ‘for 5 years you have been a member of our…’  Be professional  Use high-quality material, think of the design The tailored design method - goals  Increase participation benefits  Treat respondents in a positive manner  Emphasize the small sample size, and the importance of their answers  Personalize as much as possible  Express appreciation  Use people’s group feeling  Financial incentives or different ‘profits’  Make the survey interesting The tailored design method - goals  Decrease participation costs  Develop a clear and consequent survey to minimize time costs and effort; layout, presentation, instructions  Anonymity, confidentiality: Be careful with personal information The tailored design method – Layout, presentation, instructions  Orderly layout  The questionnaire should be neither too short and cramped nor too long and bulky.  Use spaces and formatting to cluster and group questions  Provide a consistent structure  Clear presentation  A variety of font sizes, bold print, italics, shadows, and capital letters can be used to attract attention, but be consistent.  Clear instructions  How to indicate choice of answer: i.e. a tick, a circle, an underline, etc.  If more than one answer can be given, make it clear that you want the respondent to select a number of possibilities from a list.  Keep questions and answers together.  Never spread a question on two pages.  Provide simplicity, consistency, regularity and symmetry The tailored design method - Improving Response Rates  Good response rates not just a function of survey instrument and design  Implementation procedures have a great influence on response rates  Experimental research shows that multiple contacts with respondents is the primary factor in improving response rates The tailored design method - Improving Response Rates  Mail survey  Announcement letter  Survey with cover letter  Thank you card some days or weeks after mailing of survey  Al three express appreciation for people that have filled or will fill out the survey  Mail to ‘non-respondents’ with replacement survey (2 to 4 weeks after original survey)  Mail ‘ultimate’ letter (2 to 4 weeks after last letter) The tailored design method - Improving Response Rates  Announcement mail  Sent a few days prior to the survey  Not too far in advance or recipients will forget / not connect it to survey itself  Should specifically note:  Will receive an important survey in a couple of days  Participation / response is greatly appreciated  Better if sent by a person of authority using recognizable letterhead and/or logo  If so, note in letter that the survey will be from organization on behalf of the person The tailored design method - Improving Response Rates  Cover letter  Introduction should indicate:  Who is conducting the survey and why  (Short) explanation of why survey is important  Topics to be covered in the survey  Administrative matters  State whether responses are confidential or not  Perhaps indicate how long the survey should take  Keep it to one page in length if at all possible  Again, if possible, put on letterhead and have signed by someone considered an authority by respondents  Dillman calls them “trust-inducing elements” The tailored design method - Improving Response Rates  Thank you card  Mostly respondents don’t react right away (on the first letter)  Thank You Card is for those respondents that wanted to answer but haven’t done yet for several reasons  It is NOT to convince respondents who had decided not to answer  Is sent to all respondents  Again stresses the importance of cooperation in the investigation  Includes contact information to ask for a new questionnaire The tailored design method - Improving Response Rates  Letter to non respondents  Send when amount of incoming questionnaires begins to decline  Letter may have a compelling tone  Clearly and specifically indicate in the first paragraph that the questionnaire has not yet arrived (replacement) The tailored design method - Improving Response Rates  Final letter  Continued focus on 'non-responders' with increased force  Increased coercion, not by the tone of the letter, but by repeated attempts  Since previous attempts failed: stimulus change by registered mail  Tone of the letter about the same as in previous letter  Focus on the importance of the questionnaire and the important contribution of the respondent The tailored design method - Improving Response Rates  Clearly, these are all for (traditional?) mail surveys  Basic principles still apply to other modes  Can be more challenging to implement in other modes The tailored design method - Improving Response Rates  Other techniques have a more modest effect  Professional appearance of correspondence and survey instrument  Personalization of correspondence  Introduction letter from person of authority  Should not focus on any one technique  Implement as many as relevant / appropriate to maximize response rates