Research Design Chapter IV PDF

Research Design Chapter IV 1  Design Science in CS/IT/IS Fundamentals of Research Research Problem identification and motivation The Research Process Objectives of a...

Research Design Chapter IV 1  Design Science in CS/IT/IS Fundamentals of Research Research Problem identification and motivation The Research Process Objectives of a solution Literature Review Design and development Design Science in Demonstration CS/IT/IS Research Evaluation and Communication Ethics Citation and Referencing Style 2  Introduction A research design is the arrangement of conditions for the collection and analysis of data in a manner that aims to address the research problem. The research design should be inline with: – What is the study about? (Problem definition) – Why is the study being made? (Justification) – Where will the study be carried out? (Location) – What type of data is required? (Quanti, Qual, Pri, Sec)  Introduction Where can the required data be found (target population) What will be the sample design (technique chosen) What techniques of data collection will be used? (observation, interview, questionnaire, or document analysis) How will the data be analyzed (Data Analysis techniques & tools to be employed)  We may split the overall research design into three: – The sampling design - which deals with the method of selecting items to be observed. – The statistical design - which concerns with the question of how many items are to be observed and how the information and data gathered are to be analyzed. – The operational design - which deals with the techniques by which the procedures specified in the sampling, Statistical and observational designs can be carried out. Important concepts in research design Variable: a concept which can take on different values – Continuous variable: A quantitative variable for which all values with in some range are possible. a variable which can assume any numerical value within a specific range. (eg. Age, weight, depth etc) These variables are often grouped in intervals. – Discrete variable: A quantitative variable which does not take on all values in a continuum, often the variables can assume integer values only/ the individual values fall on the scale only with distinct gaps. (eg. # of children) Important concepts … – Dependent variable: – Independent variable: – Extraneous variables: affect the Dependent variable. – Control variable: a variable used to minimize the effects of extraneous variables. Experimental and Control Groups (in experimental hypothesis testing research) – Control group: a group exposed to usual conditions – Experimental group: a group exposed to some special conditions Treatments: the different conditions under which experimental and control groups are put. Important concepts Example: Investigating the of perception employees on the usage of the new ATM services to take their salary. Control Group Exp. Group A B Usual Conditions Special Conditions (Usual study prog.) (Special study prog.) 8  Sampling Design Population: the entire group under study as defined by research objectives. – Sometimes called the “universe.” or – Reference population Researchers define populations in specific terms such as heads of households, individual person types, families, types of retail outlets, etc. Population, geographic location and time of study are also considered Sampling Design 10  Sample: a subset of the population that should represent the entire group. Sample unit: the basic level of investigation…consumers, store managers, shelf-facings, adolescence, etc. – The research objective should define the sample unit. Census: is the counting of the complete population.  Calculating sample frame error (SFE): – Subtract the number of items on the sampling list from the total number of items in the population. – Take this number and divide it by the total population. Multiply this decimal by 100 to convert to percent (SFE must be expressed in %) – If the SFE was 40%, this would mean that 40% of the population was not in the sampling frame. Practical considerations when taking samples: – cost and population size – Nature & purpose of the study – Inability of researcher to analyze large quantities of data potentially generated by a census Samples can produce sound results if proper rules are followed for the draw  Sampling Methods Two Types of Sampling Methods: – Probability sampling: members of the population have a known chance of being selected – Non-probability sampling: the chances of selecting members from the population are unknown.  Simple Random Sampling The probability of being selected is “known and equal” for all members of the population – Blind Draw/Lottery Method (e.g. names “placed in a hat” and then drawn randomly) – Random Numbers Method (all items in the sampling frame given numbers, numbers then drawn using table or computer program) – Advantages: Known and equal chance of selection Easy method when there is an electronic database – Disadvantages: Complete accounting of population needed Very inefficient when applied to skewed population distribution  Systematic Sampling It is the way to select a probability-based sample from a directory or list. This method is more efficient than simple random sampling. This is a type of cluster sampling method. – Sampling interval (SI) = population list size (N) divided by a pre- determined sample size (n)  How to draw? 1) calculate SI, 2) select a number between 1 and SI randomly, 3) go to this number as the starting point and the item on the list here is the first in the sample, 4) add SI to the position number of this item and the new position will be the second sampled item, 5) continue this process until desired sample size is reached.  Advantages: – Known and equal chance of any of the SI “clusters” being selected – Efficiency. do not need to designate (assign a number to) every population member, just those early on the list (unless there is a very large sampling frame). – Less expensive…faster than SRS Disadvantages: – Small loss in sampling precision – Potential “periodicity” problems  Cluster Sampling Method by which the population is divided into groups (clusters), Any of which can be considered a representative sample. These clusters are mini-populations and therefore are heterogeneous. Once clusters are established, a random draw is done to select one (or more) clusters to represent the population.  Advantages – Economic efficiency … faster and less expensive than SRS – Does not require a list of all members of the universe Disadvantage: – Cluster specification error…the more homogeneous the cluster chosen, the more imprecise the sample results.  Convenience Sampling Samples drawn at the convenience of the interviewer. People tend to make the selection at familiar locations Taking of samples that are readily available – E.g. those that arrive on a scene by coincidence Appropriate for some less demanding research Error occurs 1) in the form of members of the population who are infrequent or nonusers of that location and 2) who are not typical in the population  Judgment Sampling Samples that require a judgment or an “educated guess” on the part of the interviewer as to who should represent the population. Also, “judges” (informed individuals) may be asked to suggest who should be in the sample. – Subjectivity enters in here, and certain members of the population will have a smaller or no chance of selection compared to others  Steps in Sampling Process Defining the population Specifying the sampling unit Specifying the sampling frame ( the means of representing the elements of the population) Specifying the sampling method Determining the sampling size Specifying the sampling plan Selecting the sample 22  Measurement & Measurement Scales Measurement is the process through which researchers describe, explain, and predict the phenomena and constructs of our daily experiences The concept of measurement is important in a research in two key areas: – Enables researchers to quantify abstract constructs & Variables. – used to analyze sophisticated statistical data  Non metric Data vs. Metric Data Non-metric data (also referred to as qualitative data)-which cannot be quantified and are predominantly used to describe and categorize. Metric data (also referred to as quantitative data)-are used to examine amounts and magnitudes  Instrument Design Based on hypotheses identified in the stages of research process Most important question researcher can ask before begin writing Survey-type instruments can yield three types of information. A. Reports of Fact - self-disclosure of some objective information (e.g., age, gender, education, behavior) B. Ratings of Opinion or Preference -evaluative response to statement (e.g., satisfaction, agreement, like\dislike) C. Reports of Intended Behavior - self-disclosure of motivation or intention (e.g., likeliness, willingness) 25 How will administration be accomplished? A. Self-administered surveys - subject responds to printed questions (e.g., group or mail surveys) Advantages – Ask questions with long, complex or visual response categories – Ask sequences of similar questions – Respondent does not share answers with immediate person Disadvantages – Careful questionnaire design is required – Open response questions not useful – Good reading and writing skills by respondents are needed – Very little quality control over administration 26 B. Other-administered surveys – Subject responds to questions directly posed by researcher (e.g., interview, phone survey) Advantages – Most effective in joining cooperation (initial and length) – Opportunity to answer respondent questions and ensure quality of data (e.g., probe adequate answers, answer all questions) – Rapport and confidence building possible Disadvantages – Cost and time requirements – Adequate training of staff 27 – Accessibility of sample A. Open-ended Questions - permits subject freedom to answer question in own words (without pre-specified alternatives Advantages – Obtain unanticipated answers – May better reflect respondents thoughts\beliefs – Appropriate when list of possible answers is excessive Disadvantages – Flexibility in responses difficult to code and analyze – Provide incomplete or unintelligible answers B. Close-ended Questions - subject selects from list of pre- 28 determined, acceptable responses Types of closed-ended questions 1.Checklists - respondent selects certain number of pre- specified categories (nominal data) 2. Two-way (Forced Choice) -Respondent must select between two alternatives (crude ordinal\nominal) Yes or No type 3. Ranked - respondent must place items in order of importance or value (ordinal) 4. Multiple-Choice (Likert Scale) – respondent selects between range of alternatives along pre-specified continuum (ordinal\interval?) Strongly Agree Agree Neutral Disagree Strongly Disagree 29 1 2 3 4 Writing good survey questions Good question wording: – Simple sentences – No double negatives – Eliminate vagueness (poorly defined terms) – Eliminate Objectionable\Irrelevant questions 30  Pilot Testing – Pre empirical investigation to observe the reliability and validity of the instrument & – To modify the items of the instrument accordingly. Reliability of the Instrument – Deals with the extent to which the instrument yields the same results on repeated trials. – Reliability refers to the consistency, stability or equivalence of a number of measurements taken using the same measurement method on the same subject. – If repeated measurements are highly consistent (even identical), then there is a high degree of reliability with the measurement 31 method.  Validity of the Instrument – Validity of an instrument refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure through insuring the data collection instrument’s ability to collect the intended data fully and appropriately. In short, validity is concerned with the study's success at measuring what the researchers set out to measure and indicates the extent to which the data collected reflects the phenomena under investigation. 32  Written report format Traditional written reports tend to be produced in the following format. 1. Title of the page – This contains the title of the report, – The name of the researcher – The date of publication. – If the report is dissertation or thesis, the title page will include details about the purpose of the report for example ‘ a thesis submitted in partial fulfillment of the requirements of for the degree of Masters of Science in Information Systems’. – If the research has been funded by a particular organization, details of this may be included on the title page. → sponsors 33 2. Contents page - list the contents of the report either in chapter or section headings with sub headings if relevant and page numbers 3. List of illustrations – this section includes, title & page number of all graphs, tables, illustrations & charts etc. 4. Acknowledgements – Some researchers may wish to acknowledge the help of their research participants, tutors, employers and/or funding bodies – Political/Religious Affiliations/ acknowledgements 34 – No need to criticize others who made problems 5. Abstract/Summary –This tends to be a one page summary of the research, its purpose, scope, methods, main findings / Results and discussions/conclusions. 6. Introduction- This section introduces research ,settings out the aims and objectives, terms and definitions. It includes the rationale for the research and summary of the report structure. 7. Background - in this section is included all your background research, which may be obtained from literature, personal experience or both. 35  Citation → acknowledge – You must indicate from where all the information to which you refer has come, so remember to keep a complete record of every thing you read. – If you do not do this, you could be accused of plagiarism which is a form of intellectual theft. – When you are referring to a particular book or journal article, find out the accepted standard for referencing from your institution. 36 8. Methodology and Methods- this section is set out a description of ,and justification for, the chosen methodology and research methods. The length and depth of this section will depend up on whether you are a student or employee. If you are undergraduate student ,you will need to raise some of the methodological and theoretical issues pertinent to your work. If you are postgraduate student, you will need also to be aware of epistemological and ontological issues involved. 37 9. Findings/Analysis- this section should include your main findings. The content of this section will depend on your chosen methodology and methods. If you have a large quantitative survey,This section may contain tables, graphs, pie charts and associated statistics. If you have conducted a qualitative pieces of research ,this section may be descriptive prose containing lengthy questions. 38 10. Conclusion- In this section you sum up your findings and draw conclusions from them, perhaps in relation to other research or literature. 11. Recommendations –Some academic reports will not need this section. If you are an employee researcher ,this section could be the most important part of the port. In this section is set out a list of clear recommendations which have been developed from your research. 39 12. Further research – It is useful in both academic reports and work-related reports to include a section which shows how a research can be continued. Perhaps some results are inconclusive ,or perhaps the research has thrown up many more research questions which need to be addressed It is useful to include this section because it shows that you are aware of the wider picture and that your are not trying to cover up some thing which you feel may be lacking from your own work. 40 13. Reference –Small research projects will need only a reference section. – Harvard system or Chicago system This includes all the literature to which you have referred in your project. The popular method is the Harvard system which lists the authors surnames alphabetically ,followed by their initials, date of publication, Title of book in italics ,place of publication and publisher. – If the reference is journal article the title of the article appears in inverted commas and the name of the journal appears in italics, followed by the volume number and pages of the article. 41 14. Bibliography - If you have read other work in relation to your research but not actually referred to them when writing up your report ,you might need to include in the bibliography, 42  Journal Articles If you want your research findings to reach a wider audience, it might worth considering producing an article for journal. Most academic journals do not pay for articles they publish, but many professional of trade publications do pay for contribution, if published. However the competition can be fierce and your article will have to stand out from the crowd if you want to be successful. 43  Oral presentations Another method of presenting your research findings is through an oral presentation. This may be at the university or college to other students or tutors, at a conference to other researchers or work colleagues, or in work place to colleagues employers or funding bodies. Helps a wider audience to find out about a research. 44  Considerations in Slide preparation – Your slides should be clear, visible and legible to the audience Layout: consider background and text colors – blue background with white/bright yellow text Background: consistency – use same background color and style for all slides Font: size and style (capital, small letters) 45  Delivering the oral presentation – Have hard copies as a reserve in case no electric power/LCD – Begin the presentation on time – Familiarize your self to the room locations, switch, microphones – Arrive early to the presentation hall so that you can discus with the organizers – Insure that Your sound is hearable comfortable for the audience – Do not be nervous – Make eye contact with your audience – Draw the attention of your audience to important points – Make sure that you do not block the view of the audience – Mention owners of works in your literatures – Finish on time – Start the question/ answer session and welcome the audience 46  References Christian W. Dawson: Projects in Computing and Information Systems (A Student's Guide). Addison Wesley, 2005 Oates, B.J.: Researching Information Systems and Computing, Sage Publications, 2005 Zobel, J. (2004): Writing for Computer Science - The art of effective communication. 2nd ed., Springer, 2004 John W.Creswell: Research Design: Qualitative, Quantitative, and Mixed Methods (Edition: 2nd), SAGE Publications, 2002 Kothari C.R.: Research methodology (Methods and Techniques), 2nd ed., New Age International Publisher, 2004. 47