Quantitative Research Methods PDF

Quantitative Research Method An experiment is a scientific investigation in which the researcher manipulates one or more independent variables, controls any other relevant variables, and observes the effect of the manipulations on the dependent variable(s). The goal of experimental rese...

Quantitative Research Method An experiment is a scientific investigation in which the researcher manipulates one or more independent variables, controls any other relevant variables, and observes the effect of the manipulations on the dependent variable(s). The goal of experimental research is to determine whether a causal relationship exists between two or more variables. Independent and Dependent Variables 1. Independent Variable - manipulated (changed) by the experimenter 2. Dependent Variable - variable on which the effects of the changes are observed - observed but not manipulated by the experimenter Example: To examine the effect of different teaching methods on achievement in reading, an investigator would manipulate method (the independent variable) by using different teaching methods in order to assess their effect on reading achievement (the dependent variable). Exercises: Which of the following questions would be appropriate for experimental research? 1. Do high school students who participate in an educational program designed to reduce smoking report smoking fewer cigarettes after 1 year? 2. What do elementary teachers think about retaining low achievers? 3. Are first-born children higher achievers than their younger siblings? 4. Do new teachers who are assigned a mentor report greater satisfaction with teaching than do new teachers not given a mentor? CHARACTERISTICS OF EXPERIMENTAL RESEARCH Essential Requirements for Experimental Research 1. Control 2. Manipulation of the independent variable 3. Observation and measurement 1. Control eliminating all other possible explanations by controlling the influence of irrelevant variables purpose: to arrange a situation in which the effect of a manipulated variable on a dependent variable can be investigated Assume that you wish to test the hypothesis that children taught by the inductive method (group A) show greater gains in learning scientific concepts than children taught by the deductive method (group B). To draw a conclusion concerning the relationship between teaching method (independent variable) and the learning of scientific concepts (dependent variable), you must rule out the possibility that the outcome is due to some extraneous, usually unmeasured variable(s). An extraneous variable is a variable that is not related to the purpose of the study but may affect the dependent variable. In this experiment, aptitude is a factor that certainly affects the learning of scientific concepts; therefore, it would be considered a relevant extraneous variable that you must control. The best way to control for aptitude is to randomly assign subjects to the two groups. 2. Manipulation of the independent variable involves setting up different treatment conditions Treatment is another word for the experimental manipulation of the independent variable. The different treatment conditions administered to the subjects in the experiment are the levels of the independent variable. The levels represent two or more values of an independent variable and may involve differences in degree or differences in kind, depending on the nature of the manipulation. If a researcher is interested in the effects of a stimulant on college students’ learning of nonsense syllables, the researcher may begin by specifying the stimulant to be used and the amount to be administered. If the researcher is interested in the effect of the stimulant amount on learning, he or she would perhaps set up three levels of the independent variable: high, medium, and low dosage. In this case, the levels differ in degree. The researcher could also compare the effects of one stimulant with another stimulant, or with nothing at all. In this case, the levels differ in kind. 3. Observation and measurement After applying the experimental treatment, the researcher observes to determine if the hypothesized change has occurred. Some changes can be measured directly, whereas other changes are measured indirectly. EXPERIMENTAL COMPARISON An experiment begins with an experimental hypothesis, a prediction that the treatment will have a certain effect. The research hypothesis expresses expectations as to results from the changes introduced—that the treatment and no- treatment groups will differ because of the treatment’s effects. For the simplest experiment, you need two groups of subjects: the experimental group and the control group. The experimental group receives a specific treatment; the control group receives no treatment. Researchers commonly compare groups receiving different treatments. These are called comparison groups. The majority of educational experiments study the difference in the results of two or more treatments rather than the difference in the results of one treatment versus no treatment at all. VALIDITY OF RESEARCH DESIGNS Two Types of Validity 1. Internal Validity 2. External Validity 1. Internal Validity o validity of the inferences about whether the effect of variable A (the treatment) on variable B (the outcome) reflects a causal relationship 2. External Validity o The validity of the inference about whether the cause– effect relationship holds up with other subjects, settings, and measurements CLASSIFYING EXPERIMENTAL DESIGNS According to the number of independent variables: 1. Single-Variable Designs - has one manipulated independent variable 2. Factorial Designs - have two or more independent variables, at least one of which is manipulated According to how well they provide control of the threats to internal validity: 1. Pre-experimental designs o do not use random assignment of subjects to groups and do not control extraneous variables 2. True Experimental Designs (Randomized Designs) o use random assignment of subjects to groups and control extraneous variables 3. Quasi-experimental o do not use random assignment of subjects to groups but control extraneous variables o used (for instance) when intact classrooms are used as the experimental and control groups Notational Conventions for Designs ! independent variable (experimental variable or treatment) " dependent variable dependent variable before the manipulation of the independent "! variable ! (usually a pretest) dependent variable after the manipulation of the independent "" variable ! (usually a posttest) # subject or participant used in the experiment $ experimental group % control group indicates random assignment of subjects to the experimental groups & and the random assignment of treatments to the groups indicates that the subjects are matched and then members of each '! pair are assigned to the comparison groups at random PREEXPERIMENTAL DESIGNS Design 1: One-Group Pretest–Posttest Design Three steps: (1) Administering a pretest measuring the dependent variable (2) Applying the experimental treatment ! to the subjects (3) Administering a posttest, again measuring the dependent variable Assume that an elementary teacher wants to evaluate the effectiveness of a new technique for teaching fourth-grade math. At the beginning of the school year, the students are given a standardized test (pretest) that appears to be a good measure of the achievement of the objectives of fourth-grade math. The teacher then introduces the new teaching technique and at the end of the semester administers the same standardized test (posttest), comparing students’ scores from the pretest and posttest in order to determine if exposure to the new teaching technique made any difference. Is this a good design? Design 2: Static Group Comparison It uses two or more preexisting or intact (static) groups, only one of which is exposed to the experimental treatment. Although this design uses two groups for comparison, it is flawed because the subjects are not randomly assigned to the groups and no pretest is used. The researcher makes the assumption that the groups are equivalent in all relevant aspects before the study begins and that they differ only in their exposure to !. Although this design has sometimes been used in educational research, it is basically worthless! Because neither randomization nor even matching on a pretest is used, we cannot assume that the groups are equivalent prior to the experimental treatment. TRUE EXPERIMENTAL DESIGNS Design 3: Randomized Subjects, Posttest-Only Control Group Design No pretest is used; the randomization controls for all possible extraneous variables and ensures that any initial differences between the groups are attributable only to chance and therefore will follow the laws of probability. After the subjects are randomly assigned to groups, only the experimental group is exposed to the treatment. Design 4: Randomized Matched Subjects, Posttest- Only Control Group Design It is similar to Design 3, except that it uses a matching technique to form equivalent groups. Subjects are matched on one or more variables that can be measured conveniently, such as IQ or reading score. The matching variables used are those that presumably have a significant correlation with the dependent variable. It is most useful in studies in which small samples are to be used and where Design 3 is not appropriate. Design 5: Randomized Subjects, Pretest–Posttest Control Group Design It is one of the most widely used true (randomized) experiments. It randomly assigns subjects to the experimental and control groups and administers a pretest on the dependent variable !. The treatment is introduced only to the experimental subjects (unless two different treatments are being compared), after which the two groups are measured on the dependent variable. The researcher then compares the two groups’ scores on the posttest. Design 6: Solomon Three-Group Design It is similar to Design 5, but it employs a second control group labeled "! that is not pretested but is exposed to the treatment #. The main purpose of this additional control group is to overcome the interactive effect of pretesting and the experimental treatment. FACTORIAL DESIGNS A factorial design is one in which the researcher manipulates two or more variables simultaneously in order to study the independent effect of each variable on the dependent variable, as well as the effects caused by interactions among the several variables. The independent variables in factorial designs are referred to as factors. Factors might be categorical variables such as gender, ethnicity, social class, and type of school, or they might be continuous variables such as aptitude or achievement. Design 8: Simple Factorial Design The simplest factorial design is the 2×2, which is read as “2 by 2.” This design has two factors, and each factor has two levels. QUASI-EXPERIMENTAL DESIGNS Quasi-experimental designs are similar to randomized experimental designs in that they involve manipulation of an independent variable but differ in that subjects are not randomly assigned to treatment groups. Because the quasi-experimental design does not provide full control, it is extremely important that researchers be aware of the threats to both internal and external validity and consider these factors in their interpretation. Although true experiments are preferred, quasi- experimental designs are considered worthwhile because they permit researchers to reach reasonable conclusions even though full control is not possible. Design 9: Nonrandomized Control Group, Pretest– Posttest Design It is one of the most widely used quasi-experimental designs in educational research. It is similar to Design 5 but with one important difference: Design 9 does not permit random assignment of subjects to the experimental and control groups. Design 10: Counterbalanced Design A design that can be used with intact class groups. It rotates the groups at intervals during the experimentation. For example, groups 1 and 2 might use methods A and B, respectively, for the first half of the experiment and then exchange methods for the second half. The distinctive feature of Design 10 is that all groups receive all experimental treatments but in a different order. This design involves a series of replications; in each replication the groups are shifted so that at the end of the experiment each group has been exposed to each $. The order of exposure to the experimental situation differs for each group. The following shows a counterbalanced design used to compare the effects of two treatments on a dependent variable: In using a counterbalanced study to compare the effectiveness of two methods of instruction on learning in science, the teacher could choose two classes and two units of science subject matter comparable in the nature of the concepts, difficulty of concepts, and length. It is essential that the units be equivalent in the complexity and difficulty of the concepts involved. During the first replication of the design, class (group) 1 is taught unit 1 by method !! and class (group) 2 is taught unit 1 by method !". An achievement test over unit 1 is administered to both groups. Then class 1 is taught unit 2 by method !" and class 2 is taught unit 2 by method !! ; both are then tested over unit 2. After the study, the column means are computed to indicate the mean achievement for both groups (classes) when taught by method !! or method !". A comparison of these column mean scores through an analysis of variance indicates the effectiveness of the methods on achievement in science. Quantitative Research Method Consider the following educational research questions: “Why are some children better readers than others?” “What is the effect of single-parent homes on achievement?” “Why do some youths become delinquent while others do not?” Only some questions can be investigated through experimental research! If you want to investigate the influence of such variables as home environment, motivation, intelligence, parental reading habits, age, ethnicity, gender, disabilities, self-concept, and so forth, you cannot randomly assign students to different categories of these variables. Independent variables such as these are called attribute independent variables. An attribute variable is a characteristic that a subject has before a study begins. Ex Post Facto Research o from Latin for “after the fact” o conducted after variation in the variable of interest has already been determined in the natural course of events o sometimes called causal comparative because its purpose is to investigate cause-and-effect relationships between independent and dependent variables Ex Post Facto Research vs. Experimental Research Consider the question of the effect of students’ anxiety in an achievement testing situation on their examination performance. v The ex post facto approach would involve measuring the already existing anxiety level at the time of the examination and then comparing the performance of “high anxious” and “low anxious” students. v The experimental approach could randomly assign subjects to two exam conditions that are identical in every respect except that one is anxiety arousing and the other is neutral. If you wish to reach a conclusion that one variable (X) is the cause of another variable (Y), three kinds of evidence are necessary: 1. A statistical relationship between X and Y has been established. 2. X preceded Y in time. 3. Other factors did not determine Y. Think about this! 1. Why does the current administration of the Department of Education prefer randomized experimental research to ex post facto research? 2. Why do researchers conduct ex post facto research? PLANNING AN EX POST FACTO RESEARCH STUDY 1. State the research problem, usually in the form of a question. Examples: What is the relationship between variable A and variable B? What is the effect of variable A on variable B? The researcher then states a hypothesis about the expected relationship and defines the variables in operational terms. 2. Select two or more groups to be compared. These two groups should differ on the variable of interest, but they should be similar on any relevant extraneous variables. They are selected because they already possess the variable of interest, for example, smoker/nonsmoker and retained/not retained. 3. Determine whether your question requires a proactive or a retroactive design. 3. Determine whether your question requires a proactive or a retroactive design. a. The proactive ex post facto research design begins with subjects grouped on the basis of an independent variable (such as father present/ father not present or retained/promoted). The researcher then compares these preexisting groups on measurers of dependent variables (such as self-confidence, mental health, and academic performance). b. The retroactive ex post facto research seeks possible antecedent causes (independent variables) for a preexisting dependent variable. Whether students graduate from high school or drop out is a variable that cannot be manipulated. Therefore, a researcher would use retroactive ex post facto research to investigate hypotheses about possible causes (such as truancy, attitude toward school, ambition Quantitative Research Method Correlational Research employ data derived from preexisting variables (similar to ex post facto) no manipulation of the variables assesses the relationships among two or more variables in a single group uses an index called a correlation coefficient o positive coefficient means a direct relationship o negative coefficient means an inverse relationship o zero coefficient means no relationship  ± 1 Perfect correlation (negative) correlation ±0.91 - ± 0.99 Very high positive (negative) correlation ±0.71 - ± 0.90 High positive (negative) correlation ±0.50 - ± 0.70 Moderately positive (negative) correlation ±0.31 - ± 0.49 Low positive (negative) correlation ±0.01 - ± 0.30 Slight correlation, negligible positive (negative) correlation 0 no correlation Interpret each of the following: 1. The correlation between time spent watching television and time spent reading is −0.44. 2. The correlation between socioeconomic status and number of museums visited is 0.21. 3. The correlation between days absent from school and kindergarten reading scores is −0.98. Think about this! Uses of Correlational Research The most useful applications of correlation are: (1) assessing relationships (2) assessing consistency (3) prediction Assessing Relationships Possible Questions: o Is there a relationship between math aptitude and achievement in computer science? o What is the relationship between self- esteem and academic achievement? o Is there a relationship between musical aptitude and mathematics achievement among 6-year-olds? Assessing Consistency Possible Questions: o How consistent are the independently assigned merit ratings given by the principal and the assistant principal to teachers in a school? o How much agreement is there among Olympic judges rating the performance of a group of gymnasts? Prediction Correlational research has shown that high school grades and scholastic aptitude measures are related to college grade point average (GPA). If a student scores high on aptitude tests and has high grades in high school, he or she is more likely to make high grades in college than is a student who scores low on the two predictor variables. Positive Linear Relationship Relationship is NOT Linear Negative Linear Relationship No Relationship Correlation Coefficients Pearson Product Moment Coefficient of Correlation (Pearson !) Assumptions: 1. The data pairs fall approximately on a straight line and are measured at the interval or ratio level. 2. The variables have a joint normal distribution. This means that given any specific value of x, the y values are normally distributed; and given any specific value of y, the x values are normally distributed. Pearson Product Moment Coefficient of Correlation (Pearson !) Sample Questions: 1. What is the relationship between years of education and salary potential of teachers? 2. Is there a relationship between math aptitude and achievement in computer science? 3. What is the relationship between the amount of time spent for self-study and academic performance of students during online classes. Spearman Rho Coefficient of Correlation (Spearman ") Assumptions: o The data are measured at the ordinal level. o The data are measured at the interval or ratio level but the condition for normality is not met (so that Pearson % can be used). Spearman Rho Coefficient of Correlation (Spearman ") Sample Questions: 1. Do the rankings of the principal and the assistant principal for 15 teachers in their school agree? 2. How much agreement is there between two Olympic judges rating the performance of a group of gymnasts? The Phi (#) Coefficient o used when both variables are genuine dichotomies scored 1 or 0 o Phi would be used to describe the relationship between the gender of high school students and whether they are counseled to take college preparatory courses or not. Gender is dichotomized as male = 0, female = 1. Being counseled to take college preparatory courses is scored 1, and not being so counseled is scored 0. If you find the phi coefficient in school A is −0.15, it indicates that there is a slight tendency to counsel more boys than girls to take college preparatory courses. Multiple Regression and Factor Analysis Multiple Regression Multiple regression is a correlational procedure that examines the relationships among several variables. Specifically, this technique enables researchers to find the best possible weighting of two or more independent variables to yield a maximum correlation with a single dependent variable. Multiple Regression Examples: Using high school GPA, NCAE GPA and NAT ratings to predict the GPA of a student in college. Predicting the height of a child by looking into the height of the mother, the height of the father, nutrition, and environmental factors. Identifying the selling price of a house by considering the desirability of the location, the number of bedrooms, the number of bathrooms, the year the house was built, the square footage of the lot and a number of other factors. Think about this! Factor Analysis Factor analysis, or exploratory factor analysis, is a family of techniques used to detect patterns in a set of interval-level variables (Spicer, 2005). Factor analysis begins with a table of pairwise correlations (Pearson r’s) among all the variables of interest; this table is called a correlation matrix. The purpose of the analysis is to try to reduce the set of measured variables to a smaller set of underlying factors that account for the pattern of relationships. Imagine that you have scores on six different subscales of an aptitude measure for 300 subjects. The correlations among all the pairs of scores are shown below. The question is: Is there a simpler structure underlying these 15 correlations? The pattern of correlations among these variables seems to reflect three underlying factors, which we can label verbal, numerical, and spatial. Factor 1 - verbal Factor 2 - numerical Factor 3 - spatial SURVEY RESEARCH Quantitative Research Method Skezeer John B. Paz Notre Dame of Marbel University What is a Survey Research? In survey research, investigators ask questions about peoples’ beliefs, opinions, characteristics, and behavior. A survey researcher may want to investigate associations between respondents’ characteristics such as age, education, social class, race, and their current attitudes toward some issue. Survey research typically does not make causal inferences but, rather, describes the distributions of variables in a specified group. Which of the following questions would best be answered by survey methods? 1 3 Do voters in our school What do voters consider district think we should the most important issues raise taxes in order to build in the upcoming election? new classrooms? 2 4 Does dividing second-grade Do people who have taken math students into ability driver education have groups produce greater math fewer accidents than achievement than doing math people who have not? instruction in a single group? TYPES OF SURVEYS  Types of Survey According to According to Focus and Time Frame for Scope Data Collection Census Sample Survey Longitudinal Cross-Sectional SURVEYS CLASSIFIED ACCORDING TO FOCUS AND SCOPE A survey that covers the entire population of interest is referred to as a census. The term population is used to refer to the entire group of individuals to whom the findings of a study apply. The researcher defines the specific population of interest. It is often difficult or even impossible for researchers to study very large populations. Hence, they select a smaller portion, a sample, of the population for study. A survey that studies only a portion of the population is known as a sample survey. SURVEYS CLASSIFIED ACCORDING TO FOCUS AND SCOPE Surveys may be confined to simple tabulations of tangibles, such as what proportion of children ride school buses and the average class enrollment. The most challenging type of survey is one that seeks to measure intangibles, such as attitudes, opinions, values, or other psychological and sociological constructs. 1. A Census of Tangibles o seeking information about small population and when the variables involved are concrete Example: If a school principal wants to know how many desks are in the school, how many children ride the school bus, or how many teachers have master’s degrees, a simple count will provide the information. 2. A Census of Intangibles o seeking information about small population and when the variables that are not directly observable but must be inferred from indirect measures Example: Suppose the school principal now seeks information about pupil achievement or aspirations, teacher morale, or parents’ attitudes toward school. 3. A Sample Survey of Tangibles o seeking information about concrete variables from a large population by using sampling techniques and using the information collected from the sample to make inferences about the population as a whole Example: Identifying the distribution of obese and non-obese students of the public schools in the Philippines. 4. A Sample Survey of Intangibles o seeking information about not directly observable variables from a large population Example: Using opinion polls on people’s choice from the list of presidentiables. What type of survey is illustrated in the following examples? 1 3 Randomly selected teachers The division superintendents’ are asked how many years list of the enrollment in each experience they have been of the schools in the division. teaching. Sample Survey of Tangibles Census of Tangibles 2 4 Some students in NDMU are The superintendent of School given a physical fitness test to District has a Test of Basic get an estimate of the fitness Skills scores for all second- of all the students in the graders in his district. school. Sample Survey of Intangibles Census of Intangibles a) Panel Studies b) Trend Studies c) Cohort Studies 1. Longitudinal Surveys o gather information at different points in time in order to study the changes over extended periods of time o Three different designs: a) panel studies b) trend studies c) cohort research a) Panel Studies o the same subjects are surveyed several times over an extended period of time Example: A researcher studying the development of quantitative reasoning in elementary school children would select a sample of first- graders and administer a measure of quantitative reasoning. This same group would be followed through successive grade levels and tested each year to assess how quantitative reasoning skills develop over time. b) Trend Studies o different individuals randomly drawn from the same general population are surveyed at intervals over a period of time Example: Researchers who have studied national trends in mathematics achievement sample middle school students at various intervals and measure their math performance. Test scores from year to year are compared to determine if any trends are evident. c) Cohort Studies o a specific population is followed over a length of time with different random samples studied at various points o Typically, a cohort group has age in common Example: A school system might follow the high school graduating class(es) of 2021 over time and ask them questions about higher education, work experiences, attitudes, and so on. From a list of all the graduates, a random sample is drawn at different points in time, and data are collected from that sample. Thus, the population remains the same during the study, but the individuals surveyed are different each time. 2. Cross-Sectional Surveys o study a cross section (sample) of a population at a single point in time Example: Longitudinal vs. Cross-Sectional In a longitudinal study of vocabulary development, a researcher would compare a measure of first-grade students’ vocabulary skills in 2015 with one when they were fourth-grade students in 2018 and seventh-grade students in 2021. A cross-sectional study would compare the vocabulary skills of a sample of children from grades 1, 4, and 7 in 2006. Answer This! You are going to conduct a research by using a questionnaire on some experiences of the junior high school students regarding online education. How will you administer a (a) cross-sectional approach? (b) panel study? (c) trend study? (d) cohort study? In the cross-sectional study, draw a random sample from each of the four year levels and administer the questionnaire to them at the same time. Answer This! You are going to conduct a research by using a questionnaire on some experiences of the junior high school students regarding online education. How will you administer a (a) cross-sectional approach? (b) panel study? (c) trend study? (d) cohort study? In the panel study, randomly draw a sample of freshmen from your population of interest, and assess your original sample and study the same individuals again when they are sophomores, juniors, and seniors. Answer This! You are going to conduct a research by using a questionnaire on some experiences of the junior high school students regarding online education. How will you administer a (a) cross-sectional approach? (b) panel study? (c) trend study? (d) cohort study? In the trend study, randomly draw a sample of freshmen from your population of interest. A year later, draw a random sample of sophomores and then after a year, draw a random sample of juniors, and then in the final year you draw a random sample of seniors. Answer This! You are going to conduct a research by using a questionnaire on some experiences of the junior high school students regarding online education. How will you administer a (a) cross-sectional approach? (b) panel study? (c) trend study? (d) cohort study? The cohort study would differ from the trend study in that the subsequent samples are drawn only from the population who were enrolled as freshmen when the study began and does not include students who transferred in later.