Understanding Research Design

• A research design is an investigation's detailed plan, it is critical to objectively analyze data and test hypotheses related to research problems.
• The purpose of a research design is to maximize relevant information while minimizing costs.
• Research designs objectively, validly, and economically answer research questions, often epitomized by hypotheses.
• Research design acts as a control mechanism by enabling researchers to control unwanted variances: experimental, extraneous, and error variance

Experimental Variance

• Experimental variance is produced by the manipulation of experimental or independent variables.
• Researchers aim to maximize experimental variance to obtain valid and objective data and design experiments to maximize differences between experimental conditions.

Extraneous Variance

• Extraneous/control variance is produced by extraneous/relevant variables.
• Researchers control relevant variables to eliminate variances produced by them and can be achieved through control, and discussed in Chapter 20.

Error Variance

• Error variance refers to variabilities in measures due to factors not controllable by the experimenter, like individual differences among subjects (attitude, motivation, etc.) or errors of measurements.
• Error variance is self-compensating with positive and negative variability, tending to cancel out in repeated measurements, resulting in a mean of zero.
• Error variance is unpredictable and is based on random errors, distinguishing it from predictable systematic variance.
• Minimizing improves the reliability of measures, strengthening the experiment's external validity, and allows systematic variance to show its significance.
• Controlling experiment conditions minimizes error variance, while uncontrolled conditions lead to higher proportions of error variance.
• Good research design provides unambiguous results, avoiding "confounding" where multiple variables change concurrently, causing vagueness in cause-effect relationships.

Criteria of Research Design

• Research design is a data guide, with some designs being stronger than others.
• Behavioral researchers have established criteria to distinguish weak from strong designs, aiding researchers.

Capability to Answer Research Questions Adequately

• Answer research questions adequately and designs that do not constitute weak research.
• Matching subjects on irrelevant variables or using an inadequate number of groups exemplifies weak design.
• Factorial design is appropriate for testing interaction hypotheses, whereas a two-group randomized design is not.

Control of Variables

• Extraneous variables, similar to independent variables, influence dependent variables.
• Designs lacking control over such variables are weak and should be avoided.
• Randomization is a method for controlling extraneous variables involving random participant selection, random assignment to control/experimental groups, and random assignment of experimental treatments among groups

Generalizability

• Generalizability is the external validity which determines to what populations is the studies generalizable.
• If the design allows generalizing results to larger groups or subjects, it's considered good with technical aspects (sampling, etc)
• Generalizability is less crucial in basic research, but it is important to consider the intention of real world practice of generalization in applied research

Basic Principles of Experimental Design

• Experimental designs should allow for objective data in a way that provides a cause/effect relationship between the independent and dependent variable.
• Independent variables are manipulated by the experimenter, while dependent variables are affected by these manipulations.
• The experimental design basics: replication, randomization, and local control

Replication

• Replication combines duplication and repetition, involving the deliberate repetition of an experiment with nearly identical procedures, different subjects, settings, and times.
• Replication is important for revalidating experiments and raises accurate questions of error. Winer (1971:391) notes that replications of an experiment in the same or near identical conditions under the same conditions, as permissible.

Randomization

• Randomization is the second basic principle of the experimental design. It makes the test valid.
• Randomization maintains the independence of observations, making statistical tests valid.

Local Control

• Local control uses balancing, blocking, and subject/experimental grouping. Grouping assigns homogenous experimental units/subjects into a group, so that different groups of homogenous subjects may be available for treatments during experimentation.
• Balancing is when designs appear to be balanced, where they are grouped, blocked and assigned appropriately. The statistic is experimentally sound and must possesses property of local control.

Types of Research Design

Selecting a design depends partly on whether the experimenter will use 1 or more subject groups.

• If there will be only one group, tested under different values design/conditions and experimental setup of the dependent variable, the results may be known as the "within-groups" design
• If there will be separate groups for each value of the independent variable, the design may be referred as "between-groups"
• Kantowitz & Roediger (1984) note that Between-group and With-in groups are respectively "between-subjects" and "within-subjects".

Between-Groups Design

• The most common: randomized-groups design, matched-groups design and factorial design
• Randomized-groups assume there is random assignment of subject into groups, indicating a statistical equivalence.
• Equal assignments result in 2 randomized-groups designs, while more than two assignments results in a multi-groups design.
• Match-groups depend of mean, standard deviation, pairs
• Factorial design is one in which 2 or more independent variables are studied in all possible combinations

Within-Groups Design

• Two types: complete and incomplete
• Complete: practice effects are balanced
• Incomplete: only one condition is administered at each test
• Results for each test become interpretable, with each subject combined in differing orders
• Neutralizing practice effects is achievable when testing conditions per-subject only once while modifying experiment orders.

Two-Randomized-Groups Design

• Subjects are randomly assigned to two groups
• Define independent and dependent Variables
• Two Values are selected (conditions, treatments, effects)
• The experimenter defines the population which must be a specified set, where sample subjects represent the population properly.
• Subjects must be split properly and randomly within groups
• Subjects are split equally to ensure equal groups in experimentation
• Coins are often used to differentiate between the two groups
• Scores on the dependent variable are recorded and statistically evaluated to gauge differences between group actions
• If statistics on the dependent variable differ, then there is a link and difference within the manipulation of the independent variable.
• Reward has been shown to accelerate the learning of verbal in kindergarten cohorts
• This design allows achieving unbiased groups or at random, with captive assignment or sequential assignment.

Captive Assignment Technique

• Captive assignment is where everyone is present, known to the experimenter, and can be tested within the experiments duration.