Research Design: Hypothesis Testing & Data Analysis

• Research design serves as the detailed plan and blueprint for testing hypotheses and analyzing data.
• Steps are taken ahead of time to ensure relevant data collection for objective analysis in the hypothesis testing process.
• Research design assists in hypothesis testing, leading to valid conclusions about the relationship between independent and dependent variables.
• Research design should be based on the purpose of the investigation, variable types, and research conditions, not the researcher's personal preferences.
• The main purpose of any research design is to maximize relevant information while minimizing costs.
• Research design serves two main functions in a scientific investigation.
• Answering research questions objectively, validly, and economically.
• Acting as a control mechanism to manage unwanted variances.

Answering Research Questions

• Research problems are summarized by forming hypotheses.
• Research design guides data collection for testing hypotheses.
• It also determines how to treat control variables, adequate manipulation methods, statistical analyses, and potential solutions.
• Through related steps, research design helps the researcher to reach valid, objective answers.

Control Mechanism

• Research design controls unwanted variances.
• Three common variances in investigations.
• Experimental variance
• Extraneous variance
• Error variance

Experimental Variance

• Experimental variance is produced in the dependent variable due to the manipulation of the independent variables by the experimenter.
• Researchers aim to maximize experimental variance to obtain valid, objective data.
• Maximizing experimental variance involves designing the experiment with distinct experimental conditions.
• For example, when studying the impact of reward on learning rate, subjects are split into groups based on reward levels.
• Learning rate is measured by the number of trials taken to reach a criterion.
• The experimental variance is maximized when learning rates differ significantly between high and low reward conditions.

Extraneous Variance or Control Variance

• Extraneous variance results from extraneous or relevant variables.
• Researchers control relevant variables and eliminate variances they produce.
• Proper control of extraneous variables is essential for eliminating extraneous variance.

Error Variance

• Minimizing error variance is the third function of a research design.
• Error variance is the variability in measures due to factors not controllable by the experimenter.
• Factors include individual differences (attitude, motivation, ability) and measurement errors (trial differences, experimental conditions).

Features of Error Variance

• Error variance is self-compensating, with variability that is sometimes positive and sometimes negative.
• It tends to cancel out in repeated measurements, making the mean error variance zero.
• Error variance is unpredictable and based on random errors.
• It is different from systematic variance because systematic variance is predictable.
• Minimizing error variances improves the reliability and accuracy of generalizations.
• It strengthens the external validity of the experiment.
• It allows systematic variance to show its significance and gives the systematic variance a chance to show its significance, if it is really significant.
• Error variance is reduced by controlling experiment conditions.
• Uncontrolled conditions lead to the accumulation of error variance.
• A well designed study avoids "confounding".
• Confounding occurs two or more variables change together.
• Consequently changes in dependent variables cannot be linked to a single variable.
• This introduces vagueness to relationships.

Criteria of Research Design

• Researcher's use research design as data discipline.
• Certain criteria distinguish a strong research design from a weak one.
• Capability to answer research questions.
• Control of variables.
• Generalizability.

Capability to Answer Research Questions Adequately

• A good research design has the ability to adequately answer research questions.
• Selecting an inappropriate design results in weak research.
• If no relation connects age and the dependent variable, matching age will be irrelevant.
• Designs based upon subject matching would be a weak design.
• Designs should include enough groups.
• Randomized designs may not be sufficiently adeqaute.
• Factorial designs are best for testing interaction hypotheses instead of randomized designs.

Control of Variables

• A good research design should control the effects of extraneous variables, which can influence dependent variables.
• Uncontrolled extraneous variables weaken the design.
• Randomization is a top tier technique for controlling extraneous variables.
• Three phases of randomization.
• Selection of subjects.
• Assignment of subjects.
• Assignment of experimental treatments; may be done among groups.

Techniques to Randomization

• Randomly assign selected subjects to different experimental groups.
• Randomly assign treatments among experimental groups if random assignment is not possible.
• Randomization is shown to control extraneous variables.
• A well-controlled extraneous design = the best design for studies, increasing internal validity.

Generalizability

• Generalizability is the external validity of the research design.
• It is the extent to which research results generalize to other subject groups or conditions.
• Designs which do not have results generalized to large groups suggests a positive study.

Concerns of Generalizability

• Sampling.
• Research design.
• Larger problems of basic and applied research.
• Fundamental research does not focus on generalizability.
• Applied research is concerned with generalizability to give results to other people, conditions, and/or situations.

Basic Principles of Experimental Design

• The experimental design is the basis of all experimental investigations.
• An experimental design is a sequence of steps that allows objective analysis of objective data.
• It is designed to infer a cause-effect relationship between the independent and dependent variables.
• Independent variables being things the experimenter manipulates.
• Dependent variables being the response or behavioral measure affected by the manipulation, such as intensity of illumination upon retinal fatigue.

Sound experimental design principles

• Replication
• Randomization
• Local Control
• Replication involves repeating an experiment using identical processes with a different set of users and at a different place and/or time.
• This allows you to address questions from the prior study.
• Replication also provides an accurate experimental error estimate.
• Experimental errors must not be confused with multiple measurements.
• Experimental error is generated from these four areas.
• Faulty experimental design
• Faulty measurements
• Biased Observation
• Uncontrolled variation (experimental units for subjects)
• Randomization is critical to making a test/experiment valid.
• Each statistical test has to have results drawn independently.
• Independence is maintained when subjects are selected at random.
• When subjects are selected for an experiment, this avoids subjects that are left uncontrolled.
• Do not be too rigid with trying to randomize.
• Local Control means the balancing, blocking, and grouping of subjects/experimental units in a design.
• It's a part of every experimental idea.
• Groupings assign groups of people and/or units for assignments.
• Blockings assigns experimental units to blocks so blocks can be homogeneous.
• An experimental design is done so groups and blockings are set up in a way that is setup in a way that statistically and experimentally possess local control.