BIOL208 - Lab 0: Experimental Design PDF

# Lab 0: Part 4: Experimental Design ## Overview This section overviews the basics of experimental design in ecology. These guidelines will help you with designing experiments and testing hypotheses in BIOL208 labs. ## Objectives At the conclusion of this lab, participants will be able to: - Di...

# Lab 0: Part 4: Experimental Design ## Overview This section overviews the basics of experimental design in ecology. These guidelines will help you with designing experiments and testing hypotheses in BIOL208 labs. ## Objectives At the conclusion of this lab, participants will be able to: - Distinguish between observational/mensurative and manipulative experiments. - Define and apply appropriate terminology related to experimental design. - Apply three main principles of experimental design to your own research questions. - Critique experimental designs using the FINER criteria. ## Connection to broader concepts - How is the scientific method used to answer questions? ## General Principles of Experimental Design ### Types of Experiments In ecology, experiments can be either: - **Observational (mensurative)** - **Manipulative** In an observational experiment, measurements are made on what currently exists and conclusions are drawn from patterns in the data. This can be as simple as measuring deer density in forest versus grassland habitats to determine which habitat is preferred, or using natural history observations or observations collected by a citizen science project to support or refute your hypothesis. In BIOL208 you will design and conduct your own observational experiments in the "Scientific Method in the River Valley" labs and the "Disease Ecology" lab. A manipulative experiment involves intentionally manipulating a variable that you have control over. A manipulative experiment requires at least two groups - a control (that does not manipulate the variable) and a treatment group that intentionally changes the variable you are testing. ## 0-24 Experimental Design Manipulative experiments can vary multiple levels of one type of treatment, or can even test multiple variables at multiple levels. In both types of experiments your data is analyzed by statistical test(s) to draw conclusions about significance of the results. Whether you conduct a mensurative/observational experiment or a manipulative experiment, it is important to use proper experimental design to ensure that your data collection, and the results you later conclude from it, are robust and reliable. ### Experimental Design: Main Principles Experimental design seeks to answer three main questions: - **What to measure** - **How to measure it** - **How many samples to take to be confident in your results** What you measure and how you measure it will be determined by your research question and your hypothesis. How you measure your variable will depend on what it is, but in ecology measurement devices can range from high-tech instrumentation to simple counts, it all depends on what you are doing, and often on your budget for the experiment. How many samples you take can be a complicated question. You will often need replication to have a statistically valid experimental design, but it is not always feasible or recommended to collect as many samples as possible. Amount of samples taken can be limited by resources such as time, budget, equipment, and effort and there is often a trade-off that needs to be considered when designing an experiment. ### The three "R"s of experimental design - **Randomization** means to randomly allocate treatments to experimental units. This helps neutralize any other effects that may be occurring that are not due to your treatment. Most statistical tests assume randomization. - **Replication** is the repetition of treatments within an experiment. Replication helps quantify the natural variation between experimental units and to increase accuracy of your results. In order to ensure that the response you see is truly due to the manipulation or treatment you completed, the results need to be reproducible, and this is where replication comes in. - **Reducing noise** means to try to control other conditions as much as possible to help you again be certain that the effects you see are caused by what you are proposing. ## 0-25 Terminology in Experimental Designs - **Independent Variable:** This is the variable we hypothesize will cause a change in the variable we are measuring, also called the "manipulated" variable. When creating a graph, the independent variable will go on the x-axis. - **Dependent Variable:** This is the variable that we suspect changes related to changes in the independent variable, also can be called the "responding" variable. Graphically, the dependent variable is shown on the y-axis. - **Treatment:** Groupings that we are comparing in the experiment. Can be: - different levels of a single factor - complex combinations of more than one factor. - different levels of the treatments would be shown as discrete groups on the x-axis - **Experimental Unit:** a physical unit that receives a particular treatment. - Eg. a plot in a field, or a single plant exposed to a certain level of a manipulated variable - **Measurement Unit:** the level at which observations are made. Usually one measurement per experimental unit, but can also do repeated measurements on a single experimental unit. - but these repeated measurements do not increase the statistical power if you are measuring the same experimental unit. (This is called pseudoreplication). - **Replication:** the number of independent instances of a treatment that occur in an experiments. - Eg. Separate plots receiving the same fertilizer treatment. ## 0-26 Experimental Design ### Sampling Designs and How to Sample Organisms We usually cannot measure total populations; instead, we take samples and use statistics to make estimates on the population. It is essential that samples be representative of the whole population if we want good estimates, and this can be done through careful experimental design. Two features will determine if the sample is representative of the entire population. The sample must be: - **Unbiased** - **Adequate in size** ### Sampling Design Sampling design refers to how you decide which areas to sample. There are three basic types: - **Random** - **Systematic** - **Combination of random and systematic** The usual approach for obtaining an unbiased sample is to sample randomly. Taking a random sample requires special procedures to assure true randomness. In contrast to random sampling, systematic sampling involves taking samples that have some sort of regular arrangement. The advantage of systematic sampling is that it is usually simpler to conduct. Bias will be present only if the sampling pattern reflects some pattern in the population. Another sampling design is to use a combination of random and systematic sampling. Overall, random sampling is usually recommended, as most statistical tests are built around the assumption of random sampling. ### Sampling Methods Two common sampling methods we use in BIOL208 labs are: - **The quadrat method** - A quadrat is a (usually square-shaped) sampling area of a known/defined size. - used for small, sessile or relatively sedentary organisms (non-motile) - we will practice quadrat sampling for plants in the river valley labs and in a prelab simulation activity - **The mark/recapture method** - used for relatively large and/or mobile organisms. - We will use mark and recapture in a future lab to make estimates about bean beetle populations. Other sampling methods: - Will depend on what you are studying - See the Table 0-1 for a list of common sampling methods for a wide diversity of organisms ## 0-28 Experimental Design ### Evaluating an experimental design using the FINER framework FINER (Feasibility, Interest, Novelty, Ethics, Relevance) is a framework that can be used to formulate an appropriate experimental design. Keep these points in mind when you are designing your own experiments in BIOL208. | Component | Criteria | |---|---| | **Feasible** | - Research design is adequate - Aims an achievable sample size - Optimises technical resources - Opts for appropriate and affordable frame time | | **Interesting** | - Engages the interest of the researchers - Attracts the attention of readers/public - Presents a different perspective of the problem | | **Novel** | - Provides novel findings - Generates new hypotheses - Resolves a gap in the existing literature | | **Ethical** | - Complies with local ethical committees - Safeguards the main principles of ethical research | | **Relevant** | - Generates new knowledge - Stimulates further research - Provides an accurate answer to a specific research question | ### Example of an experimental design You want to study how understory species biodiversity is affected by light availability in the river valley. You hypothesize that lower light availabilities will negatively impact plant biodiversity by decreasing the amount of photons available for photosynthesis. You predict that species richness will be lower in closed canopy areas of the river valley than in open canopy areas. **Image description:** A diagram depicting three sites of each type "open" canopy quadrat and "closed" canopy quadrat. ## 0-29 Here is an overview of experimental design example: | Variable | Description | |---|---| | **Independent variable:** | the amount of light penetration/canopy coverage | | **Dependent variable:** | Average understory plant species richness | | **Treatments:** | closed canopy and open canopy | | **Experimental unit:** | need to define the size of area you will sample under each canopy condition. Eg. 5 m x 5m quadrat under areas that are pre-designated as "open" or "closed" by visual estimate of canopy coverage. | | **Measurement unit:** | 1 value per experimental unit Eg. a single count value of the number of individual species present in each quadrat (species richness) | | **Replication:** | need to decide how many "closed" canopy and how many "open" canopy plots you will measure. Eg. three of each type is the minimum to be able to conduct meaningful statistical analyses such as calculating means and confidence interval, or running a t-test. Eg. 3 open sites, and 3 closed sites. | | **Sampling design:** | This is an example of a combination design, since you are predetermining sites based on a separating characteristic (open vs closed canopy), but you will randomize where the open and closed sites are located within the larger space of the river valley (not having all the "open" sites located close to each other). | ### Summary These guidelines will help you design your own experiments in BIOL208 Labs, and will help you critique experimental designs in published scientific literature. After completing this lab you should be able to: - Distinguish between observational/mensurative and manipulative experiments - Define and apply appropriate terminology related to experimental design - Apply three main principles of experimental design to your own research questions - Critique experimental designs using the FINER criteria. **Make sure to refer to the following additional resources:** - Lab 2: Scientific Method in the River Valley - Lab 3: Collecting ecological data