BIOL 391 Intro to Biological Research, Fall 2024 PDF

11/17/24 Chapter 22 Definition of the Experiment. The Framework for an Individual The Experiment...

11/17/24 Chapter 22 Definition of the Experiment. The Framework for an Individual The Experiment Experiment. Ch. 22 - 24, Glass (2014) “What is the question that you want to BIOL 391 answer?” Intro to Biological Research Burman, Fall 2024 1 2 What is a framework? Refers to multiple aspects of the project... 1. Why is the experiment being considered? 2. What type of experiments need to be done? 3. How will the experiment be designed? 4. How will the data be analyzed? 5. What can be obtained from an experiment? 6. How does one use the experimental results? 3 4 Project Framework Experimental Framework A project contains individual Each experiment has its own framework, which is a subset of the project framework that experiments and is surrounded by the governs the whole project. framework. Each experiment is performed in The project framework governs many response to a particular question or hypothesis. of the choices made within a project, If the experiment’s framework is a question, such as what system to use, and what data is the experiment is designed to answer the important. question (just as the project is designed to answer the question that constitutes the project framework). 5 6 1  11/17/24 Experimental Program: Includes Distinct Projects, Experimental framework performed under particular Frameworks Project B The experiment is designed so that the data Experiment 1 from a successful experiment will be Project A useful in building a model that can be used Experiment 2 Experiment 1 to answer the question that has been posed. Experiment 2 The scientist must show that the data derived from the experiment are predictive – i.e., FRAMEWORK successful experiment will give a consistent FRAMEWORK answer when the experimentt is repeated, and thus the data can be used in building a predictive model. 7 8 eg., Project framework: “What Color is the Sky?” System to 1) measure wavelengths of light; 2) system must be able to determine the color of the sky. System of measuring color is validated before it is used to measure sky color. Expt 1 framework: “What color is the sky at 9 am?” (subset of project framework that requires a measurement at a particular time of the day). 9 10 eg., Project framework: “What Color is the Sky?” Expt 2 framework: “What are the colors observed over a 24-hour period?” (subset of project framework that requires measurements of sky color during the course of a single day). In each type of expt, data from each expt can be verified to see if resultant data is reproducible. The broad project framework functions to establish more discrete, less broad measurements. 11 12 2  11/17/24 eg., Project framework: “What Color eg., Project framework: “What Color is the Sky?” is the Sky?” In each type of expt, data from each expt Note that each individual experiment: can be verified to see if resultant data is reproducible. – Requires a distinct experimental method and setup. Other questions can be asked, like “Is the sky blue (or red) during the 24-hour period?” – The way the question is asked dramatically The broad project framework demands the changes the sort of the experiment to be ability to measure the colour spectrum, and performed. thus allows the specific subset questions to be – Answers from individual experiments is asked in individual experiments (i.e., broad project framework creates more of a helpful in providing an answer to the demand for these subset questions). overarching project question. 13 14 eg., Project framework: “What is the Effect of Caffeine on Blood Pressure?” System – 1) measuring blood pressure, 2) administering caffeine, and 3) measuring changes in blood pressure after exposure to caffeine. Expt. 1 framework: “What is the effect of a particular range of caffeine on blood pressure?” – Administration of caffeine doses; measuring [blood caffeine], i.e., caffeine dose response; measurement of blood pressure. 15 16 eg., Project framework: “What is the eg., Project framework: “What is the Effect of Caffeine on Blood Pressure?” Effect of Caffeine on Blood Pressure?” Expt. 2 framework: “What is the effect on blood pressure of administering a The scientist will know that the project particular concentration of caffeine every 4 was a success if the predictive power of hours?” the model derived from the project can – Administration of caffeine in a repeat-dose successfully predict the effect of scheme and measuring blood pressure over caffeine. time. Project framework creates a demand for a system that can answer each subset question. 17 18 3  11/17/24 eg., Project framework: “What is the Effect of ras Pathway Inhibition on Pancreatic Cancer?” System – 1) measuring readouts of the ras pathway, 2) validating the ability to inhibit this pathway, 3) establishing a pancreatic tumor model, and 3) measuring changes in this model. Expt. framework: “What is the effect of knocking out/down raf on tumor cell proliferation?” – Inhibition of ras pathway at raf, and measuring the capacity of a pancreatic tumor cell to divide. 19 20 eg., Project framework: “What is the Effect of ras Pathway Inhibition on Pancreatic Cancer?” Alternate expt. framework: “What is the effect of knocking out/down raf on tumor cell death by apoptosis?” – Inhibition of ras pathway at raf, and measuring the rates of apoptosis in response to the raf inhibitor. 21 22 Semantics of a Project Statistics Definitions of keywords; it is all about In designing the first experiments, one must language. think about the statistical tests that will e.g., be used and the comparisons that will – What do you call a significant increase in be sought. blood pressure? 1. This will help determine the necessary – What do you characterize as Akt control groups to use. phosphorylation? 2. Number of subjects/plates/test tubes/etc – What are the ranges of responses? required for each group. – What is a “relevant change”? - Group size is determined by the variability (i.e., stability) of the system under investigation. 23 24 4  11/17/24 Statistics Summary To determine group size: The project framework questions asks the – During the validation stage, while one is overarching question that forces the performing trial runs, the scientist will scientist to ask smaller experimental determine the variability in the measured framework questions. parameter. e.g., variations in blood pressure first determined While the system may be similar in each to determine “noise” of normal variation. experiment under the project framework, each – One also need to know the expected degree individual experiment requires a distinct of perturbation. experimental method and setup. e.g., expected differences in blood pressure in various The way each subset question is asked may experimental groups. Larger groups provide a narrower data distribution within an dramatically change the sort of experiment experimental group and a more significant performed. difference between treatment groups. 25 26 Summary The answers/data from individual experiments provide information for model building which, one will hope, hold up to subsequent testing of its predictive power. Statistical treatment of readouts are determined by validation experiments that define number of experimental groups, group sizes, key comparisons that will be needed and what sorts of probabilities are sought. 27 28 Chapter 23 The Definition of the Negative Control The negative control demonstrates that in the The Negative Control experiment “What is effect of X on Y” you have Distinct Types sufficient data to measure the unique effect of “X”. Thus, the negative controls are the The negative control can be measurements you need to prove you can measure “X” vs the unperturbed case, AND defined as “The unperturbed by X” against everything else perturbed in your control, where X is the thing who’s system besides “X”; the negative control is more than simply “unperturbed”; it’s controlling for effect you seek to measure. everything perturbed in your system perturbed in addition to X, where X is the particular thing you’re interested in. 29 30 5  11/17/24 The Definition of the Negative Control Isolating the Perturbation as the Only Variable. The measurement in the setting of an experiment usually means a comparison: The negative control is the setting in which seeing a difference between a the experimental subject is not being perturbed by the variable under study. perturbation and a series of “negative controls.” The “unperturbed case” is often difficult to achieve. The purpose of the negative control is to Individuals in each group must be controlled ensure that the experiment can be for possible “relevant variable”, meaning analyzed for the unique effect of X on additional factors that may influence the the subject under study. experimental output. 31 32 Eliminating other variables Eliminating other variables e.g., “What is the effect of caffeine on blood Limiting Additional Variables that could effect the measurement by controlling for: pressure?” – hypertension, other sources of caffeine besides What happens if the “negative control” those in the experiment, other chemical group had other conditions that altered perturbations of blood pressure, stress their blood pressure? The inductive space may come in handy in – e.g., more subjects in this group suffer from determining these additional variables. “white coat syndrome”, a temporary condition Scientist will take steps to establish that the where they get anxious (and hence have higher unperturbed negative control group is blood pressure) around medical personnel. matched with the caffeinated group for all relevant variable that can be identified. 33 34 Eliminating other variables Eliminating other variables e.g., Screening: It is critical that the 2 groups do not – Each group is screened for preexisting vary in other factors that may make it hypertension, and candidates who are impossible to understand how caffeine hypertensive are eliminated from the study. affects blood pressure. – Starting blood pressure and BMI could be determined for each individuals, and individuals (What is the effect of caffeine on blood with similar blood pressures and similar BMIs pressure, in the absence of other known should be distributed equally between groups. pressors?) (Is that what you meant to ask?) – Questionnaire to ask if coffee, or any other caffeinated beverages, were consumed before the study. 35 36 6  11/17/24 Negative control – the “Unperturbed by X” Control. e.g., What is the effect of caffeinated coffee on blood pressure? – Does caffeinated coffee change blood pressure? – Is the caffeine in coffee the culprit? So, a more defined question would be “Does caffeinated coffee affect blood pressure, and if so, is the caffeine responsible?” Multiple components, each which could perturb the experimental subjects, need to be considered in constructing the “unperturbed by X” negative control. 37 38 Study design (e.g., Does caffeinated Study design coffee affect blood pressure, and if so, is the caffeine responsible?) BMI < 40 and > 19 6 groups of 50 people each – match groups for average BMI, and also ranges – Having a sufficient number of people in each of BMI group controls for undetermined genetic Age 18-45 yo variations. – match groups for average age and similar age Determine subjects starting blood pressure range (bp) No caffeinated beverages 72 hr before – eliminate subject if bp > 140/90 or those < 90/60 experiment. Match for gender 39 40 Study design Groups Questionnaire: – normal caffeine intake, anxiety disorders and A. No treatment family history of hypertension B. Water: four 8-oz cups/day – eliminate those with anxiety disorders and those C. Decaffeinated coffee: four 8-oz cups/day treated for hypertension D. Caffeinated water: four 8-oz cups/day; caffeine Baseline bp twice a day, everyday for 1 levels set to match those in the caffeinated month. coffee – account for variations within and between days E. Caffeinated coffee: four 8-oz cups/day Collect serum at start, and every 3 days F. Caffeinated cola: four 8-oz cups/day; caffeine during study (can be frozen and analyzed levels set to match those in the caffeinated later as needed). coffee 41 42 7  11/17/24 Study design Rationale for each group A. No treatment The degree of bp changes in each group will – This group is unperturbed by any change, be compared with one another. including the change by drinking four cups of This measurement is meaningless fluid/day; establishes baseline. unless there is a point of comparison. B. Water: four 8-oz cups/day – This group is unperturbed by the additional – That point is the negative control, the ingredients provided by the coffee or cola, each of unperturbed case. which is dissolved in water; control for volume. Each of the groups, except for the C. Decaffeinated coffee: four 8-oz cups/day caffeinated coffee group (i.e., E), establishes – This group is unperturbed by caffeine an unperturbed by X negative control. compared to the caffeinated coffee. 43 44 Rationale for each group D. Caffeinated water: four 8-oz cups/day; caffeine levels set to match those in the caffeinated coffee – Unperturbed by coffee in comparison to the caffeinated coffee; Also unperturbed by other ingredients in caffeinated cola in comparison to the caffeinated cola. E. Caffeinated coffee: four 8-oz cups/day – Test case! 45 46 Groups Rationale for each group A. No treatment F. Caffeinated cola: four 8-oz cups/day; B. Water: four 8-oz cups/day caffeine levels set to match those in the C. Decaffeinated coffee: four 8-oz cups/day caffeinated coffee D. Caffeinated water: four 8-oz cups/day; caffeine – Assumption control; this is not a negative levels set to match those in the caffeinated control. coffee – Assumption controls eliminate E. Caffeinated coffee: four 8-oz cups/day assumptions contained within the F. Caffeinated cola: four 8-oz cups/day; caffeine experimental question, e.g., assumption that levels set to match those in the caffeinated the other ingredients in coffee will not perturb coffee the effects of caffeine alone. 47 48 8  11/17/24 Inclusion of a Positive Control More realistic caffeine expt. The positive control is an internal system What is the effect of caffeine on blood pressure? control – it proves the system is Experimental design: operation in that experiment. A. Give 50 people caffeine in pill form. Group G (positive control) - Give 50 B. Give 50 different people pills that are identical to A) except without caffeine (controls for the people known perturber of blood pressure. caffeine). – Allows one to determine, with a known C. Give 50 different people nothing (controls for the perturber of blood pressure, is our system is able pill) – also controls for the placebo effect in B; determines if there is a placebo effect. to detect a measurable difference in blood D. Give 50 people known perturber of blood pressure pressures over the unperturbed group. (positive control). 49 50 More realistic caffeine expt. Intrasystem negative control Alternate design, to control for differences A negative control within an experimental between people: system that provides a point of contrast A. Give 50 people nothing (to establish their to ensure unbiased measurement à baseline). measures “not-X”. B. Give same 50 people placebo (negative controls for the caffeine). Most controls are intrasystem-type C. Give same 50 people caffeine (treatment group; negative controls. amount determined by positive control). D. Give same 50 people known perturber of blood pressure (positive control). 51 52 Intersystem negative control An “unperturbed by X” negative control – ensures that the experimental system is not in itself perturbing the outcome. – e.g., scientist with white coat causing a higher bp reading (in subjects with white coat syndrome). – Use an automated bp cuff that a subject can perform their own measurements (no human contact involved). 53 54 9  11/17/24 Blinded Analysis Blinded Analysis, e.g., caffeine study Scientific bias is a potential perturbation The water and decaffeinated coffee groups: caused by the system itself, and thus an 1. Water and Caffeinated water groups allow the scientist intersystem negative control is needed. to test the effect of caffeine on a subjects without them being able distinguish which treatment they If the scientist/subject is blinded to when were getting (as long as the caffeine is tasteless). the caffeine treatment occurs, then biases are 2. Decaffeinated coffee as a placebo control for reduced. caffeinated coffee – i.e., both groups are drinking coffee but are blinded as to whether it has caffeine. – For the scientist, there is no preferential treatment of a particular data set. “Double blinded” experiment would mean neither the investigator nor the study subjects will – For the subject, the “influencing effect”, or placebo effect is reduced. know what is being drunk. 55 56 Summary: Not just the unperturbed Chapter 24 case, but also the “all but X” control. The negative control as more than the unperturbed The Requirement of the Positive Control. case. Positive control – experiment-specific The scientist must isolate distinct relevant parts of system validation control. the perturbation. Therefore, you need to keep doing additional The positive control provides a comparison negative controls until you have tested all the between a known, previously validated variables except “X”, where “X” is the only thing agent, and the perturbation, the you are querying in your experiment. “unperturbed” and “unperturbed by The fewer the negative controls in place, the more skeptical one should be about the X” cases captured by the negative controls. conclusions drawn. 57 58 Positive Control Positive Control The main purpose of the positive control is The positive control tells the scientist system validation within the experimental framework. whether the system is operating in the same manner during performance of e.g., What is the effect of caffeine on blood pressure? the actual experiment as it was when the – This question can only be answered in a setting system was validated. in which it has been shown that changes in Also tells - Whether if the experimental blood pressure are possible and results can be believed. measurable. – This is determined in the validation part of the experimental design. 59 60 10  11/17/24 e.g., Caffeine study, Expt 1. e.g., Expt 2, with a Positive control. If the caffeine study produced the following results: The positive control has been previously validated to increase blood pressure. Caffeinated water equivalent of 1 cup of coffee. The positive control shows that the One will conclude from the above results that caffeine had no additional effect on the experimental experimental methodology is subjects in comparison to water alone, because there functioning – an increase in bp occurred, was no difference between the negative control and this change can be measured. (water) and test (caffeinated water) groups. 61 62 e.g., Expt. 3, Dose-response. Dose-response Different test groups are administered increasing levels of caffeine. This experiment provided more information than the previous two experiments: – Statistical significance detected in caffeinated water (4 cups of coffee equivalent) and hypertensive drug compared to water. – Caffeine effect on bp is dose dependent. – Confirms the results from the previous experiments that showed no significant difference between water and caffeinated water. 63 64 e.g., Expt. 4, repeated on another Positive control in these expt. day. In expt. 3, caffeine in 4-cup coffee equivalent has a smaller increase in bp compared to hypertensive drug. The positive control provided a point of comparison to show: 1. that the system is operational, and 2. how the test agent (caffeine) compares to the positive control under these experimental settings. 65 66 11  11/17/24 Expt. 4 results e.g., Caffeine study Groups With this experiment, the scientist knows that A. No treatment there is a problem with the study, because the B. Water: four 8-oz cups/day positive control did not result in an increase in C. Decaffeinated coffee: four 8-oz cups/day blood pressure. D. Caffeinated water: four 8-oz cups/day; caffeine Scientist questions system breakdown – levels set to match those in the caffeinated equipment failure? Operator failure? coffee Therefore, the study needs to be redone. E. Caffeinated coffee: four 8-oz cups/day F. Caffeinated cola: four 8-oz cups/day; caffeine Without the positive control, there would levels set to match those in the caffeinated be no rationale to question the data. coffee 67 68 Can caffeinated cola serve as a positive control? NO! A positive control should show that the system is capable of detecting the experimental readout (in this case, an increase or decrease in bp). The hypertensive drug would prove that the system could measure a change in blood It should do so using a perturbation pressure. distinct from that used in the study, i.e., It would also provide a reference point to see something other than caffeine. how potent caffeine is in perturbing blood pressure. 69 70 Positive control to test multiple aspects of the experimental system. Why? Anything that helps to show that an Allow the investigator to omit subjects who, aspect of the experimental design is for whatever reason, were not drinking their operational can be thought of as a coffee or caffeine and were thus skewing the positive control. data inappropriately. e.g., It may be useful to prove that the To control other factors that may cause experimental subjects were actually differential absorption and metabolism of consuming, absorbing and metabolizing the caffeine. caffeine they were given. To normalize data. 71 72 12  11/17/24 Measuring blood/urine caffeine (or by-products). BRCA1 and Breast Cancer By measuring caffeine, or caffeine- metabolites, the scientist can be assured that the subject had in fact been exposed to the experimental perturbation. Measuring urine/blood concentrations of these metabolites is an exercise in system validation and can be considered a positive control – as the main purpose of the positive control is system validation within the experimental framework. 73 74 75 76 e.g., Does the deletion of the brca1 gene increase the incidence of breast Experimental Design – brca1 expt. cancer in mice? 20 mice per group Each of the groups are biopsied periodically, (Ch 23 and 24) from 2 wk to 2 yr after birth. “Conditional knockout” mice were At every time point, the breast cancer generated in which the brca1 gene was deleted incidence for each group is determined. only in breast tissue. Perturbation – brca1 mutation. Experimental groups: 1. brca1+/+ or “brca1-normal” mice (negative control) Positive control – gene/chemical that is 2. brca1+/- or “brca1-heterozygote” mice (partial already known to cause breast cancer to perturbation) ensure that the mice are 1) not abnormally 3. Brca1-/- or “brca1-knockout” mice (perturbation) resistant to cancer; 2) prove that the scientist 4. Positive control can detect breast cancer in biopsies. 77 78 13  11/17/24 Experimental Design – brca1 expt. EMS (ethylmethanesulfonate), a DNA alkylating agent, has been established to induce 25 % incidence of breast cancer 1 yr after treatment (trial experiment, or “inductive space”). – Positive Control 79 80 Results - brca1 expt. Results - brca1 expt. Age Normal brca1 +/- brca1 -/- EMS-treated (brca1 +/+) Deletion of brca1 in the breast tissue of mice 2 wk 0% 0% 0% 2% results in a significant increased incidence of 1 mo 2 0 1 5 breast cancer. 2 mo 2 3 2 8 Loss of one copy (heterozygote) does not 6 mo 2 3 2 15 significantly alter the incidence of cancer. 1 yr 2 3 25 25 Tumors do not significantly appear until 1 18 mo 2 3 30 40 year after birth (middle-aged mice). 2 yr 2 3 30 80 81 82 Results - brca1 expt. Results - brca1 expt. Positive control shows that the system “Threshold” effect of brca1 mutation; was working (cancer in EMS-treated mice). significant increase in tumors evident at 1 yr Also provides a context for the experimental (a jump from 2 % at 6 mo). treatments – both EMS-treated and “knockout” “Jump” may indicate the participation groups had 25 % incidence of breast cancer at 1 of another genetic event between 6 mo yr. and 1 yr that results in higher incidence of Linear increase in tumors over time in EMS- tumors. treated mice. More questions? Such questions become Scientist can spot changes in biopsies as early more obvious in the context of the positive as 1 mo, thus, “operator error” is not control. significant. 83 84 14  11/17/24 Summary YouTube Videos: The results of the positive control revalidate every aspect of the system, such as the Experimental Design – System Validation procedures, the equipment and the https://youtu.be/qK9fXYDs--8 technician doing the measurement, Experimental Design – Negative Control because the positive control shows that https://youtu.be/KXOvPVc6zeE all these components (i.e., the system) Experimental Design – Positive Control were operational. https://youtu.be/ZHTqxczEY6Y 85 86 15