2024 BIOL391 Research - System Validation (Ch 17-21) PDF

11/5/24 Chapter 17 System Specificity. Specificity of Detection and Specificity of...

11/5/24 Chapter 17 System Specificity. Specificity of Detection and Specificity of System Validation Perturbation. Ch. 17 - 21, Glass (2014) Specificity/Selectivity - Can you measure/do the thing you want such that you can discriminate between X & Y, where X is BIOL 391 the thing you want to measure & Y are other Intro to Biological Research things which might interfere with you measuring X; X & Y are the signal to noise Burman, Fall 2024 represented by the positive & negative controls, respectively 1 2 e.g., Does caffeine cause an increase Specificity of Detection in blood pressure? To draw conclusions from the results of a It is necessary to be able to specifically particular experiment, it is necessary to measure caffeine, to be assured that caffeine determine whether the method of and not some other chemical is being given detection being used is capable of to the subject. identifying something in a “specific” – Use a well established manner. and published assay The specificity of the detection method for caffeine. must be validated. – Assay may be limited by available lab equipment. 3 4 Also determine if caffeine can be measured in e.g., Antibodies to detect a specific a sample obtained from the experimental protein. subject. – To confirm that the subject ingested the agent Specificity of detection – whether an antibody under study. binds only the protein or epitope of interest. – To relate the amount measured in the subject’s system to the biological effect of the drug. – Binding could be specific for protein in question, If the success or failure of the experiment nonspecific, or specific for many proteins with the absolutely depends on a particular substance same epitope/common domain. being present, it is mandatory that the scientist Epitope - also known as antigenic be able to measure that substance. determinant, is the part of an antigen that is Specificity is a critical element in system validation that must be determined in recognized by the immune system, specifically advance of the actual experiment. by antibodies, B cells, or T cells. 5 6 1  11/5/24 Common protein domains. Because proteins commonly share domains, the scientist must be concerned about possible antibody cross-reactivity. While the antibody is specific in its ability to detect the presence of a particular epitope or domain of a protein, cross-reactivity causes the antibody to be not specific for the single protein that the scientist seeks to study. Therefore, to detect one specific protein, the domain structure of the protein should be known, and the antibody raised should avoid regions of similarity. 7 8 While polyclonal antibodies are useful reagents, monoclonal antibodies are usually more specific for protein domains/epitopes, and to differentiate closely related proteins. 9 10 11 12 2  11/5/24 Specific antibody reagent. Antibody validation An adjuvant (from Latin, adjuvare: to aid) A common form of antibody validation, is an agent that is added to the antigen to especially in a complex experimental system boost the immune system such as to produce where there are thousands of proteins, is to a higher amount of antibodies. perform a negative (or “null”) control – Some adjuvants are proteins, like bovine serum experiment where the sample does not albumin (BSA). contain the particular protein of interest. – Some of the antibodies produced (with – i.e., if your protein is a liver-specific protein, the adjuvants) can cause nonspecific detection of negative control experiment will demonstrate a proteins because they are cross-reacting to adjuvant components. negative antibody-binding result in another tissue, like the kidney. 13 14 Antibody validation Alternately, if purified protein of interest is available, it can be used to determine whether the antibody can detect the purified protein. – However, the experiment does not prove if the antibody is specific, but as a first step, it proves that the antibody is competent to detect the isolated protein that it was raised against. 15 16 Specificity of Perturbation Examples of Perturbations Perturbation: a change, or disturbance, in 1. Chemical/drug inhibiting enzyme or a protein’s normal activity. the normal state. – Scientist must validate that chemical/drug does Perturbations should result in an specifically what the scientist intents it to do and nothing else. effect/observation. Therefore, you need to – i.e., no “off-target” effects. – Some may be specific at a certain range of be confident that the perturbation is causing concentrations but have “off-target” effects at higher the effect. concentrations. 2. Likewise, activating effects of chemicals/drugs Therefore, perturbations also need to be have to be validated. They may have similar validated. characteristics as inactivating chemicals. 17 18 3  11/5/24 Examples of Perturbations Examples of Perturbations 3. Consistency in action of independent molecules, each known to act on the same molecule studied, indicates that the perturbation of that molecule is specific. – e.g., siRNA molecules to different regions of mRNA showing the same inhibitory effect indicates that destruction of specific mRNA is validated. 4. In a gene “knockout” experiment, one must be certain that only the gene of interest is deleted and not a second gene, or control elements (enhancer/silencer) of another gene, encoded within the same region is affected. 19 20 Examples of Perturbations Summary 5. Changing experimental parameters, e.g., pH, [ ]s, temperatures, could have Validity of a reagent in its ability to unintended consequences. either specifically detect or perturb – If there is a need for change, only one something in the experimental parameter is varied at a time. system. 6. Perturbations in protein structure - Strategy for maximizing specificity functions (e.g., mutations, truncations, and validating that the result is in domain deletions, etc.) must be confirmed fact as selective as desired or needed. for those structural perturbations. 21 22 Chapter 18 Antibody/Reagent Sensitivity System Sensitivity Ensure antibody specificity: – Making sure that the antibody (or reagent) can Minimizing Signal to Noise to Improve detect the substance of interest with a purified Sensitivity of Detection. extract, or overexpressed cell/tissue lysate. Sensitivity - How much of the thing can – Detection in a normal lysate and no detection in you measure? To what degree of difference a “null” lysate (or negative control lysate). can you measure it? – Inactivate mRNA translation using siRNA (i.e., RNA interference). – Sensitivity is defined as the ability of a reagent or method to detect a substance of Antibodies tend to suffer from batch interest at the concentration at which it exits variability, i.e., some antibody in the experimental system. preparations are more sensitive than others. 23 24 4  11/5/24 Antibody/Reagent Sensitivity Antibody/Reagent Sensitivity Determine sensitivity by using a dilution If physiological/cellular/tissue assay where the protein is diluted serially, concentrations are not known, sensitivity 10-fold each time, and detection is validation is more difficult. performed. – Purify protein of interest – however, original This determines the minimum protein [protein] cannot be determined (i.e., not quantitative); only allow to know if protein is detectable, hence sensitivity of the antibody. present (qualitative). Antibody/reagent should be able to – Use more sensitive assays, like Western blot detect known physiological/cellular/ and ELISA (more sensitive then Western). tissue concentrations of this protein/substance. 25 26 e.g., Low-level expressed genes. Low-level expressed and low-copy number gene products are particularly difficult to detect with “normal” protein chemistry. e.g., hormone receptors and transcription factors. If Northern blots, or DNA microarrays, are not able to detect gene expression, more sensitive techniques are used to enhance the sensitivity. – RT-PCR-coupled DNA microarrays amplify rare mRNA sequences (only qualitative usually). 27 28 e.g., Detecting controlled substances. e.g., Early tumor/virus detection. Performance enhancing drugs in athletes. Early detection can save lives! – [Low] as drugs were taken long time ago and have An x-ray can provide the detection of a mass exited the body. after sufficient tumor cells have – Masking agents that may be present to help avoid accumulated. Fluorescent-labeled antibodies detection. can detect just a few tumor cells in a tissue – Tests have to be sensitive as well as specific. and thus more sensitive than x-rays. Drug tests for employees need to be rapid and PCR-based assays can be used to detect sensitive to detect controlled substances, e.g. viruses (traditionally detected by ELISA) transportation operators like pilots, bus drivers, during the earliest stages of infection. etc. 29 30 5  11/5/24 e.g., detecting colocation, or Sensitivities in different conditions. interacting proteins. Two proteins that are colocated together in a Assay sensitivities may be affected by distinct cellular/tissue location might likely experimental conditions like pH and work together or physically interact with each temperatures. other. In this cases, the assay is performed to determine the pH or temperature optimums, Techniques that detect molecular location and even when these parameters fall outside of measures cellular distance must be sensitive. normal cellular conditions if sensitivity is – Flourescent antibodies needed. – Precipitating antibodies that can copurify the other If sensitivity and specificity is both important, interacting protein (when 2 proteins are physically then physiological conditions of pH and interacting with each other). temperature are maintained. 31 32 Sensitivity in measuring a change. Summary Detecting changes is critical to understanding the effect caused by the The sensitivity of a system must be perturbation. validated to measure substances and Validating that these measurements can be effects as they occur under made reproducibly and dependably requires experimental conditions. using standards and controls to confirm that the measuring devices are sensitive The goal is to increase the signal over enough. the noise measurements (i.e., increase Natural variations may increase noise. signal : noise ratio). 33 34 Chapter 19 System Stability Fidelity/Stability - Can you measure the thing consistently? Do you get the same answer under the same conditions Minimizing variability. 35 36 6  11/5/24 Tissue-Culture Examples Primary Culture – Cells that have been harvested directly from an animal or human subject. – Not immortal and usually have a limited number of replications before they become senescent. – Therefore, need to determine maximum number of times these cells can replicate before they senesce. – Timing of experiment to ensure the “age” of these primary cultures are the same to be predictive of the experimental outcomes. 37 38 Tissue-Culture Examples Animal or Human Subjects Culture conditions – Maintain similar culture conditions to assure Stresses, changes in activity, diet, drugs, or cells undergo predictable differentiation. other substance intake can perturb these – If culture conditions are not met and maintained, subjects, which can change their reactions to some cell lines can lose their competence to the study. differentiate. Cell contamination System must be validated to determine how – Since animal cells grow much slower, bacterial reliable the subject will be a component contamination may result in different experimental of the system and the parameters that can outcomes compared to the effect of the same degrade this reliability; so that these can be perturbation in uncontaminated cells. tested for and system degradation be – e.g., system degradation by some external prevented. event, rendering the system unstable. 39 40 Animal or Human Subjects Animal or Human Subjects Yeast and Bacterial Strains: Mouse colonies – with distinct mutations: – Strains must be validated and genotyped periodically. – Screening for viral and bacterial infections. – If instability arises, one may have to begin anew from a parental stain. – Operator mistakes that lead to mouse lines Growth Factors: mixed up (genotype of mouse line must be – Protein growth factors are susceptible to denaturation validated at regular intervals). from repeated freeze-thaw cycles and proteases present in cellular extracts. – System must be revalidated when new – When their shape is destroyed, denatured proteins information about mouse line is obtained. become nonfunctional. – Storage: Cryoprotectants are added to minimize ice – Similar to other cases of system instability. crystal formation; samples stored in small aliquots so that proteins are not thawed more than once or twice; protease inhibitors are added. – Test protein function tested regularly. 41 42 7  11/5/24 Animal or Human Subjects Reagents/Chemicals: – Critical reagents are tested for stability (i.e., degradation). – Directions to maintain their stability adhered to. – e.g., drugs loosing competency, autolysis, light sensitivity, pH induced changes. RNA Stability: – mRNA is notoriously unstable because RNase is ubiquitous; only a small amount is needed. – RNase is present in body secretions, and is quite stable. – Rnase-free environments are necessary, e.g., certified Rnase-free pipette tips, microfuge tubes, etc. 43 44 Equipment Stability Summary Mechanical equipment that is part of the System validation is an essential part of experiment must be maintained and experimental design and must be validated for stability. performed both before an experiment Micropipettes – checked, restandardized and maintained frequently. begins and regularly while it is in Microscopes – cleaned, lenses and mirrors progress. aligned, light sources calibrated. Stability also involves the consistency of a pH meter – validated with control/standard human observer – if a scientist changes pH soultions frequently. his or her criteria for judging an effect, this Centrifuges – maintained to prevent damage to expensive machines. will obviously perturb the report of an experimental readout. 45 46 Chapter 20 Dose Response System Efficacy For any perturbation reagent/vector, the standard way to determine how much to administer is to Determining Conditions to Measure perform a dose response, in which increasing Efficacy concentrations of the reagent are given to the cell or animal, until evidence that the pathway or Efficacy – How often and what target that the scientist desires to activate/inhibit is concentrations of reagents is required in fact perturbed as intended. in order to induce the desired effect (or The effective dose (ED50) to be used in the final experiment is usually several orders of magnitude the ability to produce a desired or smaller than toxic side effects are seen (i.e., TD50). intended result). 47 48 8  11/5/24 e.g., Insulin-like growth factor 1 (IGF-1) IGF-1 is investigated in its role in signaling apoptosis. It is known that IGF-1 acts by stimulating a pathway involving a protein kinase called Akt. Akt is phosphorylated when it is activated. Akt-P has many downstream effects like apoptosis, cell cyle control, protein synthesis, etc. 49 50 Insulin-like growth factor 1 (IGF-1) e.g., Insulin-like growth factor 1 (IGF-1) Signaling Pathway A dose-response curve of IGF-1 at 0 to 100 ng/ml was tested for peak phosphorylation of Akt. Peak Akt-P occurred with 10 ng/ml (Fig b, next slide). Therefore, from this dose response, the scientist knows that 10ng/ml is sufficient to induce a maximal Akt response. However, it is noted that maximal apoptosis activation, indicated by Caspase 3 activity, is maximum at a much lower value of IGF-1 (0.1 ng/ml; Fig. a, next slide). Therefore, much less IGF-1 is needed for this effect. 51 52 e.g., Rapamycin Rapamycin is a drug that can block mTOR, a molecule in the signaling pathway down stream from IGF-1-induced Akt activation. mTOR is capable of stimulating protein synthesis, in part by activating the enzyme S6K1 and various other proteins. To see the degree to which mTOR inhibition can block S6K1 synthesis, a dose response of rapamycin is done. It is noted that different amounts of rapamycin is needed to inhibit protein synthesis of a specific protein, i.e., less for S6K1 but more for survivin, etc. 53 54 9  11/5/24 Insulin-like growth factor 1 (IGF-1) Signaling Pathway 55 56 “Physiologic Range” e.g., Dose response with siRNA With a IGF-1 being present naturally, it is also If exogenous perturbations are not desired, important to not exceed the physiologic range of concentrations that is found naturally in the then “knocking” out the gene like mTOR cell/tissue as [IGF-1] > physiologic range could result is possible with siRNA technology. in toxic effects that are not normally observed under normal situations. This provides an alternate method to inhibit If maximal activation of a protein is desired, then []s the mTOR pathway (if there is questions higher than the physiologic range can be used. The about specificity of rapamycin). dose response provides these limits before harmful, or not natural, effects are seen. One can then quantitate the amount of However, with rapamycin, since it is an artificial drug mTOR mRNA present after a dose response administered, the dose response is absolutely useful experiment with different [siRNA]. for determining a useful dose to use. 57 58 Frequency of Administration A time course is performed if the scientist must answer how often to give the agent to maintain the desired effect. e.g., IGF-1 induction of Akt phosphorylation indicate a dose of IGF-1 can sustain an increase of Akt-P for up to 3 hours before a decrease is noted. 59 60 10  11/5/24 Pharmacokinetics (PK) Pharmacodynamics (PD) When administering a perturbing agent to an animal or a human, additional factors needs to be The data obtained in the setting of the animal considered – pharmacokinetics. or human experiment after the addition of a Pharmacokinetics – defined as the study of the perturbing agent is called pharmacodynamics. time course of drug absorption, distribution, Pharmacodynamics refers to the metabolism, and excretion. relationship between drug concentration A drug that is rapidly cleared will need a larger dose at the site of action and the resulting or more frequent dosing. effect, including the time course and intensity Mode of administration (oral, subcutaneous, of therapeutic and adverse effects. intravenous, or intraperitoneal) can have distinct PK – Determine how much perturbing agent needed to profiles, thus different peak levels, and a particular reach the target tissue and have the desired effect. time course over which the drug might achieve – Time course to determine how often the agent concentrations range sufficient to have the desired needs to be administered to maintain that effect. effect. 61 62 PK vs PD Summary Dose-response and time course experiments are performed to determine the ideal concentration of a perturbing agent and frequency of administration, respectively. In vivo experiments are supported by PK/PD studies to determine the method of Pharmacokinetics is the study of what the administration of the perturbing reagent as well body does to the drug, and as the required concentration and frequency of Pharmacodynamics is the study of what the the drug or perturbing agent being used. drug does to the body. 63 64 Chapter 21 System Safety Determining Conditions to Measure Safety Safety – Safety of the investigator, animal or human subject, and the wider community. 65 66 11  11/5/24 Safety of the Investigator Safety of the Investigator The investigator needs to be familiar with Handling chemicals/reagents: the hazards and dangers associated – Fume hood for chemicals with inhalation risk. with the chemicals and equipment that – Explosion risk. are components of the experimental system. Training for handling, monitoring and Proper safety precautions must be disposal. understood and followed every time when – Radioactivity safety certification, radioactivity handling chemicals/reagents: personnel-badge monitoring, laboratory – MSDS (Material Safety and Data Sheet). decontamination procedures and radioactive transit records (i.e., records for arrival, storage – Gloves for handling chemicals disolved in and disposal). DMSO that can diffuse through the skin. 67 68 Safety of the Investigator Safety to the Animal or Human Subject Chemical exposure risk is not a one-time Ethics boards govern the use of experimental event – i.e., small risks can add up. material. Safety equipment and training of – “Is the exposure of the experimental equipment usage. conditions/chemicals necessary to get your answer?” – Safety goggles (UV, etc), lab coats and other – One needs to weigh the harm versus the “splash” guards. benefit accrued from achieving an answer to the question. – Safety showers, eyewash and first-aid stations. Other issues: providing appropriate anesthesia – Electrocution risk. to the animal or choosing a lower organism (when it can provide an answer to a question) instead of humans or primate. 69 70 Safety to the Wider Community Summary Genetic modification that confer a selective Safety issues must be considered in advantage to an organism pose a risk by advance of experimentation. outcompeting the unmodified organism. – Precautions taken if used in field tests. Experimentation in human pathogens can carry potential safety risk, not just to the scientist engaged in the project, but also to the surrounding community. 71 72 12