Evaluation of Scientific Medical Texts PDF

Evaluation of scientific medical texts “The class of questions” Evaluating Medical Literature Examining the results of a literature search can be performed at different levels of complexity. This may range from a superficial scan of the references to decide on relevance, to a detailed anal...

Evaluation of scientific medical texts “The class of questions” Evaluating Medical Literature Examining the results of a literature search can be performed at different levels of complexity. This may range from a superficial scan of the references to decide on relevance, to a detailed analysis of the quality of each article. The aim of this module is to present an overview of these levels of analysis. We’ll focus on general review points, followed by descriptions of how best to evaluate quantitative and qualitative research. Article Relevance Once you have performed your search and have accumulated a large number of articles to be evaluated; – Scan the list of references and remove duplicates – Assess the relevance of the material your search produced by reading the abstracts – If you cannot effectively do this by reading the abstract, then the next step is to collect original, full length copies of the articles Email the author Inter library loans The internet Quantitative Research The Majority of Medical Scientific Publications  Identifying types of research : quantitative Quantitative research attempts to generate Description: numerical, quantifiable results and to record – Quantative research attempts to quantify them against a study problem. and connect key variables These studies are generally carried out in a – The variables are manipulated in some form (called treatment) controlled environment – The results are measured and statistically Variables are manipulated in order to evaluate analysed their effects – The objective is to identify the cause- Statistical analysis is integral to the study type effect relationships Quantitative researchers create models and test them out in order to determine the main Strengths- Very scientific and reproducible, effects and interactions based on strong figures Weaknesses- Results can sometimes be artificial (they may reduce the relevance to the “real world”) or focus on too narrow an area Main components Title Methodology Abstract/ Summary – General Introduction – Outline of the investigation Summary of bibliography – Evidence Research question – Equipment – Types of Sources in – Common threats to Medical Literature validity Hypothesis – Procedures Summary of results Discussion & Conclusion References Literature Review: Sources of Medical Literature Primary Sources Secondary Sources Terciary/General Sources The main question in an investigation/hypothesis Is it clearly established? Is there a hypothesis? How well does it relate to the other components? Is it ethical to ask it? Methodology: General I Experimental – Most conlusive kind of research; random assignment of the subjects; manipulation of the variables is strictly controlled (e.g. Randomised controlled trials) Survey- There are two types: attitude surveys (likes and dislikes) or investigative scientific campaigns (which test out a hypothesis); A valid and reliable questionnaire is essential. (e.g. transversal or cross-sectional studies) Correlation – This kind of methodology tries to establish a relation between two or more variables. It helps in making intelligent predictions (e.g. case-control studies) Factorial studies – qualitative; introduces at least two independent variables, needs a broad sample Methodology: General I Meta-analysis – The researcher “calculates the averages of the results from select studies to get a complete pattern or relationship between the results” Causal-comparative – This type of study tries to determine the cause or consecuences of the differences between groups (e.g. cohort studies) Regression analysis – This study tries to predict something with regard to the existing data and knowledge Ethnographic Study – An investigation into the background and surroundings of a patient. The aim is to gather insight into how people live; what they do; how they use things; or what they need in their everyday or professional lives. Methodology: Research design Do the authors justify their decisions about design? Are difficulties analysed? Are the variables identified? – Dependent and independent. Are the external variables identified? Are the external variables controlled? Was the design appropriate? Methodology: Samples Is the population identified? Are the samples representative of this population? How were the samples collected? Do the techniques used compromise the results? Can the information be generalized to the proposed population? Methodology: Equipment ¿What equipment was used to collect the data? Was the choice of equipment justified? Were tests of reliability and validity supplied? Were the study limitations mentioned? Methodology: Common threats to validity I Selection bias - not selected at random, perhaps members of a particular group Mortality / Loss - Loss of volunteers from the study due to moving, promotion, retirement, death Location effect – different environments have different resources and distractions Equipment bias – Problems with the test instruments Maturation effect – any change that is created by the passing of time Methodology: Common threats to validity II Regression effect – Tendency for the data to move towards a mean average if the participants are specifically chosen for for their very high or low scores Hawthorne Effect – Changes simply because the subjects were assigned to a treatment group. It may be a negative change Unexpected interference –Events that were not anticipated/expected. Sprinklers activated in the middle of a study. Snowstorm closes the workplace in the middle of a project Methodological bias - Discovering an error in the way the study was conducted Methodology: Procedures Is the description of the procedures sound? Is there any threat to validity, from the procedures described? Are you able to identify new threats to validity? Are there any ethical problems with the procedures? Summary of results Do the authors present results without previous interpretation? Are the results directly linked to the question, hypothesis or problem? Does the author provide enough detail so that you can verify the results independently? Is the description sufficient for you to be able to explain the results in context? Discussion and conclusions What are the results? – Are they clearly established? Are the findings related to the study and the bibliographic review? Are there any weaknesses or limitations? Do the authors make note of any generalizations? Do the authors make recommendations for future studies? – Are they reasonable? References: Key questions Are the majority of the references used in the article primary sources? How many citations are offered? Are the cited references recent? Based on the information given, can you find articles for your own bibliographic review? Qualitative Investigation Comparisons and Contrasts with Quantitative Research  Identifying types of research : qualitative Qualitative research is often carried out outside of the Definition: laboratory, in the “real world”, or in an environment – The investigation is carried out in which is as natural as possible its natural environment It is often very descriptive – It is descriptive – Where was the research carried out? – What was that place like? – The investigator observes a – What size was it? specific situation – Who was there? – The objective is to increase – What was that person like? understanding, and identify – Who else was there? variables and new questions for – What else was there? future investigations The researcher “observes” the situation as discreetly as possible. Strengths – It shows things how they The main objective is to gain a greater understanding are of the subject of study with a final aim of gaining a deeper understanding. Weaknesses- It’s not always easy to evaluate the results Similarities to quantitative research Qualitative research: the how and why of human behaviour – It works with a wide range of models and theories, related to human phenomena – It involves small groups of participants, in order to concentrate on interpretation and reflection – It consists of discourse and texts, and their interpretation is very important – People’s opinions are important – It is not positivist: there are no objective truths; different interpretations; no definitive certainty of knowledge Similarities to quantitative research Articles for qualitative research have basically the same format as those for quantitative research. All articles contain – A title – An abstract or summary – An introduction – A bibliographic review – Discussion and Conclusions Results, discussion and conclusions: I The presentation and discussion of the results is probably the section you should pay most attention to when reading and evaluating a medical investigation article The results should be presented fairly and without prejudice The discussion should be related to the results and the two should address the initial hypotheses Each of the key findings should link to a conclusion and the global impact of the area of study should be discussed The work should be placed in context and possible errors or faults, whether they be experimental or ideological, should be evaluated Results, discussion and conclusions: II When evalutating the sections on results, discussion and conclusions, you should be critical without being unfair. You can use the following questions as a base to start your evaluation: – What were the results? – Are the individual results tied to the study of the specific objectives from the introduction? – Did the researcher form a hypothesis? – What conclusions did the researcher propose? – What consecuences can be deduced from the research? – Are there suggestions for future research? – What limitations have been mentioned? I suggest the following general questions about research work: Who were the authors? What methods did the authors – What is their standing and can use to analyse the data? they be trusted? – And were the methods used What question or questions did adequate? they want to answer? What did they find? – And what was the clinical – Did their conclusions relate to the importance of doing it? results? Who were the subjects of the Were there faults with the study? study? – How were they chosen, and were – Do the authors recognise them? the methods used the most What are the clinical appropriate ones? implications of their results? How was the data collected? – Was this the best method? What does it all mean? Bibliography – I – Girden, E. (2001). Evaluating Research Articles: From Start to Finish (2nd edition ed.). Thousand Oaks, London, New Delhi: Sage Publications. – Goubil-Gambrell, P. (1992). A Practitioner's Guide to Research Methods. Technical Communication: Journal of the Society for Technical Communication, 39(4), pp. 582-591. – Lunsford MSE CO, T. R., & Lunsford MS MAPT, B. R. (1996). How to Critically Read a Journal Research Article. Journal of Prosthetics and Orthotics, 8(1), pp. 24-31. Bibliography - II – Spyridakis, J. (1992). Conducting Research in Technical Communication: The Application of True Experimental Designs. Technical Communication: Journal of the Society for Technical Communication, 39(4), pp. 607-624. – Sullivan, P., & Spilka, R. (1992). Qualitative Research in Technical Communication: Issues of Value, Identity, and Use. Technical Communication: Journal of the Society for Technical Communication, 39(4), pp. 592-606. – Yaw, M. (2001). Notes from Fundamentals of Graduate Research in Education.