Evaluation of Research PDF

Evaluation of research Professor Hanan Ali Sayed Professor of public health 1 Introduction  Researchers need to have the skill to assess and evaluate research papers they read, particularly those related to research topic they...

Evaluation of research Professor Hanan Ali Sayed Professor of public health 1 Introduction  Researchers need to have the skill to assess and evaluate research papers they read, particularly those related to research topic they are doing  Evaluation of research should be done before the research is planned, during implementation of the project, and before discussing the results and preparing to communicate them.  Researches may critically evaluate all accessible published papers on a specific topic to make a systematic review 2 Continued:  Researchers may also be requested to peer-review a scientific paper submitted for publication by other researchers  Or assess the scientific output of candidates for academic posts.  Need to evaluate and assess research is not limited to researchers. It is lifelong part of professional development for health professionals. They need to critically assess the value of new published research before considering its practical application. They need to be aware that there are different levels of scientific evidence. 3  Policy makers should have the ability to assess research results and their implications for policy. They need to assess new technologies and currently used ones to introduce what is new and cost effective.  Also, to discard what is not effective and potentially harmful  Promote what is effective but under-utilized  And postpone a decision when evidence is still lacking 4  Research needs to be evaluated by investors of research to evaluate the return on their investment 5 Assessment and evaluation by researchers  The title of the paper and the abstract give an indication of the novelty and relevance of the paper.  The research question must be clear in informing the reader by its aims. Terms are clearly defined.  Give attention to errors in logic regarding causation, relationship or association. 6  For the critical reader, the methods section should be the first part of the paper to assess. It will tell whether it is good science or bad science.  A good methods section should provide a sufficient detail to allow other investigators to replicate the study and confirm the results. If does not, the study results cannot be easily accepted. 7  The most important methodological issues relate to how the sample is selected and what measurements were made. 8  The sample  Must be representative of the population studied and adequate sample size having sufficient power, not less than 80%.  Representativeness depends on selection and assignment. Random assignment is more advantageous that systematic one.  If two samples are compared, they must be selected to be identical for every relevant variable, except the one to be studied. 9  Research design should be appropriate to research question,  Taking into consideration resources of time, money and number of researchers. 10  The critical reader must question whether measurement used have been assessed for their validity and reliability  Validity is an index of how well a test or procedure measures what it is intended to measure.  Reliability assesses consistency of measurement. It relates to reproducibility of the test.  When reliability is high, a test that is repeated on the same patient and under the same conditions will yield the same result, whether by different investigators (inter-rater reliability), or by the same investigator (intra-rater reliability)  Criteria and criteria measures must demonstrate reliability and validity for both, the independent and dependent variable.  Where appropriate, the investigators should provide assurance about quality control of their data. 11  Critical reader of a scientific paper takes a close look at the results and their interpretation 12  Statistics how to be evaluated?.  Statistics is a common sense.  The first question by the reader is the authors have used any statistical methods at all? If not, no reason to accept that results are not being caused by chance alone.  The second question is whether the authors have selected the right statistical tests to analyze their data?  The third question is whether they have drawn the right conclusions from the statistical analysis?  Data analysis:  Tables and figures should be clearly labelled. 13 Evaluating articles (published research)  Purpose  Publication  Authors  bibliography 14  Publication:  Who published the article?  Scholarly peer-reviewed journals  When was the article published?  The date is relevant to those in the STEM disciplines. Research in science, technology, engineering, and mathematics moves quickly, so have the most recent articles is more useful.  Is the publisher affiliated with a reputable institution? 15  Authors:  Who wrote the article?  Good sources come from experts in the field (authority). Those employed in research institutions as universities or labs.  What else have the author/authors written?  More than one article in a topic in their field, support their reputation and be a great source for more articles.  Has the author been cited?  Citation is a great indicator to the effectiveness of the article. 16  Bibliography or references:  What is listed in the bibliography?  Trusted articles will have sources that are scholarly in nature and authorized by individuals with authority in their field.  Are the sources listed in the bibliography relevant?  Sources are up-to-date.  Should be recent  Adequate, cover items mentioned in introduction, methods (if required) and discussion 17 Other items for critical evaluation  Control of confounding variables  Discussion and conclusion  Ethics  references 18  Control of confounding variables: 19  Discussions and conclusions:  Does the study allow generalization  Limitations of the study to be mentioned  The discussion and conclusion should be consistent with the study’s results.  Ethics:  Are the ethical standards met? 20  Thank you 21