chp 3-reading and writing LR [JZ].pdf

Finding Research Papers 23 all significant work is a critical part of doing research. It is typically the case, too, that the scope of literature that is relevant to you may only be obvious once you have completed the investigative phase of your research and have a good draft of your literature revi...

Finding Research Papers 23 all significant work is a critical part of doing research. It is typically the case, too, that the scope of literature that is relevant to you may only be obvious once you have completed the investigative phase of your research and have a good draft of your literature review. Your topic and interests are likely to shift, focus, or broaden during your research, and your perspective will change as your understanding develops— update your searching as you go. Searching and reading are separate activities, and it is a mistake to try and do both at once. I recommend that you uncritically gather material and then later critically analyze and categorize. Save the papers you find into a directory, and go through it later to understand what you have found. In the context of a single search session, it is also helpful to restrict your attention to one or two specific topics. Having explored the literature, you may discover that your original idea is not so original after all. If so, be honest—review your work to see what aspects may be novel, but don’t fool yourself into working on a problem that is already solved. Occasionally it happens, for example, that the same problem has been investigated by several other teams over a considerable period. At the same time, the fact that other people have worked on the same problem does not mean that it is impossible to make further contributions in the area. Critical Reading A key aim of reading is to develop critical thinking skills. Good researchers must demonstrate their ability to objectively analyze the work and claims of others. With experience, you can place each paper in a context of other work that you know, and assess it on a range of characteristics. In doing so, you will become alert to common mistakes and bogus claims. A challenge of research literature is whether to believe what you read. Work published in a reputable journal or conference is peer-reviewed; work available online could have any history, from being a prepublication version of an accepted journal paper to being plagiarised work poorly translated from a non-English original. A cynical but often accurate rule of thumb is that work that is more than one or two years old and has not been published in a significant venue probably has some serious defect. When you find a version of a paper on the Web, establish whether it has been published somewhere. Use evidence such as the quality of the author’s other publications to establish whether it is part of a serious program of research. Much research—far too much—is just misguided. People investigate problems that are already solved and well understood, or solve problems that technology has made irrelevant, or don’t realise that the proposed improvement actually makes the method worse. Mathematics may be pointless; the wrong property may be proved, such as complexity instead of correctness; assumptions may be implausible; evaluation strategies may not make sense. The data set used may be so tiny that the results are meaningless. Some results are just plain wrong. And, while the fact that a paper is 24 3 Reading and Reviewing refereed is an indicator that it is of value, it is not a guarantee. Many people undertake work that did not deserve to be written; sometimes it gets published. Indeed, few papers are perfect. They are a presentation of new work rather than a considered explanation of well-known results, and the constraints of writing to a deadline mean that mistakes are undiscovered and some issues unexplored. Some aspects of older papers may be superseded or irrelevant, or may rely on limited or technically outdated assumptions. A paper can be seen as a snapshot of a research program at a moment in time—what the researchers knew when they submitted. For all these reasons, a reader needs to be questioning, balanced, and skeptical. In short, don’t accept something as true just because it was published. But that does not justify researchers being dismissive of past work; rather, they should respect it and learn from it, because their own work is likely to have similar strengths and weaknesses. Some inexperienced researchers see other work as either perfect or poor, with nothing in-between. Usually, neither of these extremes is correct. While many papers may be flawed, they define scientific knowledge. (In contrast, textbooks are usually consolidations of older, established work that is no longer at the frontier.) If many researchers trust a particular paper, it is still reasonable to be skeptical of its results, but this needs to be balanced against the fact that, if skepticism is justified, these other researchers are all mistaken. Read papers by asking critical questions of them, such as: Is there a contribution? Is it significant? Is the contribution of interest? Are the results correct? Is the appropriate literature discussed? Does the methodology actually answer the initial question? Are the proposals and results critically analyzed? Are appropriate conclusions drawn from the results, or are there other possible interpretations? Are all the technical details correct? Are they sensible? Could the results be verified? Are there any serious ambiguities or inconsistencies? That is, actively attempt to identify the contributions and shortcomings rather than simply reading from one end to the other. This analysis of a paper can be thought of as verifying that each component is robust. If the paper is important to your work, you should analyze it until you have formed a reasoned opinion about each of its components; and if some components are questionable, this should be reflected in your literature review. Write down your analysis of the paper—you will read hundreds of papers, and in some cases will not formally describe them until months or years later. However, detailed analysis can be difficult before you have undertaken your own work, and in so doing developed a mature perspective. Thus reviewing of literature should not stop when the investigation begins, but continue alongside the research. Developing a Literature Review 25 Developing a Literature Review A literature review is a structured analysis of a body of literature, and may cover work from several separate areas of research. This review is not simply a list of these papers. Rather, the papers should be grouped by topic, and critically discussed in a way that allows the reader to understand their contribution to the field, their limitations, and the questions that they leave open. The task of writing a literature review for a paper can be challenging, and for a thesis can be more demanding than any other single activity. It therefore makes good sense to develop the review progressively. Begin a rough literature review as soon as you start reading, and, when you read a paper that you think will need to be discussed, add it in. (You should also capture the bibliographic data as you go, and also keep a copy of every paper you read.) Initially, the literature review will be sketchy and unstructured, but as you add papers you can group them by topic and contribution, and add notes on each paper and how they relate to each other. Briefly summarize each paper’s contribution and the evidence used to support the claims, and also note any shortcomings or features that are of interest. You might also want to note, for your own reference, how the work might have been done better: for example, are there obvious experiments that should have been tried, or plausible counter-arguments to the claims? Keep in mind that your understanding of other work helps examiners to judge your abilities as a researcher. There is no need for these drafts of your literature review to be polished—in all likelihood, no-one but you will ever read these early versions—so think of the writing as a letter to your future self. That is, at this stage you should focus on organization and content rather than on presentation. The rewriting, editing, and polishing that produces the final literature review will probably be done in a focused way only once your research is complete. As your review proceeds, it will become easier to decide whether to include each of the papers you read. Many obvious factors will guide your decisions: how close some other work is to yours, or how influential it has been. Some factors may be more subtle. For example, you may find a survey paper, or a recent paper with a thorough literature review of its own, that means that many of the older papers do not need to be discussed; some papers that initially seemed important might on reflection seem less relevant, and can be set aside or noted in passing; while a paper that at first seemed too theoretical or abstract may on further investigation be revealed as foundational. I suggest that in early drafts you be as inclusive as possible. When you do remove discussion of a paper, put the discussion in another file (or comment it out) rather than deleting it altogether, as this text is your record of having read the paper.