Inquire Biology: A Textbook That Answers Questions PDF

Articles Inquire Biology: A Textbook that Answers Questions Vinay K. Chaudhri, Britte Haugan...

Articles Inquire Biology: A Textbook that Answers Questions Vinay K. Chaudhri, Britte Haugan Cheng, Adam Overholtzer, Jeremy Roschelle, Aaron Spaulding, Peter Clark, Mark Greaves, Dave Gunning n Inquire Biology is a prototype of a earning a scientific discipline such as biology is a daunt- new kind of intelligent textbook — one that answers students’ questions, engages their interest, and improves their understanding. Inquire Biology L ing challenge. In a typical advanced high school or introductory college biology course, a student is expect- ed to learn about 5000 concepts and several hundred thou- sand new relationships among them.1 Science textbooks are provides unique capabilities through a difficult to read and yet there are few alternative resources for knowledge representation that captures conceptual knowledge from the textbook study. Despite the great need for science graduates, too few and uses inference procedures to answer students are willing to study science and many drop out students’ questions. Students ask ques- without completing their degrees. New approaches are need- tions by typing free-form natural lan- ed to provide students with a more usable and useful resource guage queries or by selecting passages of and to accelerate their learning. text. The system then attempts to The goal of the Inquire Biology textbook is to provide better answer the question and also generates learning experiences to students, especially those students suggested questions related to the query who hesitate to ask questions.2 We wish to create an engag- or selection. The questions supported by ing learning experience for students so that more students the system were chosen to be education- ally useful, for example: what is the can succeed — and specifically to engage students in more structure of X? compare X and Y? how actively processing the large number of concepts and rela- does X relate to Y? In user studies, stu- tionships. Inquire Biology aims to achieve this by interactive dents found this question-answering features focused on the relationships among concepts, capability to be extremely useful while because the process of making sense of scientific concepts is reading and while doing problem solv- strongly related to the process of understanding relationships ing. In an initial controlled experiment, among concepts (National Research Council 1999). To community college students using the encourage students’ engagement in active reading, our peda- Inquire Biology prototype outperformed students using either a hard copy or con- gogical approach is to help a student to articulate questions ventional ebook version of the same about relationships among concepts and to support them in biology textbook. While additional finding the answers. research is needed to fully develop Inquire Biology incorporates multiple technologies from the Inquire Biology, the initial prototype field of artificial intelligence. It includes a formal knowledge clearly demonstrates the promise of representation of the content of the textbook, reasoning applying knowledge representation and methods for answering questions, natural language process- question-answering technology to elec- ing to understand a user’s questions, and natural language tronic textbooks. generation to produce answers. It is based on a systematic knowledge-acquisition process that educators can use to rep- resent the textbook’s knowledge in a way that the computer can reason with to answer and suggest questions. A unique Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602 FALL 2013 55 Articles aspect of our approach is the use of human-comput- The Campbell Biology textbook contains the origi- er interaction design methods to create a product nal content of the textbook as published by Pearson that combines these advanced AI technologies into a Education. The textbook content is hyperlinked to user experience that is compelling for students. the concept summary pages. These pages summarize Inquire Biology is one of the products of Project Halo,3 salient facts and relationships that a student needs to a long-term research effort to develop a capability to know about a concept. Summary pages combine con- answer questions on a wide variety of science topics. tent from the glossary entries from the back of the Inquire Biology is based on Campbell Biology (Reece book with automatically generated descriptions of et al. 2011), which is a widely used textbook in salient properties and relationships for each of the advanced high school and introductory college 5000 plus concepts in our knowledge base. These courses. Campbell Biology is published by Pearson pages also include links to related concepts, relevant Education, and we are using it under a research-use passages from the book, and follow-up questions use- license. Using a popular textbook as the basis for our ful for further exploration. research ensures that the content is pedagogically A student can ask questions of Inquire Biology in sound and the results have immediate applicability three ways. First, a question may be typed into a free- to a large group of students already using the text- form dialogue box, from which Inquire Biology com- book. The basic concepts in the design of Inquire Biol- putes the closest questions that it can answer and ogy are, however, broadly applicable to other text- gives the user a choice for selecting the best match books and scientific disciplines. among them. Second, in response to the student Exploration of the application of AI technology to highlighting in the text, Inquire Biology computes the an electronic textbook comes at an opportune time most relevant questions it can answer for the selec- when we are witnessing a large-scale transition from tion. Finally, Inquire Biology suggests questions for fur- paper to electronic textbooks. Textbooks made avail- ther exploration on the concept summary pages and able in electronic medium offer tremendous oppor- with each page that contains an answer. We next tunities to leverage a variety of AI techniques much illustrate how students interact with these features of broader than what we have considered in our work so Inquire Biology. far. Textbooks made available in electronic medium offer tremendous opportunities to leverage a variety Textbook of AI techniques much broader than what we have Figure 1 shows the textbook interface of Inquire Biol- considered in our work so far, for example, recom- ogy. Like most electronic textbooks, Inquire Biology mending books (Pera and Ng 2012) and enhancing supports highlighting the text, taking notes in the electronic textbooks with content from online margin, and interacting with graphics. Inquire Biolo- resources such as images and videos (Agrawal et. al. gy leverages its knowledge base to expand on these 2011). Our focus on the use of knowledge represen- standard features in the following three ways: (1) The tation leverages the synergy between the need for toolbar provides a table of contents, index to concept precise knowledge content in an electronic textbook summaries, and navigation history. Students can ask and the strength of knowledge representation and a question at any time by tapping the Q icon. (2) reasoning to facilitate that. Within the text, biology terms are automatically We begin this article with a description of Inquire linked — students can tap to see a quick pop-up defi- Biology, introducing its key features and describing nition or to navigate to the full concept summary. (3) how a student might use Inquire Biology for active Students can highlight with a quick and easy gesture, reading and homework. We give an overview of the and each highlight serves as the anchor for a note AI technology used in Inquire Biology and then pres- card and a list of related questions, encouraging stu- ent the results of a day-long pilot experiment show- dents to dig deeper into the material. ing that the students studying from Inquire Biology received approximately 10 percent higher grades on Concept Summary Page homework and posttest problems as compared to stu- For every biology term in the book, Inquire Biology dents studying from the paper or electronic version presents a detailed concept summary page. Each of of the textbook. We discuss the generalizability and these pages begins with a human-authored definition scalability of the approach and conclude the paper of a concept (marked as 1 in figure 2). Most of these with a discussion on related work and directions for definitions are already available at the back of Camp- future work. bell Biology, but some were added as part of the knowledge base (KB). The text definition is followed by key facts and relationships about that concept Features of Inquire Biology (marked as 2 in figure 2). These are automatically Inquire Biology connects three kinds of information to generated from the KB. Since the KB is specific to meet student needs: the Inquire Biology textbook, con- Campbell Biology, the concept summary never has the cept summary pages, and computer-generated sort of unnecessary details students might find on answers to questions. Wikipedia or other general-purpose sites. 56 AI MAGAZINE  Articles 1 3 2 Figure 1. Textbook Interface. The textbook content is from page 132 of Biology (9th edition) by Neil A. Campbell and Jane B. Reece; Copyright © 2011 by Pearson Edu- cation, Inc. Used with permission of Pearson Education, Inc. The sidebar of a concept summary page contains help illustrate the types of questions Inquire Biology the figures from the textbook that are most relevant can answer. to that topic (marked as 3 on figure 2). Finally, ques- The question-asking interface, shown in figure 3, tions relevant for further exploration of the topic are is invoked by tapping the Q icon. The student can listed (marked as 4 in figure 2). Knowledge in a con- either enter a free-form question directly, or enter a cept summary page can span multiple chapters, but list of biology terms. Inquire Biology will suggest ques- the concept summaries put all the relevant facts and tions related to free-form questions as the student figures in one place. Unlike most glossaries, the types, helping students formulate their questions and Inquire Biology concept summary is not a dead end offering alternatives for cases where Inquire Biology but can act as a catalyst to further learning — pop-up cannot understand the free-form question directly. definitions and follow-up questions help students These suggested questions are in three main cate- explore relationships between concepts and dig deep- gories: definitions and reference information, com- er into the material. parisons between two concepts, and questions that explore relationships, such as how structure affects Asking Questions function. These three types provide the best overlap Students can ask questions at any time, but they may between the system’s capabilities and the types of not always know what to ask. Inquire Biology is proac- questions students find useful. tive, suggesting questions in a variety of contexts to In response to each question, Inquire Biology returns FALL 2013 57 Articles 1 3 2 4 Figure 2. Concept Summary Page for Plasma Membrane. This figure contains figure 6.6 from Biology (9th edition) by Neil A. Campbell and Jane B. Reece; Copyright © 2011 by Pearson Education, Inc. Used with permission of Pearson Education, Inc. a detailed answer with context to help students focus from what might exist in the underlying KB, with on what is important. Answers are designed to be con- relations designed to be clearly understandable to a cise and understandable, which means that different biology student. Labels alone may not be enough — answers require different presentations. brief definitions of each term appear next to the Comparison questions compute the key differ- graph, plus the links to follow-up questions. ences between two concepts, building on the learn- For more difficult problems, our goal is to support ing principle of conceptual contrast (Ausbel 1965). student investigations. In accordance with the learn- The result is displayed in a table, keeping the infor- ing principle of scaffolding (Bliss, Askew, and Macrae mation well organized. Context is provided with 1996), Inquire Biology links to related concepts and short definitions and superclass information at the suggests follow-up questions, helping students ask top, followed by important differences near the top. the right questions and work their way toward a solu- Figure 4 shows the answer to a comparison question tion. By providing such scaffolding, students can about chloroplasts and mitochondria. make progress on more difficult investigations than Answers to relationship questions are rendered as they could handle unaided. In the current system, graphs (figure 5), as these best highlight the connec- such scaffolding is always available to them. Further, tions among concepts, building on the learning prin- by providing ready access to basic facts, Inquire Biolo- ciple of helping students make sense of concepts by gy may reduce cognitive load (Sweller 1988) associat- exploring relationships among concepts (Scar- ed with information retrieval and allow students to damalia and Bereiter 2006). The graph is simplified stay focused on higher-order learning objectives. 58 AI MAGAZINE  Articles ask a question define Define cellular respiration structure What is the structure of a chloroplast? function What is the function of a plasma membrane in a eukaryotic cell? compare What are the differences between chloroplasts and mitochondria? relate If the chloroplasts were removed from a plant, what events would be affected? search Search book for photosynthesis Figure 3. Asking Questions with Assistance. Concept of Use lowing is a scenario to illustrate how a student might engage in active reading while using Inquire Biology: Inquire Biology is designed to support two education- I’m reading section 10.2 on photosynthesis. I have al use cases: active reading and homework support. It already highlighted some sentences in this section, is certainly possible to use Inquire Biology in a class- and I encounter a term that I do not understand: thy- room and for exam preparation, but we did not lakoid membrane. I tap on this term to see more infor- investigate such use in our work. Here, we illustrate mation about it and I am taken to a concept summa- how a student might use its capabilities in the con- ry page. I am first shown the definition: thylakoid text of active reading and homework support. membrane is the membrane surrounding the thy- lakoid. This definition is the same as the definition in Active Reading the paper version of the textbook and not particular- Active reading is reading with a purpose, usually to ly useful. It is followed by additional information that is not directly available in the glossary of my paper understand a concept or to answer a question. Active textbook: that a thylakoid membrane is a type of dou- reading has a strong track record in empirical learning ble membrane. Further, I can see the parts that are research (Palinscar and Brown 1984). As a learning unique to it: the electron transport chain and ATP strategy, it consists of four activities that take place synthase. I can tap on electron transport chain and before, during, and after reading a text: predict, ask, view detailed information about its parts such as connect, and explain. The predict activity encourages cytochromes. From this summary page, I can also see students to look ahead before reading a passage. For that the functions of a thylakoid membrane are light example, students may skim a section, or look at reaction and photophosphorylation. But, what about headings, figures, and captions for organizational the functions of its parts? I can ask a question: What cues. The student combines this preview with prior does an electron transport chain reaction do? I get an knowledge to form predictions of how the topic fits answer: noncyclic electron flow. As I return to the into the larger themes of the domain and to antici- textbook, I can see that it is the next section that I am about to read. pate the content that will follow. In the ask phase, stu- dents make questions to reflect on what they are In this example, Inquire Biology has supported learning and are encouraged to create practice test active reading in the following ways: questions. In the connect phase, students relate the Connect: The pop-up definitions and concept sum- new content to related knowledge, including person- mary pages help students connect to additional al experience, current events, prior knowledge, and material and allow a quick and seamless way to concepts in other subjects. In the explain phase, stu- refresh and review previous knowledge. In this dents restate the read content in their own words, example, the student was able to review the structure through notes, summarizations, or diagrams, such as and function of thylakoid membrane, and also tables, flow charts, outlines, or concept maps. Fol- obtained more detailed information about some of FALL 2013 59 Articles Figure 4. Comparison Question. This figure contains figures 10.17, 6.16, 10.8, and figure 6.18 from Biology (9th edition) by Neil A. Campbell and Jane B. Reece. Copyright © 2011 by Pearson Education, Inc. Used with permission of Pearson Education, Inc. its parts — for example, electron transport chain. student makes a highlight. Related questions appear Explain: Inquire Biology’s highlighting and note-tak- on concept summary pages and answers, and stu- ing capabilities allow students to mark passages they dents can tap to see Inquire Biology’s answer. In addi- find particularly important and to extend and sum- tion Inquire Biology allows the student to ask free-form marize the information in their own words. Inquire questions. In this example, the student asked a ques- Biology also explains the basics of a concept — for tion about the function of an electron transport example, in this case it points out that a thylakoid chain, deepening the understanding of the overall membrane is a double membrane. concept of a thylakoid membrane. Ask: Inquire Biology suggests questions whenever a Predict: Students can attempt to answer Inquire Biol- 60 AI MAGAZINE  Articles Figure 5. Relationship Question. This figure contains figures 5.3, 5.5, 5.6 and 5.7 from Biology (9th edition) by Neil A. Campbell and Jane B. Reece. Copyright © 2011 by Pearson Education, Inc. Used with permission of Pearson Education, Inc. ogy’s suggested questions on their own and test their constructing answers for complex conceptual home- predictions by tapping through to see the generated work problems. Inquire Biology generally cannot answers. In this example, the answer to a student’s answer complex homework questions directly, but question about the function of an electron transport provides support in three ways: (1) reducing the chain leads to a preview of what is coming up next in emphasis on memorization by providing quick the textbook, appropriately setting up the student’s access to facts and relationships needed to solve a learning goals. homework problem, (2) comparing and relating con- cepts that may not necessarily be discussed in the Homework Support same section of a textbook, and (3) deconstructing Inquire Biology assists students in understanding and problems by suggesting simpler questions. We illus- FALL 2013 61 Articles CONCEPT CHECK 6.5 1. Describe two common characteristics of chloroplasts and mitochondria. Consider both function and membrane structure. 2. Do plant cells have mitochondria? Explain. 3. A classmate proposes that mitochondria and chloroplasts should be classified in the endomembrane system. argue against the proposal. Figure 6. An Example Homework Problem. Concept check 6.5 is from Biology (9th edition) by Neil A. Campbell and Jane B. Reece. Copyright (c) 2011 by Pearson Educa- tion, Inc. Used with permission of Pearson Education, Inc. trate this capability by considering a homework The third question is open ended, and Inquire Biology problem from section 6.5 of Campbell Biology (figure helps me in deconstructing this question into simpler 6). The scenario below suggests how a student might questions. When I highlight this question in Inquire use Inquire Biology to understand and answer this Biology, it suggests several questions some of which problem. are: What is an endomembrane system? What are ele- To answer the first question, I ask Inquire Biology: What ments/members of an endomembrane system? What is the similarity between a chloroplast and a mito- are the differences between a mitochondrion and chondrion? Inquire Biology handles this question endoplasmic reticulum? What event results in mito- directly and gives me two nicely organized tables: one chondrial membrane? What event results in endo- listing the similarities and the other the differences plasmic reticulum membrane? None of these ques- between a mitochondrion and a chloroplast. The tions directly answers the third question, but they give answer to the similarity question tells me that both me starting points for my exploration that might help have a function of energy transformation. The simi- me eventually construct an argument of the kind that larity answer gives me little information about struc- this problem is asking for. ture. So, I review the answer to the differences section (See figure 4). Under the structure section of the In the previous scenario, we can see that Inquire answer, I am told that a chloroplast has a chloroplast Biology does not directly answer a homework prob- membrane and a mitochondrion has a mitochondrial lem, but provides answers to portions of the ques- membrane. They are named differently, but seem sim- tion, which a student must then assemble to con- ilar, and since the first problem asks me to consider structure, I ask the question: What is the similarity struct an overall answer. Through this process, it between a chloroplast membrane and a mitochondri- provides an easier access to facts and relationships, al membrane? In the answer, I am told that they are but the student is still required to develop a concep- both double membranes. I bet that this is one of the tual understanding of the material to answer a prob- key structural features that should be an answer to the lem. By using Inquire Biology’s question-answering problem. facility, students may not necessarily be able to do To answer the second question, I can directly ask their homework faster, but we do expect them to Inquire Biology factual questions such as: Is it true that acquire a deeper understanding of material and be plant cells have a mitochondrion? Inquire Biology tells me that the answer is: Yes. This straight fact retrieval more engaged in doing their homework. The fact- reduces the need to memorize, but developing an retrieval questions as we saw in the solution of the explanation is more challenging. I could ask the ques- second question help the student learn basic facts, tion: What is the relationship between a mitochon- and the comparison question that we saw in the first drion and a plant cell? Inquire Biology presents a graph question relates pieces of information that may not showing the relationship between the two. This helps me develop an explanation that a mitochondrion is a be presented next to each other in the textbook. This part of plant cells and functions during cellular respi- reduces cognitive load and encourages a focus on ration. conceptual understanding. 62 AI MAGAZINE  Articles KR Suggested Suggested Authoring Language Question Question Guidelines and Concept Generation Selection Library Knowledge Question Domain Authoring Knowledge Question Inquire User Inference Answering User Expert System Base Interpretation Interface Methods Answer Answer Textbook Generation Presentation server client (app) Legend external components components Figure 7. System Architecture AI Technology in Inquire Biology ering axioms), rules (attached to a frame’s slots), and the use of prototypes (canonical examples of a con- We now give an overview of the AI technology that cept). The knowledge base itself uses an upper ontol- enables the functioning of Inquire Biology. Detailed ogy called the component library or CLIB (Barker, descriptions of various system components are avail- Porter, and Clark 2001). The CLIB is a domain-inde- able in previously published papers (Chaudhri et al. pendent concept library. The SMEs access the CLIB 2007, Clark et al. 2007, Gunning et al. 2010). through AURA and use it to create domain-specific The overall system consists of modules for creating representations for knowledge in Campbell Biology, knowledge representation from the book, modules resulting in a biology knowledge base. The current for performing inference and reasoning methods on knowledge base has been well-tested for the ques- the knowledge, and modules for asking questions tions used in Inquire Biology for chapters 2–12 of and presenting answers to the student. Figure 7 illus- Campbell Biology. trates the functional components of the system. KM also provides the core inference services for For the purpose of Inquire Biology, a subject-matter AURA. KM performs reasoning by using inheritance, expert (SME) is a biologist with a bachelor’s degree in description-logic-style classification of individuals, biology or a related discipline. The SME uses a knowl- backward chaining over rules, and heuristic unifica- edge authoring system and the authoring guidelines tion (Clark and Porter 2012). A focus of innovation to represent knowledge from the textbook contribut- in our work has been to identify these core features ing to a growing knowledge base. Inquire Biology uses and to specify them in a declarative manner AURA as its knowledge authoring system. Details on (Chaudhri and Tran 2012). We export the biology KB AURA have been published in AI Magazine (Gunning in a variety of standard declarative languages, for et al. 2010). The authoring guidelines specify a example, first-order logic with equality (Fitting process by which SMEs systematically read the text- 1996), SILK (Grosof 2009), description logics (DLs) book content, arrive at a consensus on its meaning, (Baader et al. 2007) and answer-set programming encode that meaning, test their encoding by posing (Gelfond and Lifschitz 1990). questions, and validate the knowledge for its educa- In addition, AURA incorporates several special- tional utility. purpose reasoning methods for answering compari- AURA uses the knowledge machine (KM) as its son and relationship reasoning questions (Chaudhri knowledge representation and reasoning system et al. 2013). We briefly explain the computation used (Clark and Porter 2012). KM is a frame-based repre- in these methods. For comparison questions, we first sentation language that supports a variety of repre- compute the description of a typical instance of each sentational features including a class taxonomy, a class in terms of its types and locally asserted slot val- relation taxonomy, partitions (disjointness and cov- ues and constraints. Next, we compute the similari- FALL 2013 63 Articles ties and differences between the two descriptions. usability of the question-answering facility. Our The result is organized into a table. We use a variety approach to question generation is similar to the spir- of presentation heuristics to identify the most impor- it of other recent work on question generation4 with tant similarities and differences and to present the one important difference: our system has access to a differences that correspond to each other in the same detailed knowledge representation of the textbook row of the table. As an example heuristic, a slot that that can be used for question generation while most indeed has a value that is different is shown before a other question-generation systems attempt to gener- slot in which there is a value for one class but not a ate questions from the surface representation of the value for another. The presentation heuristics also text. rely on input from biologists on how the slots should After the inference and question-answering com- be organized. For example, the slots such as has part ponents produce the basic content of an answer, the and has region are grouped together as they corre- answer-generation module constructs the full, spond to structural information. To compute the human readable answer. Each kind of answer has a relationship between two individuals, we first calcu- specific screen layout and uses a mixture of presenta- late all paths in the knowledge base between two tion types that include bulleted lists, graphs, and nat- individuals. Since calculating all paths is expensive, ural language statements to describe the base facts we use heuristics to control the search. As an exam- from the knowledge base. The answers leverage a nat- ple, we first explore only taxonomic relationships ural language generation component that automati- and then relationships that can be found in a single cally generates English sentences from the KB (Banik concept graph. The computed paths are ranked in et. al. 2012). the order of importance. As in the case of comparison questions, the importance is computed using heuris- tics provided by the biologists. As an example, the The Evaluation of Inquire Biology paths involving only structural slots are preferred The goal of evaluating Inquire Biology was to assess the over paths that contain arbitrary slots. extent to which the AI enhancements to Campbell Inquire Biology works through an HTTP connection Biology were useful to students for the active reading to an AURA server running on a Windows machine. and homework support tasks, and to determine The AURA server contains the biology knowledge whether Inquire Biology leads to better learning. The base and supports the question-answering and ques- formal evaluation of Inquire Biology was preceded by tion-suggestion facilities, the details of which have a series of user studies, which we used to refine Inquire been previously published (Gunning et al. 2010). Biology’s capabilities and ensure that the system was AURA also pre-computes all the concept summary usable. In discussing the evaluation, we describe the pages, which are included in the Inquire Biology appli- experimental design, participant profile, procedure cation during the build process. used, and both qualitative and quantitative results. The question-answering facility in AURA relies on a controlled natural language understanding system Experimental Design that was described by Clark et al. (2007) and Gun- Our evaluation used a between-subjects research ning et al. (2010). With the direct use of controlled design, with students randomly assigned to one of natural language, the students often had difficulty three conditions. The full Inquire Biology group (N = anticipating which questions the system can or can- 25) used the system version that had all the AI- not handle. To address this issue, AURA supports a enabled features that we discussed earlier in this suggested question facility. In response to questions paper. The textbook group (N = 23) used a standard entered by the student, AURA computes the most print copy of Campbell Biology. The ablated Inquire closely matching questions that can be answered and Biology group (N = 24) used a version of Inquire Biolo- presents these suggestions to the student. This com- gy that lacked the concept summaries and question- putation relies on a large database of questions that answering capabilities but included all the other are automatically generated by traversing the knowl- ebook features such as highlighting and annotation. edge base. For example, for two concepts that are sib- The number of subjects in each group was chosen to lings of each other in the class taxonomy, the system ensure that the results are statistically significant. will generate a comparison question that asks for The purpose of these three groups was to compare similarities and differences between the two con- studying with Inquire Biology to two forms of existing cepts. Questions are generated that conform to the practice: studying with a paper textbook and study- templates that are supported in the system. As the ing with a conventional ebook reader version of a student types a question and even partially enters textbook. We wished to control for unexpected questions, the precomputed database of questions is effects that may arise from the iPad platform, or idio- used to find most closely matching questions and syncrasies with the Inquire Biology interface. Specifi- propose them as suggestions. This approach substan- cally, we wanted to avoid criticisms that any tially reduces the occurrence of questions that can- improvements observed by using Inquire Biology were not be parsed by AURA and greatly enhances the simply due to the excitement of using an iPad and 64 AI MAGAZINE  Articles that use of an unenhanced electronic textbook low- The active reading training introduced the stu- ers student performance as compared to using a dents to habits of active reading (predict, ask, con- paper textbook. nect, and explain), and illustrated these habits using In actual practice, the students might use lecture an example. The example used for this training was notes, Wikipedia, and other teacher-provided gravitation, and was specifically chosen to be unre- resources during active reading and homework prob- lated to the topic of the actual evaluation. After a lec- lem solving. However, we did not incorporate these ture on active reading, students were given a script- into our study because of the huge variation in avail- ed tutorial on active reading. This tutorial used a able supplementary resources. section from chapter 6 of Campbell Biology that con- Participants cerned cell structure and provided step-by-step instructions on how a student could engage in active We recruited 72 participants from a local communi- reading using the features of Inquire Biology, ablated ty college. The number of subjects was chosen to Inquire Biology, or the print version of the text. ensure that the study had sufficient power to detect The training segment for problem solving focused expected differences. All the participating students on understanding homework problems, different were enrolled in an introductory biology course that question types, understanding answers, and con- used the same edition of the Campbell Biology text- structing follow-up questions. The training also cov- book that is contained within Inquire Biology. The stu- dents were prescreened to ensure that they had not ered question formulation for questions that contain previously covered the sections of the text used in “how” and “why” in the question statement. Many the evaluation. Based on the student background such questions can be rephrased as questions that are variables, such as grade point average, age, gender, supported in Inquire Biology (“How” and “Why” educational background, native language, and famil- questions are not directly supported). The training iarity with iOS devices, the three groups were com- segment also included a five-minute video illustrat- parable at the onset of the study. All students were ing how a student can use Inquire Biology for solving financially compensated for their time; they did not a nontrivial homework problem. receive any academic credit for their participation; Most of the active reading training was adminis- and, there was no attrition of the participants during tered to all three groups. However, only the full the exercise. Inquire Biology group received training on using Inquire Biology features for active reading and home- Procedure work problem solving. All groups performed an identical learning task: they The problem sets used for homework and the were asked to study a unit on cellular membrane posttest were designed by a biology instructor. The structure and function. We chose this topic as it is homework problem set had six problems (with sev- fundamental to cell biology, and students frequently eral subparts), and the posttest had five problems. have difficulty understanding the material. Students’ Both problem sets included problems that the specific learning objectives were to (1) understand instructor would normally use during the teaching specific components of membranes and their func- of the course and were not specifically aligned to the tions, (2) understand what specific components of capabilities of Inquire Biology. The problem sets were membranes contribute to their selective permeability, tested for any obviously confusing information by and (3) explore how structure influences function at first trying them with students in the user studies molecular and macromolecular levels. preceding the evaluation. This ensured that any con- The evaluation took place outside of the classroom fusing information about the problem statement to ensure that study conditions were the same for would not confound the results. Students were able each group. Each session took place on a single day to use their book while doing the homework exer- and lasted between four and six hours depending on cises but not during the final posttest. the group. The evaluation consisted of four primary components: introduction and training, active read- Students were asked to read the first two sections ing task, homework support task, and a posttest. In of chapter 7 from Campbell Biology in 60 minutes, do addition, a researcher interviewed students in the homework in 90 minutes, and then take a posttest Inquire Biology groups afterwards to better understand in 20 minutes (table 1). The exercise was followed by their experience. an informal discussion session in which the students The introduction and training consisted of an ori- completed usability and technology-readiness sur- entation to the study and a 30-minute training exer- veys and provided qualitative feedback. The stu- cise designed to familiarize participants with the dents’ homework and posttest were each scored by process of active reading and how they could use two teachers, and score discrepancies were resolved. Inquire Biology for homework problem solving. The All student work was coded such that the teachers training on active reading and homework problem had no knowledge of which student or condition solving was conducted by a biology teacher. they were scoring. FALL 2013 65 Articles 1 hour 2 3 4 5 intro Training Active reading task Lunch Homework support task Posttest Debrief (60 minutes) (20 (90 minutes) minutes) Table 1. Typical Structure for Each Condition. INQUIRE ABLATED INQUIRE PAPER BOOK A 88 B 81 81 74 75 C 71 D HOMEWORK SCORES QUIZ SCORES P-value from 2 tailed t-tests: HW Scores Quiz Scores Full versus Ablated: 0.12 Full versus Ablated: 0.002 Full versus Textbook: 0.02 Full versus Textbook: 0.05 Ablated versus Text: 0.52 Ablated versus Text: 0.18 Figure 8. Quantiative Posttest Results Quantitative Results from The Posttest ding score of 74 for the ablated Inquire Biology group and a score of 71 for the textbook group. The differ- Figure 8 shows the quantitative posttest results. Both ence between the Inquire Biology group and the ablat- the homework and posttest tasks had a maximum ed Inquire Biology group was not statistically signifi- score of 100. The scores in figure 8 show the mean cant (p value = 0.12), but the difference between the score of students in each group. Inquire Biology group and the textbook group was sig- Our primary interest is in the effect on the posttest nificant (p value = 0.02). The observed trend is con- score of Inquire Biology as compared to the two con- sistent with our hypothesis that Inquire Biology trasting, more typical conditions. The mean posttest enhances learning by helping students perform bet- score of 88 for the Inquire Biology group was higher ter on homework. than both the corresponding score of 75 for the The comparison between the paper textbook and ablated Inquire Biology group and the score of 81 for the ablated version of Inquire Biology is also interest- the textbook group. The difference between the ing. Although the mean homework score of 74 for Inquire Biology group and the ablated Inquire Biology ablated Inquire Biology and a mean score of 71 for the group was statistically significant (p value = 0.002). paper textbook differ, there was no statistical differ- The difference between the Inquire Biology group and ence between these conditions. Similarly, there was the textbook group was also significant (p value = no statistical difference between the homework 0.05). These differences suggest that students who scores of the two conditions (p value = 0.18). This used Inquire Biology learned more. result is consistent with the prior research that simply The mean homework score of 81 for the Inquire changing the medium of presentation has no impact Biology group was higher than both the correspon- on student learning (Means et al. 2009). 66 AI MAGAZINE  Articles ACTIVE READING TASK HOMEWORK TASK 90 75 Asking questions* Viewing glossary pages** 60 Viewing images/graphs Writing notes minutes 45 Viewing glossary popup Reading and highlighting 30 * omitted feature in ablated Inquire ** limited feature in ablated Inquire 15 0 ABLATED INQUIRE ABLATED INQUIRE INQUIRE INQUIRE Figure 9. Average Student Usage Time by Activity. We gathered extensive data on how the different tual questions; and the remaining 40 percent were features of Inquire Biology were used but did not dis- evenly split between comparison questions and rela- cover any significant correlation between the usage tionship questions. This usage data is suggestive of of a particular feature and the improvement on the usefulness of these features to students (see figure posttest performance. 9). We did not explicitly measure to what extent stu- During the active reading task, students cumula- dents followed the specific steps of active reading, or tively asked 120 questions, and during the home- to what extent they acquired deeper understanding work task they asked a total of 400 questions. This of knowledge, or whether they were more engaged difference is expected, as students naturally have a with the material. The posttest scores are a measure greater need for asking questions during the home- of depth of understanding, and increased engage- work task. During active reading, the question asking ment was apparent in their qualitative feedback, predominantly came from students clicking on ques- which we discuss next. Additional analysis could be tions suggested in response to their highlighting of done by analyzing their note taking behavior, but text. During homework problem solving, however, since it was orthogonal to the core AI features that students were more prone to use Inquire Biology’s were the subject of the study, we left it open for question-asking dialogue. A total of 194 unique future work. questions were asked by the students, out of which only 59 questions were asked by two or more stu- Qualitative Results dents. The variety of questions would make it very We gathered three forms of qualitative data to assess difficult to anticipate all questions in the textbook the usefulness of Inquire Biology. The data included itself. This suggests that automatic question answer- the usage of Inquire Biology features as well as results ing can be a useful addition to a textbook. from a usability survey and a technology readiness In figure 10, we show the actual questions that the survey. students asked and how many different students We tracked the questions asked by students and asked each question. It can be seen that there is a the concept summary pages visited by them. All stu- good spread of different question types even though dents in the full Inquire Biology group asked questions the questions of the form “What is X” predominat- and visited concept summary pages. Students viewed ed the mix. a total of 81 concept summary pages during the exer- After use, the students rated the usability of Inquire cise; 38 of these were viewed by at least two students. Biology using the SUS scale (Brooke 1996). The SUS Approximately 30 percent of the questions were of score for ablated Inquire Biology was 84.79 with a the form “What is X?”; another 30 percent were fac- standard deviation of 11.82, whereas the score for FALL 2013 67 Articles What are the differences between a fluid mosaic model and a sandwich model? What is a transmembrane protein? What is the relationship between a biomembrane and a plasma membrane? What is a glycoprotein? What is a channel protein? Is the a sandwich model generally accepted or not? Where is a channel protein located? What is a glycolipid? What is a sandwich model? What is cholesterol? What are the differences between a sandwich model and a fluid mosaic model? What is a membrane carbohydrate? What is a peripheral protein? What is an amphipathic molecule? What is an integral protein? What is an integrin? What is the relationship between a peripheral protein and an amphipathic molecule? What are the differences between a glycolipid and a glycoprotein? What are the differences between a saturated fatty acid and a unsaturated fatty acid? What are the differences between a saturated fatty acid and an unsaturated fatty acid? What is a fluid mosaic model? What is a phospholipid? What is a plasma membrane? What is the relationship between a channel protein and an amphipathic molecule? What are the differences between a biomembrane and a phospholipid layer? What are the differences between a lipid and a protein? What are the differences between a peripheral protein and a transmembrane protein? What are the differences between an integral protein and a peripheral protein? What do cholesterol maintain? What do glycoproteins do to cholesterol? What is a carbohydrate? What is a hydrophobic core? What is a transmembrane protein across? What is the difference between a biomembrane and a plasma membrane? What is the relationship between a sandwich model and a fluid mosaic model? What is the structure of a glycolipid? 0 4 8 12 16 20 24 Figure 10. Number of Students Who Asked a Question full Inquire Biology was 84.4 with a standard deviation they felt less distracted as they could use Inquire Biol- of 9.02. This suggests a high degree of subjective sat- ogy to get additional information without losing con- isfaction in using Inquire Biology as well as no degra- text. Usage logs show students made use of question dation in usability from the addition of AI-based fea- answering and concept summary pages during the tures. active reading task, which suggests that they were The students also completed a survey to provide engaged in the content and actively seeking out relat- subjective ratings of various features of Inquire Biolo- ed and supporting information. A more extensive gy. The students were asked to indicate whether a evaluation of how Inquire Biology improves engage- particular feature was ready for use, ready for use ment is open for future work. with some improvements, or required major Overall, the qualitative assessment of the Inquire improvements. The concept summary pages had the Biology features was positive but also indicated some highest score with 76 percent of students rating them areas for improvement. Students reported that the as ready to use in the present form, 16 percent rating comparison answers were “well organized” and “to them as requiring minor improvement, and 8 per- the point,” but they also commented that the range cent rating them as requiring major improvements. of questions supported needs to be expanded and the For the question-answering facility, 51 percent of the students rated it as ready to use in present form, 37 answer quality still needs to be improved. percent rated it as requiring minor improvement, and 12 percent rated it as requiring major improve- Generalizabiltiy and Scalability ments. In the posttask debrief, students in the Inquire Biol- Let us now consider how the process of knowledge ogy group unanimously reported that they would like base construction generalizes and scales potentially to use the system for their class. They reported that to textbooks other than biology. Our most substantial Inquire Biology “motivates you to learn more” and experience in using AURA is with Campbell Biology. In “helps you stay focused.” Students also reported that addition to biology, we have prior experience in 68 AI MAGAZINE  Articles using AURA for physics and chemistry of the system needs to include mathe- time we built the first alpha versions of (Gunning et al. 2010). We have also matical problem solving. The text- Inquire Biology. Inkling 1.0 had all the performed a design-level analysis for books such as chemistry and algebra major features discussed above and, the domains of government and poli- also require capturing procedural like Inquire Biology, rendered content as tics, microeconomics, and environ- knowledge. A typical example of such speedy HTML rather than the slower mental science. Within the domain of knowledge in chemistry involves steps scans of book pages. Since 2010, biology, we have considered AURA’s for computing pH of a buffer solution. Inkling has added several features that applicability to representing a middle While solutions exist for mathemati- may address some of the specific needs school textbook and to advanced text- cal problem solving and capturing we target with Inquire Biology: provid- books on cell biology and neuro- procedural knowledge, their combina- ing definitions of key terms through science. We can draw the following tion with conceptual knowledge integrating Wikipedia into the app’s conclusions from these studies. requires novel research. The back- search feature, and shared notes that Conceptual knowledge (ability to ground knowledge provided in CLIB allow students to communicate with define classes, subclasses, stating dis- needs to be extended for each text- their classmates and ask each other jointness, defining slot values and book, requiring some research in con- questions about the material. Because rules), mathematical equations, and ceptual modeling and application of AURA’s answers never stray from the qualitative knowledge (for example, this research by the domain experts in content of Campbell Biology, they are directly or inversely proportional) cut the respective domain. New textbook more accurate and better scoped than across all domains and are widely subjects also require developing new what one might get from Wikipedia or applicable. Such knowledge can question types and answer-presenta- from one’s peers. Inkling’s work is account for at least half of the knowl- tion methods. however an important step forward for edge that needs to be captured in a students and an indication of the textbook. Inquire Biology demonstrates degree of rapid improvement that is that a useful enhancement to an elec- Related Work occurring in the electronic textbook tronic textbook can be built using The work presented here directly over- domain. these core forms of knowledge. The laps with three related areas of approach is directly applicable with lit- research: electronic textbooks, ques- Question-Answering Systems tle changes to any textbook that is tion-answering systems, and reading A survey of recent question-answering comparable in scope to Campbell such for comprehension. In this section, we systems has been published in AI Mag- as Mason, Losos, and Singer (2011) or briefly situate our work in these three azine (Gunning, Chaudhri, and Welty a middle school textbook (Miller and areas of related research. 2010). For the present discussion, we Levine 2010). The conceptual knowl- will compare AURA to Watson and edge, qualitative knowledge, and Electronic Textbooks Wolfram Alpha. mathematical equation features of To inform the initial design of Inquire Watson is a recent question-answer- AURA generalize to any of the domains Biology, in early 2010 we evaluated ing system from IBM that outper- mentioned above. existing ebook apps and electronic formed humans in the game of Jeop- Outside the context of an AI textbooks to develop a list of key fea- ardy (Ferrucci 2012). Watson is an research project, one possible tures and solutions to common prob- impressive system that showed per- approach to achieve scalability of the lems. Based on this analysis, we deter- formance over a broad domain of ques- approach in practice is for the knowl- mined that students would expect tion answering. One of the primary edge representation of the content of Inquire Biology to save their places in characteristics of the Jeopardy problem the book to become an integral part of the book and make it easy to take is that 94.7 percent of the answers to the book authoring process. This new notes, highlight text, zoom in on questions are titles of some Wikipedia requirement is no different than the images, and search the concept sum- page (Chu-Carroll and Fan 2011). The preparation of table of contents, index, maries. We also saw many approaches remaining 5.3 percent of the answers or glossary that is typically found in to displaying the highly structured that are not the titles of the Wikipedia standard textbooks. The books of the content of textbooks, from scaled- pages include multiple entities such as future could include knowledge repre- down images of actual pages (Kno5) to red, white, and blue, or even short sen- sentation for the most salient portions entirely custom iPad-optimized lay- tences or phrases, such as make a scare of the content, which could enable outs (Apple iBooks textbooks). We cow. Since there are more than 3 mil- numerous pedagogical features some chose to render the book content as a lion Wikipedia pages and a huge vari- of which are illustrated in Inquire Biol- vertically scrolling HTML page, as that ety of questions, this is not an easy ogy. gave us an optimal combination of task, yet it is constrained in a way that AURA does not cover all forms of readable text, flexible layouts, good makes it amenable to solution by knowledge found across multiple performance, and a reasonable level of machine-learning techniques. In con- domains. Some domains are more effort. trast, the answers returned by AURA mathematical than others, for exam- Standing apart from the other iPad can span from a paragraph to a page — ple, physics, algebra, or economics. For textbook apps is Inkling,6 an app especially the answers to the compari- such domains, the reasoning capability released in August 2010, around the son and relationship reasoning ques- FALL 2013 69 Articles tions. Furthermore, these answers are text structure before reading can serve pendence in nature. Expanding the not always stated in the textbook, and as a form of advance organizer that knowledge base to the full textbook even if they are, they may be stated in cues existing knowledge, supports requires ensuring the scalability of the different parts of the textbook. In addi- strategic reading, and permits readers knowledge authoring system in AURA. tion, Inquire Biology is an interactive to mentally record new knowledge Additional work also needs to be done user-centered application and has more effectively (Ausubel and Youssef on developing new reasoning methods much higher usability demands. 1963; Mayer 1979). Providing readers such as for process interruption rea- Wolfram Alpha is a question-answer- with opportunities to check their soning and to expand the range of ing system based on Mathematica, a knowledge as they read supports active questions that the system currently computational reasoning engine.7 self-regulation of learning (Bransford, suggests. The usability of the system Wolfram Alpha relies on well-defined Brown, and Cocking 1999; Brint et al. will also improve substantially by computations over curated data sets. 2010). Prompting readers to pose ques- expanding the range of English that For example, when given a question tions as they read is also useful for can be accepted as input questions. such as “orange and banana nutri- metacognitive monitoring, improving tion?” it looks up the nutritional infor- comprehension, and making connec- More Extensive mation for both orange and banana, tions among ideas (Novak 1998; Press- Educational Studies adds them, and presents the net result. ley and Afflerbach 1995; Udeani and The evaluation considered in this arti- When given an equation such as g(n + Okafor 2012). In addition, coupling cle lasted for only a few hours over a 1) = n2+g(n), Wolfram Alpha gives its visual elements in textbooks, such as single day. It remains to be shown if solution and presents a plot showing diagrams, with text can also have quite similar improvements in student learn- the value of the solution as a function robust effects on student learning ing can be obtained if the students are of n. Wolfram Alpha is similar in spirit (Mayer 2003), especially for students allowed to take Inquire Biology home to AURA in the sense that it provides who prefer visual information (Mayer and study from it over a period of sev- well-defined computations over curat- and Massa 2003). eral weeks. The current study suggested ed data, but very different in terms of that Inquire Biology may be especially the curated data and the specific com- useful for lower-performing students; putation. For AURA, the knowledge is Future additional students need to be tested curated from a biology textbook, and Inquire Biology is an early prototype of to validate this trend. We also need to the computations are based on logical an intelligent textbook. Our results better understand which specific fea- reasoning rather than mathematics. show the promise of applying AI tech- tures of Inquire Biology contributed to nology to electronic textbooks to the improvement in learning that we Reading Comprehension enhance learning. To fully realize this observed. Achieving the potential offered by promise, more work needs to be done ebook technology will depend on a in at least the following categories: Improving Learning Gains rich understanding of how students improving the representation, improv- Current results show that the use of learn from college-level science text- ing question-answering capability, Inquire Biology improved the posttest books. Past research indicates that as conducting more extensive education scores by approximately 10 percent. A the information complexity of such studies, and improving the learning natural question is whether this is the textbooks increases, the explanatory gains. We discuss each of these direc- maximum improvement possible. Sev- clarity, or text cohesion, declines tions in more detail. eral new educational supports can be (Graesser et al. 2004). To succeed, sci- included in Inquire Biology to support a ence students must move from decod- Improving Representation better model of student comprehen- ing to sense-making, which involves and Question Answering sion, identifying student-specific prob- making connections between prior As of early 2013, Inquire Biology pro- lem areas to provide targeted support, knowledge and new information vides very good coverage of chapters and incorporating better problem solv- (Kintsch 1988; Palinscar and Brown 2–12 of Campbell Biology, and some ing dialogue capabilities. In the long 1984; Scardamalia et al. 1996), yet coverage of chapters 13–21, 36, and 55. term it could have a full-fledged intel- research indicates that many college It currently handles factual, compari- ligent tutoring system. students find it difficult to read science son, and relationship reasoning ques- textbooks because of gaps in their tions. The knowledge representation understanding of science concepts needs to be both deepened and Summary and Conclusions (diSessa 1993; Driver and Easley 1978; expanded to cover the remaining Inquire Biology is an innovative applica- Ozuru, Dempsey, and McNamara chapters. The current representation tion of AI technology to an electronic 2009). handles knowledge about structure textbook. The key ideas in Inquire Biol- Inquire Biology features promote and function, process regulation, and ogy are to provide automatically gener- active reading strategies that can help energy transfer. Additional representa- ated concept summary pages and sug- readers construct better models of the tion constructs are needed in CLIB for gested questions, and to provide concepts they are learning. Priming representing experiments, evolution, answers to comparison and relation- readers to focus on their goals or the continuity and change, and interde- ship questions. This is enabled by tak- 70 AI MAGAZINE  Articles ing advanced AI technologies and man, Nikhil Dinesh, Debbie Frazier, Mind, Experience, and School. Washington, blending them into an ebook to pro- Stijn Heymans, Sue Hinojoza, Eric DC: National Research Council. vide a seamless user experience. Most Kow, David Margolies, Ethan Stone, Brint, S.; Douglass, J. A.; Thomson, G.; and natural language question-asking William Webb, Michael Wessel, and Chatman, S. 2010. Engaged Learning in the interfaces suffer from the problem that Neil Yorke-Smith. Public University: Trends in the Undergrad- uate Experience. Report on the Results of the student does not have a clear sense of what kind of questions the system Notes the 2008 University of California Under- graduate Experience Survey. Berkeley, CA: can handle. Inquire Biology addresses 1. This data is based on our analysis of a spe- Center for the Studies in Higher Education. this by the use of suggested questions. cific biology textbook and measuring the number of new concepts and relationships Brooke, J. 1996. Sus: A Quick and Dirty The evaluation results show that stu- Usability Scale. In Usability Evaluation in that needed to be encoded from it. dents find Inquire Biology usable and Industry, ed. P. W. Jordan, B. Thomas, B. A. 2. Different groups of students and courses engaging, and that they learned more. Weerdmeester, and A. L. McClelland. Lon- can have varying requirements. For exam- Although this result is from a prelimi- ple, students studying for an advanced don: Taylor and Francis. nary evaluation study, it is an impor- placement exam need to be responsive to Chaudhri, V. K.; Heymans, S.; Tran, S.; and tant and significant result as it is one of the requirements of that exam. We have Wessel, M. 2013. Query Answering in the first to conclusively demonstrate made no special effort to customize Inquire Object-Oriented Knowledge Bases. Paper the usefulness of explicitly represented Biology to such specific needs. presented at the 6th Workshop on Answer knowledge and question answering as 3. See www.projecthalo.com. Set Programming and Other Computing Paradigms, Istanbul, Turkey, 24–29 August. students study from their assigned sci- 4. See www.questiongeneration.org/. ence textbook. Chaudhri, V. K., and Tran S. C. 2012. Speci- 5. See www.kno.com. fying and Reasoning with Underspecified When students transition from print 6. See www.inkling.com. Knowledge Bases Using Answer Set Pro- textbooks to electronic textbooks, they 7. See www.wolframalpha.com/about.html. gramming. In Proceedings of the 13th Inter- will learn more only if the new media national Conference on Knowledge Representa- better supports their learning process. References tion and Reasoning. Palo Alto, CA: AAAI To date, most work with electronic Press. Agrawal, R.; Gollapudi, S.; Kannan, A.; and textbooks reproduces only capabilities Kenthapadi, K. 2011. Data Mining for Chaudhri, V.; John, B. E.; Mishra, S.; such as highlighting and annotation, Improving Textbooks. SIGKDD Explorations Pacheco, J.; Porter, B.; and Spaulding, A. which are already available for paper 13(2): 7–10. 2007. Enabling Experts to Build Knowledge textbooks. By using knowledge repre- Ausubel, D. P. 1965. A Cognitive Structure Bases from Science Textbooks. In Proceedings sentation, question-answering, and View of Word and Concept Meaning. In of the 4th International Conference on Knowl- natural language generation tech- Readings in the Psychology of Cognition, ed. R. edge Capture. New York: Association for C. Anderson and D. P. Ausubel, 103–115. Computing Machinery. niques from AI, we have shown that an electronic textbook can go beyond New York: Holt, Rinehart and Winston, Inc. Chu-Carroll, J., and Fan, J. 2011. Leveraging Ausubel, D. P., and Youssef, M. 1963. Role of Wikipedia Characteristics for Search and what is possible in paper books. Specif- Discriminability in Meaningful Paralleled Candidate Generation. In Proceedings of the ically, knowledge representation can 25th AAAI Conference on Artificial Intelli- support students in asking and answer- Learning. Journal of Educational Psychology 54(6): 331–331. gence. Palo Alto, CA: AAAI Press. ing questions about the relationships Clark, P. E., and Porter, B. 2012. KM — The Baader, F.; Calvanese, D.; Mcguinness, D. L.; among the large number of concepts Knowledge Machine 2.0 User’s Guide. Tech- Nardi, D.; and Patel-Schneider, P. F., eds. that are newly introduced in challeng- nical Report, Dept. of Computer Science, 2007. The Description Logic Handbook: Theo- ing science courses. By helping stu- ry, Implementation, and Applications, 2nd ed. University of Texas at Austin. dents understand the relationships Cambridge, UK: Cambridge University Clark, P.; Chaw, S.-Y.; Barker, K.; Chaudhri, among concepts, an AI-enriched text- Press. V.; Harrison, P.; Fan, J.; John, B.; Porter, B.; book has the potential to increase stu- Banik, E.; Kow, E.; Chaudhri, V.; Dinesh, N.; Spaulding, A.; Thompson, J.; and Yeh, P. Z. dents’ conceptual understanding as and Oza, U. 2012. Natural Language Gener- 2007. Capturing and Answering Questions well as their satisfaction with study ation for a Smart Biology Textbook. Paper Posed to a Knowledge-Based System. In Pro- materials. We hope that Inquire Biology presented at the International Conference ceedings of the 4th International Conference on on Natural Language Generation, Utica Il, Knowledge Capture. New York: Association will provide inspiration for a variety of 30 May – 1 June. for Computing Machinery. AI methods to provide supports for Barker, K.; Porter, B.; and Clark, P. 2001. A Disessa, A. A. 1993. Toward an Epistemolo- personalization and leveraging online Library of Generic Concepts for Composing gy of Physics. Cognition and Instruction 10(2– resources in electronic textbooks that 3): 105–225. Knowledge Bases. In Proceedings of the 1st facilitate the development of inquisi- International Conference on Knowledge Cap- Driver, R., and Easley, J. 1978. Pupils and tive scientists and engineers that are so ture, 14–21. New York: Association for Com- Paradigms: A Review of Literature Related to much needed. puting Machinery. Concept Development in Adolescent Sci- Bliss, J.; Askew, M.; and Macrae, S. 1996. ence Students. Studies in Science Education Acknowledgements 5(10): 61–84. Effective Teaching and Learning: Scaffold- This work has been funded by Vulcan ing Revisited. Oxford Review of Education Ferrucci, D. A. 2012. This Is Watson. IBM Inc. The authors wish to thank the 22(1): 37–61. Journal of Research and Development 56(3–4). members of the Inquire Biology devel- Bransford, J. D.; Brown, A. L.; and Cock- Fitting, M. 1996. First-Order Logic and Auto- opment team: Eva Banik, Roger Cor- ing, R. R. 1999. How People Learn: Brain, mated Theorem Proving. Berlin: Springer. FALL 2013 71 Articles Gelfond, M., and Lifschitz, V. 1990. Logic Ozuru, Y.; Dempsey, K.; and McNamara, D. national, where he designed and built the Programs with Classical Negation. In Logic S. 2009. Prior Knowledge, Reading Skill, and Inquire iPad app. Overholtzer likes to work Programming: Proceedings Seventh Interna- Text Cohesion in the Comprehension of in pixels, in code, and in his free time, Lego. tional Conference, ed. D. Warren and P. Science Texts. Learning and Instruction 19(3): Szeredi, 579–597. Cambridge, MA: The MIT 228–242. Jeremy Roschelle specializes in the design Press. and development of integrated interven- Palinscar, A. S., and Brown, A. L. 1984. tions to enhance learning of complex and Graesser, A. C.; McNamara, D. S.; Louwerse, Reciprocal Teaching of Comprehension- conceptually difficult mathematics and sci- M. M.; and Cai, Z. 2004. Coh-Metrix: Fostering and Comprehension-Monitoring ence; learning sciences–based research in Analysis of Text on Cohesion and Lan- Activities. Cognition and Instruction 1(2): mathematics education, on collaborative guage. Behavior Research Methods 36(2): 117–175. learning, and with interactive technology; 193–202. Pera, M., and Ng, Y. 2012. Personalized Rec- and the management of large-scale multi- Grosof, B. N. 2009. Silk: Higher Level Rules ommendations on Books for K–12 Students. year, multi-institutional research and eval- with Defaults and Semantic Scalability. In In Proceedings of the Fifth ACM Workshop on uation projects. Web Reasoning and Rule Systems, Proceedings Research Advances in Large Digital Book Repos- of the Third International Conference, Volume itories and Complementary Media. New York: Aaron Spaulding is a senior computer sci- 5837, Lecture Notes in Computer Science, Association for Computing Machinery. entist and interaction designer at SRI’s Arti- 24–25. Berlin: Springer. Pressley, M., and Afflerbach, P. 1995. Verbal ficial Intelligence Center. His work centers Gunning, D.; Chaudhri, V. K.; Clark, P.; Protocols of Reading: The Nature of Construc- on developing usable interfaces for AI sys- Barker, K.; Chow, S.-Y.; Greaves, M.; Grosof, tively Responsive Reading. Mahwah, NJ: tems that meet real user needs. He holds a B.; Leung, A; McDonald, D.; Mishra, S.; Lawrence Erlbaum. Master’s degree in human computer inter- Pacheco, J.; Porter, B.; Spaulding, A.; Tecu- action from Carnegie Mellon University. Reece, J.; Urry, L.; Cain, M.; Wasserman, S.; ci, D.; and Tien, J. 2010. Project Halo Minorsky, P.; and Jackson, R. 2011. Camp- Peter Clark is a senior research scientist at Update — Progress Toward Digital Aristotle. bell Biology, 9th ed. Boston: Benjamin Cum- Vulcan Inc. working in the areas of natural AI Magazine 31(3). mings. language understanding, question answer- Gunning, D.; Chaudhri, V.; and Welty, C., Scardamalia, M., and Bereiter, C. 2006. ing, machine reasoning, and the interplay eds. 2010. Special Issue on Question Knowledge Building: Theory, Pedagogy, and between these three areas. He received his Answering Systems. AI Magazine 31(3). Technology. The Cambridge Handbook of the Ph.D. in computer science from Strathclyde Kintsch, W. 1988. The Use of Knowledge in Learning Sciences, 97–115. Cambridge, UK: University, UK, in 1991. Discourse Processing: A Construction-Inte- Cambridge University Press. gration Model. Psychological Review 95(2): Mark Greaves is technical director for ana- Scardamalia, M.; Bereiter, C.; Hewitt, J.; and 163–182. lytics at the Pacific Northwest National Lab- Webb, J. 1996. Constructive Learning from oratory (PNNL). Prior to joining PNNL, he Mason, K.; Losos, J.; and Singer, S. 2011. Texts in Biology. In Relations and Biolo