Project and Portfolio Assessments PDF

Topic  Project and Portfolio 7 Assessments LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Describe the characteristics of project work; 2. Explain the procedures of u...

Topic  Project and Portfolio 7 Assessments LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Describe the characteristics of project work; 2. Explain the procedures of using project as an assessment technique; 3. Explain the role of portfolio assessment as an alternative assessment strategy; 4. Describe the characteristics of portfolios; and 5. Explain how a portfolio is developed.  INTRODUCTION Besides objective and essay tests, there are other types of assessments which can be collectively categorised as performance-based assessment. According to the definition provided by the Standards (AERA et al., cited in Reynold, Livingston & Willson, 2006), performance assessments require test takers to complete a task in a context or setting that closely resembles real-life situation. For example, to assess oral communication skills, the assessment might require test takers to participate in a group dialogue session. Likewise, to assess the teaching ability of teacher trainees, the trainees might be required to conduct a lesson with a group of learners. The emphasis of performance-based assessment is thus on doing, not merely knowing, on process as well as product. In this context, an essay test that is used to assess writing skills in language learning can be considered as a type of performance assessment. Since essay questions have been discussed earlier, we Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  129 will focus on other types of such assessments in this topic, namely projects and portfolios. Project assessments which require learners to perform practical tasks to organise or create something are fast becoming a common practice in school-based assessments. Portfolios, which are considered a specific form of performance assessment that is useful in assessing learner learning over time involve the systematic collection of a learnerÊs work produced over a specified period of time according to a specific set of guidelines (Reynold, Livingston & Willson, 2006). 7.1 PROJECT ASSESSMENT Most of us have done some form of project work at school or university and know what a project is. However, when asked to define it, one will see varying interpretations of the project and its purpose. Projects can represent a range of tasks that can be done at home or in the classroom, by parents or groups of learners, quickly or over time. Technically there are differences between project and project-based learning (PBL). While PBL also features projects, the focus is more on the process of learning and learner-peer-content interaction than the end-product itself. PBL closely resembles work done in the real world. The scenario or simulation is real whereas projects are usually based on events that have already been resolved. ACTIVITY 7.1 Go to this website http://www.friedtechnology.com/stuff Look at the charts prepared by Amy Mayer for differences between projects and project-based learning. Discuss your findings with your coursemates in the myINSPIRE online forum. A project is an activity in which time constraints have been largely removed and can be undertaken individually or by a group and usually involves a significant element of work being done at home or out of school (Firth & Mackintosh, 1987). Project work has its roots in the constructivist approach which evolved from the work of psychologists and educators such as Lev Vygotsky, Jerome Bruner, Jean Piaget and John Dewey. Constructivism views learning as the result of mental construction whereby learners learn by constructing new ideas or concepts based on their current and previous knowledge. Copyright © Open University Malaysia (OUM) 130  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Most projects have certain common defining features (Katz & Chard, 1989) such as: (a) Learner-centred; (b) A definite beginning, middle and end; (c) Content is meaningful to students and directly observable in their environment; (d) Real-world problems; (e) First-hand investigations; (f) Sensitivity to local culture and culturally appropriate; (g) Specific goals related to curriculum; (h) A tangible product that can be shared with the intended audience; (i) Connections among school, life and work skills; (j) Opportunities for reflective thinking and learner self-assessment; and (k) Multiple types and authentic assessments (portfolios, journals, rubrics and others). In project work, the whole work process is as important as the final result or product. Work process refers to learners choosing a knowledge area, delimiting it and formulating a problem or putting forward questions. It also involves learners investigating and describing what is required to solve a given problem or answer a specific question through further work, collection of materials and knowledge. Project work is planned so that it can be carried out within the available time. Preferably, the task should be drawn from knowledge areas in the current curriculum. Project work is an integrated learning experience that encourages learners to break away from the compartmentalisation of knowledge and instead involves drawing upon different aspects of knowledge. For example, making an object not only requires handicraft skills but also knowledge of materials, working methods and uses of the object. Technological support will also enhance learnersÊ learning. Thinking skills are integral to project work. Similarly, writing the project report requires writing skills learned in the language classroom and applying it when analysing and drawing conclusions for a science project. Generally, there are two types of projects, namely research-based and product-based projects. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  131 (a) Research-based project is more theoretical in nature and may consist of putting forth a question, formulating a problem or setting up some hypotheses. In order to answer the question, solve the problem or confirm the assumptions, information must be found, evaluated and used. This information can either be the result of learnersÊ own investigations or obtained from public sources without being a pure reproduction. Such project work is usually presented as a research report. (b) Product-based project would be the production of a concrete object, a service, a dance performance, a film, an exhibition, a play, a computer programme and so forth. In project work, learners actively involve themselves as problem solvers, decision makers, investigators, documenters and researchers. Project work provides an opportunity for learners to explore different approaches to solving problems. In project work a teacher follows, discusses and assesses the work in all of the different phases. The teacher is the learner's supervisor. If the work involves a group, the input from different learners should come from within their respective programme goals. Most importantly, learners should find projects fun, motivating and challenging because they play an active role in the entire process. The selection of a project may be determined by the teacher or the choice may be left to the learners, probably with the approval of the teacher (Katz, 1994). What is significant is that learners take ownership of their project. There are many types of effective projects. The following are just some project ideas: (a) Survey of historical buildings in the learnerÊs community; (b) Study of the economic activities of people in the local community; (c) Study of the transportation system in the district; (d) Recreate a historical event; (e) Develop a newsletter, blog or website on a specific issue relevant to the school or community (such as school safety, recycling and how businesses can save energy and reduce waste); (f) Compile oral histories of the local area by interviewing community elders; (g) Produce a website as a "virtual tour" of the history of the community; (h) Create a document on CD of learners graduating from local primary or secondary school; Copyright © Open University Malaysia (OUM) 132  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (i) Create a wildlife or botanical guide for a local wildlife area; (j) Create an exhibition on local products, local history or local personalities using audiotapes, videotapes and photographs; and (k) Investigate pollution of local rivers, lakes and ponds. The possibilities for projects are endless. The key ingredient for any project idea is that it is learner-driven, challenging and meaningful. It is important to realise that project-based instruction complements the structured curriculum. Project-based instruction builds on and enhances what learners learn through systematic instruction. Teachers do not let learners become the sole decision makers about what project to work on, nor do teachers sit back and wait for the learners to figure out how to go about the process which can be very challenging (Bryson, 1994). This is where the teacherÊs ability to facilitate and act as a coach plays an important part in the success of a project. The teacher would have brainstormed ideas with the learner to generate project possibilities, discuss possibilities and options, help the learner form a guiding question and be ready to help the learner throughout the implementation process such as setting guidelines, due dates, resource selection and so forth (Bryson, 1994; Rankin, 1993). SELF-CHECK 7.1 State the differences between a research-based project and a product- based project. ACTIVITY 7.2 Give examples of the two types of projects in your subject area. Share your answer with your coursemates in the myINSPIRE online forum. 7.1.1 Assessing Learners with Projects Project-oriented work is becoming increasingly common in work life. Project competence, the ability to work together with others, taking personal initiatives and having entrepreneurship skills are often required by employers. These competences can be developed during project work and thus prepare learners for Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  133 work life. Project work makes schooling more like the real world. In real life, we seldom spend several hours listening to authorities who know more than we do telling us exactly what to do and how to do it. We ask questions of the person we are learning from. We try to link what the person is telling us with what we already know. We bring our experiences and what we already know that is relevant to the issue or task and say or do something about it. You can see this with a class of young learners. When the teacher tells a story, little kindergarten children raise their hands eager to share their experiences about something related to the story. They want to be able to apply their natural tendencies to the learning process. This is how life is much of the time! By giving project work, we open up areas in schooling where learners can speak about what they already know. Project work is a learning experience which enables the development of certain knowledge, skills and attitudes which prepares learners for lifelong learning and the challenges ahead (refer to Table 7.1). These objectives may not be achieved by current instructional strategies. Table 7.1: Knowledge, Skills and Attitudes Achieved with Projects Domain Learning Outcome Knowledge application Learners will be able to make connections across different (Apply creative and areas of knowledge and to generate, develop and evaluate critical thinking skills) ideas and information so as to apply these skills to the project task.  Be able to choose a knowledge area and delimit a task or a problem.  Be able to choose relevant materials, methods as well as relevant tools.  Be held accountable to draw up a project plan and where necessary, to revise it. Communication Learners will acquire the skills to communicate effectively (Improve oral and by presenting their ideas clearly and coherently to specific written communication audiences in written and oral forms. skills)  Be able to discuss with their supervising teacher how their work is developing.  Be able to provide a written report of the project describing the progress of the work from initial idea to final product.  Be able to produce a final product which means an independent solution to the task or problem chosen. Copyright © Open University Malaysia (OUM) 134  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Collaboration Learners will acquire collaborative skills through working (Foster collaborative in a team to achieve common goals. learning skills) Independent learning Learners will be able to learn on their own, reflect on their (Develop self-directed learning and take appropriate actions to improve. inquiry and life-long  Be able to use a logbook to document the progress of learning skills) their work and regularly report the process.  Be able to assess either in writing or verbally their work process and results. Source: Blank & Harwell (1997) Project work helps learners by: (a) Developing the skills required of planning, structuring and taking responsibility for a larger piece of work and providing experience for working in „project form‰; (b) Deepening the knowledge within a subject or between subjects; (c) Providing learners with opportunities to explore the interrelationships and interconnectedness of topics within a subject and between subjects; and (d) Encouraging learners to synthesise knowledge from various areas of learning while critically and creatively applying it to real-life situations. Hence, it is important that learners are assigned to carry our authentic projects in which learners plan, implement and report on projects that have real-world applications beyond the classroom (Blank & Harwell, 1997). SELF-CHECK 7.2 What knowledge, skills and attitudes are evaluated when learners are given a project to work on? ACTIVITY 7.3 To what extent has project work been used as an assessment strategy in Malaysian schools? Share your answer with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  135 7.1.2 Designing Effective Projects There are many types of projects and there is no one correct way to design and implement a project. However, there are some questions and issues to consider when designing effective projects. It is very important for everyone involved to be clear about the goals of the project. You will be surprised that many teachers are not sure why they use projects to assess their learners. Teachers should develop an outline to explain the projectÊs essential elements and expectations for each project. Although the outline can take various forms, it should contain the following elements (Bottoms & Webb, 1998): (a) Situation or Problem The outline should contain a sentence or two describing the issue or problem that the project is trying to address. For example, the pollution levels in rivers, transportation problems in urban centres, the increasing price of essential items, crime rate in squatter areas, youths loitering in shopping complexes and students in Internet cafes during school hours. (b) Project Description and Purpose This refers to a concise explanation of the projectÊs ultimate purpose and how it addresses the situation or problem. For example, learners will research, conduct surveys and make recommendations on how to reduce the pollution of rivers. Results will be presented in a newsletter, information brochure, exhibition or website. (c) Performance Specifications A list of criteria or quality standards the project must meet. (d) Rules Guidelines for carrying out the project include a timeline and short-term goals such as having interviews completed by a certain date and specifying the completion date of the project. (e) List of Project Participants with Roles Assigned Identify the roles of team members and members of the community if they are involved. (f) Assessment How the learnerÊs performance will be evaluated. In project work, the learning process is being evaluated as well as the final product. Copyright © Open University Malaysia (OUM) 136  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Before designing the project, identify the learning goals and objectives. What specific skills or concepts will learners learn? Herman, Aschbacher and Winters (1992) have identified five questions to consider when determining learning goals: (a) What important cognitive skills do I want my learners to develop? (For example, to use algebra to solve everyday problems, to write persuasively); (b) What social and affective skills do I want my learners to develop? (For example, to develop teamwork skills); (c) What metacognitive skills do I want my learners to develop? (For example, to reflect on the research process, to evaluate its effectiveness, to determine methods of improvement); (d) What types of problems do I want my learners to be able to solve? (For example, to know how to conduct research, to apply the scientific method); and (e) What concepts and principles do I want my learners to be able to apply? (For example, to apply basic principles of biology and geography in their lives, to understand cause-and-effect in relationships) Steinberg (1998) provides a checklist for the design of effective projects (refer to Table 7.2). The checklist can be used throughout the process to help both the teacher and learner to plan and develop a project as well to assess whether the project was successful in meeting instructional goals. Table 7.2: The Six AÊs of Effective Project Checklist Six AÊs Questions Checklist Authenticity  Does the project stem from a problem or question that is meaningful to the learner?  Is the project similar to one undertaken by an adult in the community or workplace?  Does the project give the learner the opportunity to produce something that has value or meaning to the learner beyond the school setting? Academic  Does the project enable the learner to acquire and apply knowledge rigor central to one or more discipline areas?  Does the project challenge the learner to use methods of inquiry from one or more disciplines (for example, to think like a scientist)?  Does the learner develop higher-order thinking skills (for example, searching for evidences or using different perspectives)? Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  137 Applied  Does the learner solve a problem that is grounded in real life and/or learning work (for example, design a project or organise an event)?  Does the learner need to acquire and use skills expected in high- performance work environments (for example, teamwork, problem- solving, communication or technology)?  Does the project require the learner to develop organisational and self-management skills? Active  Does the learner spend significant amounts of time doing field work, exploration outside school?  Does the project require the learner to engage in real investigative work, using a variety of methods, media and sources?  Is the learner expected to explain what he or she has learned through a presentation or performance? Adult  Does the learner meet and observe adults with relevant experience relationships and expertise?  Is the learner able to work closely with at least one adult?  Do adults and the learner collaborate on the design and assessment of the project? Assessment  Does the learner reflect regularly on his or her learning, using clear practices project criteria that he or she has helped to set?  Do adults from outside the community help the learner develop a sense of the real world standards from this type of work?  Is the learnerÊs work regularly assessed through a variety of methods including portfolios and exhibitions? Source: Adaptation of Steinberg (1998). Real learning, real work: School-to-work as high school reform Besides those mentioned, it is also important to ensure the following: (a) Do the learners have easy access to the resources they need? This is especially important if a learner is using specific technology or subject-matter expertise from the community; (b) Do the learners know how to use the resources? Learners who have minimal experience with computers, for example, may need extra assistance in utilising them; Copyright © Open University Malaysia (OUM) 138  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (c) Do the learners have mentors or coaches to support them in their work? This can be in-school or out-of-school mentors; and (d) Are learners clear on the roles and responsibilities of each person in the group? SELF-CHECK 7.3 1. What are some of the factors you should consider when designing project work for learners in your subject area? 2. Give examples of projects you have included or can include in the teaching and evaluation of your subject area. 7.1.3 Possible Problems with Project Work Teachers intending to use projects both as an instructional and assessment tool should be aware of certain problem areas. They have to be as specific as possible in determining the outcomes so that both the learner and the teacher understand exactly what is to be learnt. Thomas (1998) identified the following problem areas when undertaking project-based instruction: (a) Aligning project goals with curriculum goals can be difficult. To make matters worse, parents are not always supportive of projects when they cannot see how it relates to the overall assessment of learning; (b) Projects can often take longer than expected and teachers need a lot of time to prepare good authentic projects; (c) Learners are not clear as to what is required. There is need for adequate structure, guidelines and guidance on how to carry out projects; (d) Intensive staff development is required. This is because teachers are not traditionally prepared to integrate content into real-world activities; (e) The resources needed for project work may not be readily available and there might be a lack of administrative support; and (f) Some teachers may not be familiar with how they should assess the projects. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  139 7.1.4 Group Work in Projects Working in groups has become an accepted part of learning due to the widely recognised benefits of collaborative group work for learner learning. When groups work well, learners learn more and produce higher quality learning outcomes. What are some benefits of group work in projects? Let us read the following: (a) Peer Learning Can Improve the Overall Quality of Learner Learning Group work enhances learner understanding. Learners learn from each other and benefit from activities that require them to articulate and test their knowledge. Group work provides an opportunity for learners to clarify and refine their understanding of concepts through discussions and rehearsals with peers. Many, but not all, learners recognise the value to their personal development in group work and of being assessed as a member of the group. Working with a group and for the benefit of the group also motivates some learners. Group assessment helps some learners develop a sense of responsibility: „I felt that because one is working in a group, it is not possible to slack off or to put things off. I have to keep working otherwise I would be letting other people down‰. (b) Group Work Can Help Develop Specific Generic Skills Sought by Employers As a direct response to the objective of preparing graduates with the capacity to function successfully as team members in the workplace, there has been a trend in recent years to incorporate generic skills alongside traditional subject-specific knowledge in the expected learning outcomes in higher education. Group work can facilitate the development of skills which include: (i) Teamwork skills (skills in working within team dynamics; leadership skills); (ii) Analytical and cognitive skills (analysing task requirements; questioning; critically interpreting materials; evaluating the work of others); Copyright © Open University Malaysia (OUM) 140  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (iii) Collaborative skills (conflict management and resolution; accepting intellectual criticism; flexibility; negotiation and compromise); and (iv) Organisational and time management skills: „Having to do group work has changed the way I work. I could not do it all the night before. I had to be more organised and efficient‰. (c) Group Work May Reduce the Workload Involved in Assessing, Grading and Providing Feedback to Learners Group work and group assessment in particular, is sometimes implemented in the hope of streamlining assessment and grading tasks. In simple terms, if learners submit group assignments the number of pieces of work to be assessed can be vastly reduced. This prospect might be particularly attractive for staff teaching large classes. SELF-CHECK 7.4 1. What are some project problems in the implementation of project work and how would you solve them? 2. What are the benefits of group work in projects? 7.1.5 Assessing Project Work Assessing learner performance on project work is quite different from an examination using objective tests and essay questions. It is possible that learners might be working on different projects, for instance, some may be working in groups while others may be working individually. This makes the task of assessing learner progress even more complex compared to a paper and pencil test where everyone is evaluated using one marking scheme. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  141 Table 7.3 could give you some ideas on how to assess and give marks for a project work. Table 7.3: General Marking Scheme for Projects Marks Criteria 90ă100%  Exceptional and distinguished work of a professional standard.  Outstanding technical and expressive skills.  Work demonstrating exceptional creativity and imagination.  Work displaying great flair and originality. 80ă89%  Excellent and highly developed work of a professional standard.  Extremely good technical and expressive skills.  Work demonstrating a high level of creativity and imagination.  Work displaying flair and originality. 70ă79%  Very good work which approaches professional standard.  Very good technical and expressive skills.  Work demonstrating good creativity and imagination.  Work displaying originality. 60ă69%  A good standard of work.  Good technical and expressive skills.  Work displaying creativity and imagination.  Work displaying some originality. 50ă59%  A reasonable standard of work.  Adequate technical and expressive skills.  Work displaying competence in the criteria assessed but which may be lacking some creativity or originality. 40ă49%  Limited but adequate standard of work.  Limited technical and expressive skills.  Work displaying some weaknesses in the criteria assessed and lacking creativity or originality. Copyright © Open University Malaysia (OUM) 142  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS 30ă39%  Limited work which fails to meet the required standard.  Weak technical and expressive skills.  Work displaying significant weaknesses in the criteria assessed. 20ă29%  Poor work. Unsatisfactory technical or expressive skills.  Work displaying significant or fundamental weaknesses in the criteria assessed. 10ă19%  Very poor work or work where very little attempt has been made.  A lack of technical or expressive skills.  Work displaying fundamental weaknesses in the criteria assessed. 1ă9%  Extremely poor work or work where no serious attempt has been made. Source: Chard (1992) When assessing a project work, you need to be clear of what to assess. Is it the product, the process or both? According, to Bonthron and Gordon (1999), from the onset you should be clear: (a) Whether you are going to assess the product of the group work or both product and process. (b) If you intend to assess the process, what proportion are you going to allocate for process and what criteria to use and how are you going to assess the process? (c) What criteria are you planning to use to assess the project work and how will the marks be distributed? Some educators believe there is a need to assess the processes within groups as well as the products or outcomes. What exactly does „process‰ mean? Both teachers and learners must have a clear understanding of what the process means. For example, if you want to assess „'the level of interaction‰ among learners in the group, they should know what „high‰ or „low‰ interaction means. Should the teacher involve himself in the workings of each group or rely on self or peer assessment? Obviously, being involved in many groups would be physically impossible for the teacher. As a result, some educators may say, „I don't care what they do in their groups. All I'm interested in is the final product and how they arrive at their results is their business‰. However, to provide a more balanced assessment, there is growing interest in both the process and product of group work and the issue that arises is „What proportion of assessment should focus on product and what proportion should focus on process?‰ Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  143 The criteria for the evaluation of group work can be determined by the teacher alone or both by the teacher and learners through consultation between the two parties. Group members can be consulted on what should be assessed in a project through consultation with the teacher. Obviously, you have to be clear about the intended learning outcomes of the project in your subject area. It is a useful starting point for determining criteria for assessment of the project. Once these broader learning outcomes are understood, you can establish the criteria for marking the project. Generally, it is easier to establish criteria for measuring the „product‰ of project work and much more difficult to measure the „processes‰ involved in project work. However, it is suggested that evaluation of product and process be done separately rather than attempting to do both simultaneously. We will discuss how the processes in project work may be evaluated later. Another important point to note is that we need to be clear who gets the marks ă individuals or the group as a whole? Most projects involve more than one learner and the benefits of group work have been discussed earlier. A major problem of evaluating projects involving group work is how to allocate marks fairly among group members. The following questions are those mentioned by learners: „I would like my teacher to tell me what amount of work and effort will get enable me to obtain a certain mark‰, „Do all learners obtain the same mark even though not all learners put in the same effort?‰ and „Are marks given based on individual contributions of team members?‰ These are questions that trouble teachers especially when it is common to find freeloaders or sleeping partners in group projects. The following are some suggestions how group work may be assessed: (a) Method 1: Shared Group Mark All group members receive the same mark for the work submitted regardless of individual contribution. It is a straightforward method that encourages group work where group members sink or swim together. However, it may be perceived as unfair by better learners who may complain that they are unfairly disadvantaged by weaker learners and the likelihood of „sleeping partners‰ is very high. (b) Method 2: Share-out Marks The learners in the group decide how the total marks should be shared between them. For example, a score of 40 is given by the teacher for the project submitted. There are 5 members in the group and so the total score possible is 5 x 40 = 200. The learners then share the 200 marks based on the contribution of each of the 4 learners, which may be 35, 45, 42, 38 and 40. This is an effective method if group members are fair, honest and do not have ill- feelings towards each other. However, there is the likelihood for the marks to be equally distributed to avoid ill-feelings among group members. Copyright © Open University Malaysia (OUM) 144  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (c) Method 3: Individual Mark Each learner in the group submits an individual report based on the task allocated or on the whole project. (i) Allocated Task From the beginning, the project is divided into different parts or tasks and each learner in the group completes his allocated task which contributes to the final group product and gets the marks for that task. This method is a relatively objective way of ensuring individual participation and may motivate learners to work hard on their task or part. The problem is breaking up the project into tasks that are exactly equal in size or complexity. Moreover, the method may not encourage group collaboration and some members may slow down the progress. (ii) Individual Report Each learner writes and submits an individual report based on the whole project. The method ensures individual effort and may be perceived as fair by learners. However, it is difficult to determine how the individual reports should differ and learners may unintentionally plagiarise. (d) Method 4: Individual Mark (Examination) Examination questions specifically target the group projects and can only be answered by learners who have been thoroughly involved in the project. This method may motivate learners to learn more from the group project including learning from the other members of the group. However, it may not be effective because learners may be able to answer the questions by reading the group report. In the Malaysian context, national examinations may not be able to include such questions as it involves hundreds of thousands of learners. (e) Method 5: Combination of Group Average and Individual Mark The group mark is awarded to each member with a mechanism for adjusting for individual contributions. This method may be perceived to be fairer than shared group mark. But it means additional challenge for teachers when trying to establish individual contributions. ACTIVITY 7.4 Which of the five methods of assessing group work would you use in assessing project work in your subject area? Give reasons for your choice in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  145 7.1.6 Assessing Process in a Project The assessment of the groupÊs end product is rarely the only assessment taking place in group projects. The process of group work is increasingly recognised as an important element in the assessment of group work. And where group work is marked solely on the basis of product, and not process, there can be differences in individual grading that are deemed unfair and unacceptable. Let us look at other aspects of assessing a project. (a) Peer or Self-evaluation of Roles Learners rate themselves as well as other group members on specific criteria such as responsibilities, contributing ideas and finishing tasks. This can be done through various grading forms (refer to Figure 7.1) or having learners write a brief essay on the groupÊs or membersÊ strengths and weaknesses. (b) Individual Journals Learners keep a journal of events that occur in each group meeting. These include who attended the meeting, what was discussed and plans for future meetings. These can be collected and periodically read by the instructor who comments on the progress. The instructor can provide guidance for the group without directing them. (c) Minutes of Group Meetings Similar to journals are minutes of the group meetings which are periodically read by the instructor. These include who attended, tasks completed, tasks planned and contributors to various tasks. This provides the instructor with a method of monitoring individual contributions within the group. (d) Group and Individual Contribution Grades Instructors can divide the project grade into percentages of individual and group contributions. This is especially beneficial if peer and self-evaluations are used (refer to Figure 7.1). Having a logbook can potentially provide plenty of information to form the basis of assessment while keeping minutes helps members to focus on the process which is a learning experience in itself. These techniques may be perceived as a fair way to deal with freeloaders and outstanding contributors. However, reviewing logs can be time consuming for teachers or instructors and learners may need a lot of training and experience in order to keep the records. In addition, emphasis on second hand evidence may not be reliable. Copyright © Open University Malaysia (OUM) 146  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  147 Figure 7.1: Checklist for evaluating processes involved in project work Source: Developed by Maggie Sutherland for the Biotechnology Academy at Andrew P. Hill High School © 2003 East Side Union High School District, San Jose, California Copyright © Open University Malaysia (OUM) 148  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Self-assessment in Project Work Self-assessment is a process by which learners learn about themselves, for example, what they have learnt about the project, how they have learnt and how they reacted in certain situations when carrying out the project. Involving learners in the assessment process is an essential part of the balanced assessment. When learners become partners in the learning process they gain a better sense of themselves as readers, writers and thinkers. Some teachers may be uncomfortable with self-assessment because traditionally, teachers are responsible for all forms of assessments in the classroom and here we are asking learners to assess themselves. Self-assessment can take many forms: (a) Discussions involving the whole class or small groups; (b) Reflection logs; (c) Self-assessment checklist or inventories; and (d) Teacher-learner interviews. These types of self-assessments share a common theme, which requires learners to review their work to determine what they have learnt and areas of confusion that may still exist. Although each method may differ slightly, they all include enough time for learners to consider thoughtfully and evaluate their own progress. Because project learning is learner-driven, assessment should be learner-driven as well. Learners can keep journals and logs to continually assess their progress. A final reflective essay or log can allow learners and teachers to understand the thinking processes, reasons behind the decisions, ability to arrive at conclusions and communicate what they have learned. According to Edwards (2000), the following are some questions that a learner can ask himself or herself while conducting self-assessment: (a) What were the projectÊs successes? (b) What could I have done to improve the project? (c) How well did I meet my learning goals? What was most difficult about meeting the goals? (d) What surprised me most about working on the project? (e) What was my groupÊs best team effort? Worst team effort? Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  149 (f) How do I think other people involved with the project felt about the progress and end product of the project? (g) What were the skills which I used during this project? How can I engage these skills in the future? SELF-CHECK 7.6 1. Explain how process can be measured in group project work. 2. List some of the problems with assessment of process. ACTIVITY 7.5 Do you think process should be assessed? Explain your answer to your coursemates in the myINSPIRE online forum. 7.2 PORTFOLIO ASSESSMENT Increasingly, portfolio assessment is gaining importance as an assessment strategy seeking to present a more holistic view of the learner. Portfolios tend to be associated with art, where the learner keeps his pieces of work in a type of folder to be presented for evaluation. Some people may associate portfolios with the stock market where a person or organisation keeps a portfolio of stocks and shares owned. Hart (1994) defines a portfolio as a container that holds evidence of an individualÊs skills, ideas, interests and accomplishments (see Figure 7.2). Figure 7.2: Examples of portfolios Copyright © Open University Malaysia (OUM) 150  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS Learner portfolios may take many forms. It is not easy to describe them. A portfolio is not the pile of learnersÊ work that accumulates over a semester or year. Rather a portfolio is a purposeful collection of the works produced by learners which reflects their efforts, progress and achievements in different areas of the curriculum. According to Paulson, Paulson and Meyer (1991), „portfolios offer a way of assessing learner learning that is different from traditional methods. Portfolio assessment provides the teachers an opportunity to observe learners in a broader context which involves taking risks, developing creative solutions and learning to make judgements about their performances‰. 7.2.1 What is Portfolio Assessment? The collection of works by learners are assessed and hence the term portfolio assessment. The portfolio provides for continuous and ongoing assessment (formative assessment) as well as assessment at the end of a semester or a year (summative assessment). Emphasis is more on monitoring learnersÊ progress towards achieving the learning outcomes of a particular subject, course or programme. Portfolio assessment has been described as multidimensional because it allows learners to include different aspects of their work such as essays, project reports, performance on objective tests, objects or artefacts they have produced, poems, laboratory reports and so forth. In other words, the portfolio contains samples of work over an entire semester, term or year, rather than single points in time (such as during examination week). Teachers should convey to learners the purpose of the portfolio, what constitutes quality work and how the portfolio is graded. Portfolio assessment represents a significant shift in thinking about the role of assessment in education. Teachers who use this strategy in the classroom have shifted their philosophy of assessment from merely comparing achievements based on grades, test scores and percentile rankings to improving learner achievements through feedback and self-reflection. 7.2.2 Rationale for Portfolio Assessment In has been frequently suggested that paper and pencil tests (objective and essay tests) are not able to assess all the learning outcomes in a particular subject area. For example, many higher-level cognitive skills and the affective domain (feelings, emotion, attitudes and values) are not adequately assessed using traditional assessment methods. Portfolio assessment allows for the evaluation of a wider range of skills and understanding and most importantly, it provides an opportunity for the teacher to track or monitor change and growth over a period Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  151 of time. Since portfolio assessment is an ongoing process, it provides an opportunity for learners to reflect on their own learning and thinking. They have an opportunity to monitor their understanding and approaches to problem- solving and decision-making (Paulson, Paulson & Meyer, 1991). Upon reflection, learners can identify where they went wrong or how they can further improve themselves. Epstein (2006) in Introduction to Portfolios, Synapse Learning Design, cited in Teachervision.com, argues that portfolio assessment: (a) Allows the teacher to view the learner as an individual, each with his own unique characteristics, needs and strengths; (b) Emphasises improving learner achievement rather than ranking learners according to their performance based on tests; (c) Helps learners to be more accountable for their work; (d) Allows the adaptation of instruction to the learning styles of learners; (e) Involves learners in the assessment process; and (f) Invites learners to reflect upon their growth and performance as learners. However, Epstein (2006) also mentioned some of the problems with portfolio assessments. Portfolio assessments may be less reliable because they tend to be more qualitative rather than quantitative in nature. Society is still strongly oriented towards grades and test scores. In addition, most universities and colleges still use test scores and grades as the main admission criteria. Moreover, portfolio assessment may be time consuming for teachers and data from portfolio assessments can be difficult to analyse. 7.2.3 Types of Portfolios There are two main types of portfolios, namely process-oriented and product- oriented portfolios. (a) Process-oriented Portfolios These portfolios tell a story about the learner and how the learner has grown. It includes earlier drafts and how these drafts have been improved upon. For example, the first draft of a poem written by a Year Three learner reworked based on the comments by the teacher and the learner reflecting on his or her work. All the drafts and changes made are kept in the portfolio. In this manner, learnerÊs works can be compared to and it provides evidence with regard to the how the learnerÊs skills have improved. Copyright © Open University Malaysia (OUM) 152  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (b) Product-oriented Portfolios These portfolios contain the works of a learner which he considers the best. The aim is to document and reflect on the quality of the final products rather than the process that produced them. The learner is required to collect all his work at the end of the semester, at which time he must select those works which is of the highest quality. Learners could be left to make the decision themselves or the teacher can set the criteria on what a portfolio must contain and the quality of the works to be included. 7.2.4 How to Develop a Portfolio The design and development of a portfolio involves four main steps (as stated in Epstein (2006), Introduction to Portfolios, Synapse Learning Design, cited in Teachervision.com). (a) Collection This step simply requires learners to collect and store all of the work. Learners have to get used to the idea of documenting and saving their work, something which they may not have done before. (i) How should the works be organised? According to subjects or themes? (ii) How should the works be recorded and stored? (iii) How to get learners to form the habit of documenting the evidence? (b) Selection This will depend on whether it is a process or product portfolio and the criteria set by the teacher. Learners will go through the work collected and select certain works for their portfolio. This might include examination papers and quizzes, audio and video recordings, project reports, journals, computer work, essays, poems, artwork and so forth. In short, (i) How does one select? What is the basis of selection? (ii) Who should be involved in the selection process? (iii) What are the consequences of not completing the portfolio? Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  153 (c) Reflection This is the most important step in the portfolio process. It is the reflection involved that differentiates the portfolio as being a mere collection of learnerÊs works. Reflection is often done in writing but it can also be done orally. Learners are asked why they have chosen a particular product or work (for example, an essay) and how it compares with other works, what particular skills and knowledge were used to produce it and how it can be further improved. In addition, (i) Learners should reflect on how or why they chose certain works. (ii) How should learners go about the reflection process? (d) Connection As a result of „reflection‰, learners will begin to ask themselves, „Why are we doing this?‰ They are encouraged to make connections between their school work and the value of what they are learning. They are also encouraged to make connections between the works included in their portfolio with the world outside the classroom. They learn to exhibit what they have done in school to the happenings and situations in the community. Issues to consider include: (i) How is the cumulative effect of the portfolio evaluated? (ii) Should learners exhibit their works? 7.2.5 Advantages and Disadvantages of Portfolio Assessment As a formative assessment tool, learner portfolios can be used by the teachers as informal diagnostic techniques or feedback. The feedback enables the learners to reflect on what they are learning and why. Assessment portfolios require learners to continuously reflect and perform self-evaluations of their work. The advantages and disadvantages of portfolio assessment can be summarised as follows: (a) Advantages of Portfolio Assessment The following are several advantages of portfolio assessment: (i) Ongoing assessment is the main benefit of portfolio assessment; (ii) Promoting learner self-evaluation, reflection and critical thinking; Copyright © Open University Malaysia (OUM) 154  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS (iii) Holding learners accountable for mastering content standards in a subject area; (iv) Measuring performance based on genuine samples of learnerÊs work; (v) Providing flexibility in measuring how learners accomplish their learning goals; (vi) Promoting communication between teachers and learners. Enabling teachers and learners to share the responsibility for setting learning goals and for evaluating progress towards meeting those goals; (vii) Giving learners the opportunity to have extensive input into the learning process; and (viii) Facilitating cooperative learning activities including peer evaluation and tutoring, cooperative learning groups and peer conferencing. (b) Disadvantages of Portfolio Assessment There are people who think that portfolios cannot provide trustworthy information needed for sound assessment of learner learning. To them the faults of the portfolio assessments are: (i) Not standardised; (ii) Not feasible for large-scale assessment; and (iii) Potentially biased. In addition, other disadvantages of portfolio assessment according to Venn (2000) are as follows: (i) Requiring extra time to plan an assessment system and conduct the assessment especially for large groups of learners; (ii) Gathering all of the necessary data and work samples can make portfolios bulky and difficult to manage; Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  155 (iii) Scoring portfolios involves extensive use of subjective evaluation procedures such as rating scales and professional judgment which limits reliability; and (iv) Scheduling individual portfolio conference is difficult and the length of each conference may interfere with other instructional activities. 7.2.6 How and When Should a Portfolio be Assessed? If the purpose of the assessment is to demonstrate progress, the teacher could make judgments about the evidence of progress and provide those judgments as feedback to the learner. The learner could also self-assess his progress to check whether his goals have been met or not. The portfolio is more than just a collection of learnerÊs work. The teacher may assess and assign grades to the process of assembling and reflecting upon the portfolio of a learner's work. The learner might have also included reflections on growth, strengths and weaknesses, on goals that were or are to be set, on why certain samples tell certain stories about them or on why the contents reflect sufficient progress to indicate completion of designated standards. Some of the process skills may also be part of the teacher's, school's or district's standards. As such, the portfolio provides some evidence of attainment of those standards. Any or all of these elements can be evaluated and/or graded. Portfolio assignments can also be assessed or graded with a rubric. Rubric is useful in avoiding personal judgment which goes into assessing a complex product such as a portfolio. Rubric can provide some clarity and consistency in assessing and judging the quality of the content and the elements that make up that content. Moreover, application of a rubric increases the likelihood of consistency among teachers who are assessing the portfolios. Copyright © Open University Malaysia (OUM) 156  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS The following portfolio rubric may be used for self-assessment and peer feedback (refer to Figure 7.3). Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  157 Figure 7.3: Sample rubric for self-assessment and peer feedback Source: Adapted from eportfolio (digital portfolio rubric at https://www2.uwstout.edu/content/profdev/rubrics/eportfoliorubric.html) Copyright © Open University Malaysia (OUM) 158  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS SELF-CHECK 7.7 1. What are the strengths and weaknesses of portfolio assessment? 2. What are advantages and disadvantages of portfolio assessment? ACTIVITY 7.6 1. To what extent is portfolio assessment used in Malaysian classrooms? 2. Do you think portfolio assessment can be used as an assessment technique in your subject area? Explain. Share your answers with your coursemates in the myINSPIRE online forum.  Performance-based assessments require test takers to complete a task in a context or setting that closely resembles real-life situation.  A project is an activity in which time constraints has been largely removed and can be undertaken individually or by a group, and usually involves a significant element of work being done at home or out of school.  A research-based project is more theoretical in nature and may consist of asking a question, formulating a problem or setting up some hypotheses.  A product-based project would be the production of a concrete object, a service, a dance performance, a film, an exhibition, a play, a computer programme and so forth.  Project work is a learning experience which enables the development of certain knowledge, skills and attitudes which prepares learners for lifelong learning and the challenges ahead. It involves knowledge application, collaboration, communication and independent learning. Copyright © Open University Malaysia (OUM)  TOPIC 7 PROJECT AND PORTFOLIO ASSESSMENTS  159  An effective project should contain elements such as a situation or problem, project description and purpose, performance specifications, rules, roles of members and assessment.  The Six AÊs of a project are authenticity, academic rigour, applied learning, active exploration, adult relationships and assessment practices.  Working in groups has become an accepted part of learning due to the widely recognised benefits of collaborative group work for learner learning.  Various ways for allocating marks to a project work include shared group marks, shared-out marks, individual mark, individual mark (examination) and combination of group average and individual mark.  Self-assessment is a process by which learners learn about themselves, for example, what they have learned about the project, how they have learnt and how they reacted in certain situations when carrying out the project.  A portfolio is a purposeful collection of the works produced by learners which reflects their efforts, progress and achievements in different areas of the curriculum.  Teachers need to know the benefits and weaknesses of portfolios in order to use them to help in learnersÊ learning. Artefacts Project design Group work Project idea Marks allocation Projects Peer evaluation Research-based project Performance-based assessments Self-evaluation Portfolios Shared group mark Product-based project Shared-out marks Copyright © Open University Malaysia (OUM)