Generative AI in Education Evaluation

• The guide discusses the potential uses of generative AI within the evaluation process in education.

Why This Guide Is Necessary

• Generative artificial intelligence has quickly become part of daily processes involving ICT and other technologies.
• Education is a field where generative AI is expected to have a significant impact.
• Current discussions include evaluation, the role of teachers, and student competencies, with a focus on the appropriate use of AI while maintaining pedagogical orientation.
• Evaluation includes designing assessment and learning activities, planning support actions and feedback, and establishing assessment criteria and instruments.
• These actions require students and teachers to develop and use various skills and competencies related to ICT and digital literacy.
• A thorough review of elements that guarantee the identity and authorship of students' work is also necessary.
• The guide aims to assist teachers in the evaluation process by exploring how generative AI can facilitate or improve associated tasks.
• It provides recommendations to ensure the quality of the learning process when using generative AI for evaluation.

Introduction

• The rise of artificial intelligence (AI) in education is prompting a re-evaluation of traditional educational models based on summative assessment.
• AI can now write texts on specific topics, generate audiovisual content, translate, and generate code, with capabilities expected to grow.
• Formative assessment should be valued.
• Formative assessment occurs throughout learning, providing students with feedback on their competence development and awareness of their learning progress.
• It also helps teachers understand what students are learning and make decisions to guide teaching.
• The concept of plagiarism will be rethought to maintain rigor and reliability in validating student identity and the originality of their work.

Considerations

• Generative AI tool outputs must be reviewed and verified.
• Many tools are still in trial or free versions with limited features compared to paid versions.
• Tools like ChatGPT or Bing Chat can help draft, structure documents, organize ideas, and summarize texts, but are less reliable for information retrieval.
• The recommendations in this guide may need modification to suit specific user needs.
• Specific tools are indicated for certain functionalities.
• European and local data protection authorities do not recommend using ChatGPT and other generative AIs until the legal context is clear.

Evaluation Process

• Evaluation involves designing activities that also serve as learning experiences, where students can develop competencies or demonstrate learning outcomes.

Design of the Evaluation

• Examples of evaluative activities or tests are included
• Information searching, selection, and management: Analyze and evaluate sources located with AI
• Debate or discussion: AI can provide answers to open-ended questions which can be used as a starting point for classroom debate
• Case studies: Design a case, comparing AI's application to identify improvement areas
• Objective tests/questionnaires: Use AI to generate multiple-choice questions for a specific topic.

Criteria and Assessment Tools

• Sharing evaluation criteria with students is crucial for continuous and formative learning.
• AI can help formulate criteria and create instruments like rubrics, observation guides, and checklists.

Guidelines for Evaluating AI-Completed Activities

• Request used prompts from students for verification.
• Consider the accuracy and quality of the AI-generated content, and how the student verified it.
• Evaluate the student's reasoning in using AI and their reasons for choosing specific AI-generated responses.
• Dialogue and debate-based activities help teachers assess skills like expression and how students defend their views based on AI results.

Feedback Strategies

• It involves planning and designing support actions by the teaching team, and the type of feedback to be provided.

Feedback Examples Using AI

• Peer assessment: students use AI to help review others' work.
• Self-assessment: students question AI to compare it to their work if included in the activity.
• Individual/group feedback: teachers use AI insights to improve communication and learn new communication strategies.
• It is recommended to provide support instruments (rubrics, checklists) for peer and self-assessment, with guidance as needed.

Identity and Authorship

• Taking into account the student as author or creator
• Maintain student identity throughout the class and assignments

Aiding Identity Recognition and Authorship

• Introducing AI to detect plagiarism helps in identifying copied text and paraphrasing.
• Tools include natural language processing, neural networks, and machine learning.
• Writing patterns: Use a system with biometric patterns of facial recognition or voice to secure student identity

Activities to Ensure Authorship and Identity

• Portfolios: Facilitate activity tracking for decision making during the learning process
• Mind maps, graphics, and infographics help involve ideas related to content, selection to limit content, and higher mental processing
• Machine correction forces the student to be the teacher and question and format responses in a formative way
• Collaborative work: Group activities reinforce integrity with dialogue, review, and revision
• Continuous feedback improves experience and ensures authorship and identity
• Synchronous tests or interviews: the teacher can personalize the process and test competence

Recommendations

• Design training activities to apply the proposals related to educational design with AI
• Vary activity types to prevent students from delegating entire assignments to AI tools.
• Encourage tasks which promote metacognitive reflection on the activity
• This helps evaluate work that takes into account the processes as well as the end result, with capability to make well-informed decisions
• Use progressive strategies like introducing incremental levels of the design which improve on the same learning content
• In this way, students internalize the knowledge, apply professor feedback, and apply the diverse tasks

Evaluation Criteria

• Ensure that material is original
• Content should have proper citations and references consistent with set standards
• Include any tools like ChatGPT appropriately