Compliance & Responsible AI Fall 2024 PDF

FALL 2024 Compliance & Responsible AI Session 3: Compliance of AI systems Pr Dr Nathalie DEVILLIER PGE 3A – 2024/2025  Part 2 Course presentation BlackBoard Platform Context and benchmarks The risk-based approach and prohibited systems...

FALL 2024 Compliance & Responsible AI Session 3: Compliance of AI systems Pr Dr Nathalie DEVILLIER PGE 3A – 2024/2025  Part 2 Course presentation BlackBoard Platform Context and benchmarks The risk-based approach and prohibited systems Contact Prof. Dr. Nathalie DEVILLIER Compliance obligations for high-risk systems Transparency of AI systems with limited risk Forum Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 2 1. Context and benchmarks AI Legislation Around the World European Union's Lawfare Strategy The European Regulation on AI: Timeline of application Supervisory Authorities Definitions of AI Systems and Models Pr Dr Nathalie DEVILLIER – Fall 2024 – © aivancity 3 AI Legislation Around the World Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 4 European Union's Lawfare Strategy March 9, 2018 October 19, 2022 September 14, 2022 November 30, 2023 Statement on AI and Digital Services Digital Markets Regulation on digital Robotics Regulation (DSA) Regulation (DMA) resilience (end of trialogue) February 19, 2020 Artificial Intelligence: a European September 2022 June 13, 2024 approach focused on And Proposed guidelines: Adoption of the AI excellence and trust, Report on the safety and liability Extra-contractual civil liability Regulation White paper implications of AI, the Internet of in AI matters Things and robotics - Defective products Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 5 Application timeline 6 months 24 months →2 Feb. 2025 → Aug 2, 2026 Prohibited practices All obligations Entry into force 01/08/2024 12 months 36 months → Aug 2, 2025 → Aug 2, 2027 General-purpose AI Obligations applicable to high-risk models AI systems of art.6-1 Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 6 Supervisory Authorities Member States (France: not yet designated): ○ Application of the AI Regulation and harmonization with peers in the Member States ○ Oversight of the national regulatory sandbox ○ Controls and sanctions EU: ○ AI Office : methodologies assessment and monitoring in conjunction with the authorities national ○ AI Committee : State - driven​members ○ Civil Society Forum ○ Scientific Group of Independents Experts Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 7 Definitions and categories General purpose AI AI system model Definition Art.3(63) Art.3(1) Special categories Systemic risk model 4 levels: risk-based approach Art.2(65) Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 8 AI Systems "an automated system that is designed to operate at varying levels of autonomy and that can exhibit adaptive capability after deployment, and that, for explicit or implicit purposes, infers from the inputs it receives how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments" Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 9 General Purpose AI Models “ an AI model, including when the AI model is trained using large amounts of data using large-scale self-supervision, that exhibits significant generality and is capable of competently performing a wide range of distinct tasks, regardless of how the model is brought to market, and that can be integrated into a variety of downstream systems or applications, except for AI models used for research, development, or prototyping prior to their commercialization” This does not cover AI models that are used before they are put on the market for research, development and prototyping activities (excluded systems). 10 Systemic risk “ a risk specific to high-impact capabilities of general-purpose AI models, having a significant impact on the Union market due to their scale or actual or reasonably foreseeable negative effects on public health, safety, public security, fundamental rights or society at large, which may be propagated on a large scale along the value chain” Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 11 2. The risk approach The level of risk determines the legal qualification and therefore the object of the obligations Risk Practical Obligation Unacceptable Forbidden Deletion High Regulated Compliance Limited Authorized Transparency Minimal Free None (or almost!) Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 12 Risk-based approach: AI systems Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 13 Prohibited AI Practices Placing on the market/into service or use of an AI System which: Uses subliminal techniques, Exploits the vulnerabilities of a person or group of people, Conducts risk assessments to predict the likelihood of an offence being committed Uses real-time biometric identification in a public place for law enforcement purposes outside of specific exemptions (with safeguards) Creates or expands a facial recognition database through non-targeted harvesting of images from the Internet or video surveillance Infers emotions in the workplace or educational setting (except for safety or medical reasons) Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 14 Subliminal or intentionally deceptive techniques Element Meaning System subliminal techniques beyond a person's awareness or intentionally manipulative or deceptive techniques Objective or effect Material alteration of the behavior of a person or group of people by significantly impairing their ability to make informed decisions Result Thus leading the person to make a decision that they would not otherwise have made Additional elements of the significant harm to that person, another person or group of result persons Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 15 Exploitation of vulnerabilities Element Meaning System Exploits a vulnerability of the person or a specific group of people due to their age, health status or specific social or economic situation Objective or effect Having the aim or effect of materially altering their behavior Result [Not applicable] Additional elements of significant harm to that person or any other person the result Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 16 Rating people based on their social behavior Element Meaning System Evaluation or classification of individuals or groups over a period of time based on their known, inferred, or predicted social behavior or personal characteristics Objective or effect Using a social score leading to Result Less favourable or unfavourable treatment of certain individuals or groups of individuals Additional elements of Which is unjustified or disproportionate to their social behavior or the result its seriousness in relation to the social context , OR In social contexts unrelated to the contexts in which the data was originally collected or generated Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 17 3. Compliance obligations: high-risk AI systems Risk management system Appropriate and targeted measures to address identified risks Data and data governance Quality of training data, best practices, relevant and unbiased datasets Technical documentation v. Annexes to the AI Regulation Traceability Archives available throughout the system lifecycle Human supervision Integrated human-machine interfaces to prevent or minimize risks upstream, the user understands, interprets and uses the AI tool with confidence Accuracy, robustness and Constant cybersecurity measures, reporting of accuracy metrics, security resilience against errors and bias handling measures Quality management system Annex Elements : Design, Development, Testing, Risk Management, Post-Market Surveillance, Incident Reporting, Communication, Data Management, Resource Management and Accountability 18 Exceptions Exceptions if they do not pose a significant risk of harm to the health, safety or fundamental rights of individuals : narrow procedural task improve the result of a previously carried out human activity… The Commission must provide guidelines and concrete examples within 18 months of entry into force However, Annex III AI systems carrying out profiling are STILL considered high risk. Risk of fine in case of declaration error Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 19 Presumption of conformity With a harmonized European standard : CEN, Cenelec , ETSI Possibility of relying on codes of good practice to demonstrate compliance with its obligations, until a harmonised standard is published. ○ The Codes are negotiated by stakeholders under the auspices of the European AI Bureau ○ The EC can approve them to give them general validity within the EU Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 20 Supplier Obligations Declaration of conformity For each high-risk system, copy to national authorities, update, retention (10 years) CE marking Visible, legible and indelible or digitally accessible Registration Pre-marketing, company, system, EU database Automatically generated logs 6 months where possible (unless otherwise provided cf GDPR) Immediate corrective actions Withdrawal, deactivation, recall and information of distributors and deployers , investigation of causes, informing authorities Cooperation with the competent Provision of information (compliance with confidentiality authorities obligation) Penalty: €15M or 3% of total turnover Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 21 Models and Open Source Basic model Systemic risk model A. Compliance: Copyright A+B+ Directive Technical documentation Open source B. Detailed summary of training datasets No open source A+B+ A+B+ Technical documentation Technical documentation + Assessment, Risk Mitigation, Serious Incident Reporting, Adversity Testing, Cybersecurity, Energy Consumption Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 22 Obligation: Technical documentation Write and adapt technical documentation on the model, the training and testing process, the results of its evaluation. Provide it to AI systems providers who integrate it into their systems to enable them to have a good understanding of its capabilities and limitations, to comply with their obligations with regard to the risk-based approach Establish an intellectual property rights policy and a summary of the content used for training: See the European AI Bureau model Duty to cooperate with the Commission, the European AI Office and national authorities Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 23 Technical documentation: General description Tasks performed, type and nature of AI systems in which it can be integrated Applicable Acceptable Use Policies Release date and distribution methods Architecture and number of parameters Modality (e.g. text, image) and format of inputs and outputs and Model License Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 24 Technical documentation: Detailed description Technical means (e.g., operating instructions, infrastructure, tools) necessary for the general-purpose AI model to be integrated into AI systems; design specifications and training process, including training methodologies and techniques; key design choices, including rationale and assumptions made; what the model is designed to optimize and the relevance of different parameters, if any; Information about data used for training, testing and validation, Computing resources used to train the model (e.g. number of floating point operations – FLOPs), training time, and other relevant training-related details; Known or estimated energy consumption Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 25 Additional information… Detailed description of evaluation strategies , including evaluation results, based on publicly available evaluation protocols and tools or other evaluation methodologies. Evaluation strategies should include evaluation criteria, measures, and methodology for identifying limitations. If applicable, detailed description of measures put in place for the purpose of conducting internal and/or external adversarial testing (e.g. red teaming), model adaptations, including alignment and fine-tuning. If applicable, a detailed description of the system architecture explaining how the software components build on or feed into each other and integrate into the overall processing. Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 26 4. Transparency of AI systems with limited risk The AI Office encourages and facilitates the development of codes of good practice at Union level to facilitate the effective implementation of obligations on the detection and labelling of artificially generated or manipulated content. Case 1: Emotion recognition or biometric categorization Case 2: Generative AI Pr Dr Nathalie DEVILLIER – Fall 2024 – © aivancity 27 Case 1: Emotion recognition or biometric categorization Inform people that they are exposed to it Obligations arising from the GDPR: Obtain user consent before processing biometric and other personal data Conditions for processing biometric data Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 28 Case 2: Generative AI Generates or manipulates: images, audio, video that resemble existing people, objects, Generates or manipulates text places, events Deep fake To inform the public about matters of public Disclose that the content was artificially interest Reveal that the text was artificially generated or manipulated generated or manipulated In a manner that does not interfere with the Unless the generated content has enjoyment of the work where applicable undergone a human review or editorial control process and a natural or legal person assumes editorial responsibility for the publication of the content. Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity Sanctions Prohibited practices €35M or 7% of total turnover Other shortcomings €15M or 3% of total turnover Inaccurate, incomplete or misleading €7.5M or 1% of total turnover information Moderation for SMEs/start-ups The lowest amount is retained Factors Nature, severity… (mechanism close to the GDPR) Pr Dr Nathalie DEVILLIER – Fall 2024 – ©aivancity 30 Conclusion – Key Takeaways Critical need for collaboration between AI system vendors and regulators to ensure compliance and responsible innovation. Obligation of transparency between the designers of the models vis-à-vis the suppliers of systems integrating them. Act obligations are dynamic: importance of codes of practice and harmonised standards The higher the level of risk, the greater the obligations and the heavier the penalties. Pr Dr Nathalie DEVILLIER – Fall 2024 – © aivancity 31 advance education in artificial intelligence