Compliance & Responsible AI Fall 2024

Compliance & Responsible AI

• Presented by Dr. Nathalie Devillier
• Fall 2024 session
• Focuses on the compliance of AI systems

Course Presentation

• Covers context and benchmarks
• Explores risk-based approach and prohibited systems
• Outlines compliance obligations for high-risk systems
• Discusses transparency of AI systems with limited risk

Context and Benchmarks

• Examines AI legislation globally
• Presents the European Union's Lawfare Strategy
• Details the European Regulation on AI, including its timeline
• Discusses supervisory authorities and definitions of AI systems and models

AI Legislation Around the World

• Presents a global tracker map
• Illustrates jurisdictions in focus (e.g., Australia, Brazil, Canada)

European Union's Lawfare Strategy

• Provides a timeline, including key dates for statements on AI and robotics; digital services regulation; digital markets regulation; regulation on digital resilience (end of trialogue); artificial intelligence; and others

Application Timeline

• Shows phased implementation for AI regulation
• Introduces various deadlines concerning prohibited practices and all obligations

Supervisory Authorities

• Highlights the role of member states (France is not yet designated) in applying AI regulation
• Outlines oversight of national regulatory sandboxes
• Mentions controls and sanctions
• Describes the EU perspective on the Al Office, methodologies for assessment and monitoring, national authorities' role, and the development of the Al Committee

Definitions and Categories

• Details general purpose AI model (Art. 3(63)) and AI system (Art. 3(1)) definitions
• Explains systemic risk model (Art. 2(65)) 
• Explains risk-based approach in 4 levels

AI Systems

• Defines AI systems as automated systems designed for varying autonomy levels
• Explicit or implicit inputs determine outputs like predictions, content, recommendations, or decisions influencing physical or virtual environments

General Purpose AI Models

• Defines general purpose AI models trained with large datasets, using large-scale self-supervision and exhibiting wide task generality
• Excludes systems used for research, development, or prototyping before commercial launch

Systemic Risk

• Defines systemic risk as high-impact capabilities of general-purpose AI models, significantly impacting the Union market due to their scale or potential for foreseeable negative consequences in public health, safety, public security, fundamental rights or society

The Risk Approach

• Links risk levels to practical applications & legal obligations
• Categorizes risk levels from unacceptable (forbidden actions) to minimal (no or almost no obligations)

Risk-based Approach: AI Systems - EU Artificial Intelligence Act

• Classifies risk levels with examples: unacceptable (social scoring, manipulation), high risk, limited risk, minimal risk
• Relates risks to prohibited, conformity assessment, transparency obligations, and no obligation categories

Prohibited AI Practices

• Outlines practices prohibited in AI systems (e.g., subliminal techniques, exploitation of vulnerabilities, risk assessments for predicting offenses, real-time biometric identification in public places, database expansion via non-targeted harvesting of data)
• Includes limitations for workplace or educational settings (except for safety/medical reasons)

Subliminal or Intentionally Deceptive Techniques

• Describes the meaning of subliminal and intentional deceptive techniques
• Explains the objective effect (material alteration of behavior) and result (decision-making impairment), with potential harm to individuals

Exploitation of Vulnerabilities

• Explains how AI systems exploit vulnerabilities due to age, health, or economic standing
• Focuses on activities aiming to alter target behavior or causing significant harm

Rating People Based on Social Behavior

• Explains how systems evaluate or classify individuals based on characteristics using a social score
• Highlights potential for less favorable or unfavorable treatment which might be unjustified or disproportionate for the original data

Compliance Obligations: High-Risk AI Systems

• Lists management systems for high-risk AI, including data and data governance, technical documentation, traceability, human oversight, accuracy & robustness, and security plus quality management
• Explains that appropriate and targeted measures need to address the identified risks

Exceptions

• Identifies situations where AI systems, such as those with specific tasks, may not pose significant risk of harm to individual safety.
• States that Annex III systems performing profiling still fall within the high-risk category.

Presumption of Conformity

• Describes the possibility of harmonized EU standards to demonstrate compliance
• Highlights the use of codes of good practice

Supplier Obligations

• Enlists requirements for suppliers of high-risk AI systems, including declaration of conformity, CE marking, registration, logs, corrective actions, cooperation with authorities, information provision, and potential penalties

Models and Open Source

• Explains differences between basic and systemic risk models, particularly documentation requirements: copyright and training datasets for open-source; technical documentation for others

Obligation: Technical Documentation

• Emphasizes the need to provide clear documentation concerning models (training, testing, evaluation results)
• Explains obligations of Al providers integrating models, intellectual property policy, and summary of training content
• Outlines cooperation with the Commission, European AI Office, and national authorities for compliance

Technical Documentation: General Description

• Covers essential elements like tasks, acceptable use policies, release dates, architecture, modality, and model licenses

Technical Documentation: Detailed Description

• Outlines detailed technical requirements based on means, design specifications, training process, and information concerning data usage, resources, training times, operations, and estimations of energy consumption and other relevant details

Additional Information

• Outlines evaluations and adversarial testing processes
• Explains system architecture and how components interact.

Transparency of AI Systems with Limited Risk

• Describes the Al Office's focus on developing codes of good practice for transparency
• Presents examples of case studies (emotion recognition/biometric categorization and generative AI) as specific issues needing attention

Case 1: Emotion Recognition or Biometric Categorization

• Discusses obligations arising from the GDPR for transparency regarding emotion recognition and biometric data processing, such as user consent and data processing conditions.

Case 2: Generative AI

• Highlights obligations pertaining to the disclosure of artificially generated or manipulated content
• Includes measures for transparency, human review, or legal person control

Sanctions

• Lists financial penalties for various violations related to prohibited practices, shortcomings in information accuracy, and moderation practices for SMEs

Conclusion - Key Takeaways

• Underscores collaboration between AI vendors and regulators for responsible AI practices
• Emphasizes the obligation of transparency between designers and suppliers
• Highlights that AI obligations are dynamic and harmonization, especially in codes of practice, is important.
• Links penalties to the level of risk in AI systems