AI Skills and Impacts

Coming Up

• April 1 will cover AI Skills, Agentic Solutions architectures, tools, data stores, and Project Walk-through.
• Project Walk-through meetings are scheduled.
• HW 9 and 10 are due on April 4 and are important for your project.
• April 8 will continue Agentic AI, covering the role of data in AI, AI acquisition strategy, AI cloud platforms and services, and AI management issues.
• Remaining project walkthrough will occur
• The interim report is due by midnight.
• Exam 2 is scheduled for April 15, after the exam date will be TBD.
• April 22 will address AI social impacts, AI legal environment, and AI ethics.
• April 29 will feature project presentations.

Understand AI Technical Capabilities

• The AI Index report by Stanford University provides insights.
• Annual updates on AI capabilities are available.
• Access the AI Index report at https://aiindex.stanford.edu/report/.
• Chapter topics include Research and Development, Technical Performance, and Responsible AI.
• Other chapters cover Economy, Science and Medicine, and Education.
• Governance, Diversity, and Public Opinion are also addressed.

AI Cognitive Task Performance

• State-of-the-Art AI has surpassed human-level baselines:
  • In image classification since 2015
  • In basic reading comprehension since 2017
  • In visual reasoning since 2020
  • In natural language inference since 2021
• AI is trailing human baselines in more complex cognitive tasks and expert-level reasoning.
• This includes moral and abstract reasoning tasks.

AI Technical Capabilities

• Language understanding is at T2 and generation is at T3.
• Factuality and truthfulness of AI-generated language is at T4.
• Coding and code generation capabilities are at T5.
• Image Computer Vision and Image Generation are covered, with generation at T6.
• Instruction following, editing, segmentation, and 3D reconstruction at T7 are included.
• Video Computer Vision and Video generation are at T8.
• General reasoning is at T9, while mathematical reasoning is at T10.
• Visual reasoning, moral reasoning (T 11), and causal reasoning are also covered.
• Also audio generation
• Agent capabilities are at T12, both general and task-specific.
• Robotics and reinforcement learning are part of AI technical capabilities.

Week 11 In-Class Exercise

• Open chapter 2 of the AI index report
• Review the report in project teams.
• Prepare a 1-slide presentation on the assigned capability.
• Propose an example of a use case (existing or proposed).
• Submit a group participation assignment by 8 PM today.
• Be ready to present in class at short notice.

What are AI Agents?

• An AI agent is a system or program capable of autonomously performing tasks.
• They do this on behalf of a user or another system by designing its workflow and utilizing available tools.
• AI agents are important because they can handle complex tasks across various applications.
• They can be deployed in enterprise settings like software design, IT automation, code generation, and conversational assistants.
• Large Language Models enable today's agents to understand goals from natural language.
• Agents use provided information, plan actions, use tools/other agents, and combine outputs into responses.

Rise of Agentic AI

• Gartner, on October 1, 2024 projected the future of AI is about agency and productivity.
• Gartner predicts that by 2028, 33% of enterprise software will include agentic AI.
• This is up from less than 1% in 2024, enabling 15% of daily work decisions to be made autonomously.
• McKinsey noted in July 2024 that agents represent the next frontier of generative AI.
• There is an evolution towards gen AI-enabled agents using foundation models.
• These models execute complex workflows in a digital world, moving from thought to action.

Gartner on Agentic AI and the Agency Gap

• Intelligent AI agents are goal-driven software that use AI techniques to complete tasks and achieve goals"
• Current LLM apps are typically "prompted" and respond reactively
• Chat agents like ChatGPT, DALL-E, and Google Bard do not have the agency to plan and act independently.
• A fully mature intelligent agent learns, plans, and acts autonomously.

Benefits of GenAI-Based Agents

• McKinsey & Co stated that agents are the next frontier of generative AI in July 2024.
• Foundation models enable benefits for agents which include the ability to manage multiplicity.
• Can deal with less predictable workflows and exceptions
• Agent systems can be directed with natural language
• Complex workflows can be encoded more easily and learn existing policies/procedures to enable human in the loop
• Agents can work with existing software tools and platforms
• They can learn how to interface with tools through natural language or other interfaces.

Agentic AI Solutions Defined

• Defined as a sequence or flow of activities and tasks performed to achieve a specific organizational goal.
• Process workflow defines the sequence or conditional transitions between the process tasks.
• Promises to deal with unstructured or emergent processes/workflows.

Core Theory of AI Agents

• Agents are anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators
• An agent can be described by its percepts, percept sequences, agent function, agent program and agent actions -Percept - agent's perceptual input at any given instant -Percept sequence - the complete history of everything that the agent has ever perceived -Agent function - maps any given percept sequence to an action -Agent program is a concrete implementation of an agent function -Agent actions - modify the state of the environment and may help collect further information
• Performance measure – this is an evaluative measure (usually in terms of desirability) of a given sequence of environmental states achieved through agent’s actions

Agent Architecture

• Agent equals the agent program, plus the agent architecture
• The agent program maps percepts into actions
• Agent architecture is the environment in which the agent program runs.
• The environment encompasses percepts, actuators, and communication.

Types of Agent Programs

• Simple reflex agents map actions to current percepts without considering past history.
• Model-based reflex agents maintain an internal state using knowledge of the environment.
• Knowledge is used about the states and how the agent's actions affect these states
• Goal-based agents select actions based on achieving specific goals, also consider the environment states.
• Utility agents uses a utility function to represent their performance measure and maximizes expected utility.

Goal-Based Planning

• Enable agents to find actions and a destination
• Goal-based describe it’s situation to allow the agent to do that
• Achieve goals through one or multiple actions
• LLMs effectively make search and planning and development of goal-based agents

Utility Agent Breakdown

• Agents select actions to maximize utilities:
  • Actions select based on utility function - internal representation of performance measurement of the agent

Learning Agents

• Use adaptive learning processes and ML approaches:
  • They subdivided into
    • Learning agent
    • Performance agent
    • Critic
    • Problem generator

LLM-Based AI Agents

• Use LLMs as reasoning engine
• LLMs use comprehension and the capability of NLU
• Agentic technology uses tool calling on backend
• Optimized workflows and achieve complex goals

How LLM-Based AI Agents Work

• They have engines and reasoning and allow effective action planning and search
• Include tools like: -Granite (IBM) -Gemini (Google) -GPT-4 (OpenAI)
• They use a LLM-based reasoning engine - combine goal information and are able to identify the actions to achieve
• Actions may include tool calling and actions for agents
• Orchestration use framework for collaboration an related to worklow
• Equipping with the purpose built for models with a model hub

LLM Agent Architecture

• Memory and cognitive skills combine with the LLM in the reasoning process and affect the decision. The decision feeds back into the LLM
• The tasks affect the LLM which affect the tools
• The LLM Agent results affect the environment:

Agentic Task Agents Architecture

• Handles certain function or issues :
  • Includes all aspects of reasoning and making of decisions
  • Able to map all queries or tasks

Orchestrated RAG Multi Agent Systems

Include coordination and implementation of the Agent system:

• Use as central routing agents such as data an analysis tool

LangChain and Related Applications

• Langchain framework helps developed application use large language models- framework has open source libraries:
• Abstraction and language and 3rd integrations:
  • Package splits some into less weight packages
  • Langgraph help build multi actor apps: use modeling with the edges and graphs

Types of LangChain AI Support

• Router is an LL selection
• Multi step decision allows a sequence
• Tools access allows a selection
• ReAct action combines 3 core concepts
  • memory, tool access and planning