Lecture 2 Artificial Intelligence PDF

Artificial Intelligence Emmanuel Ali(PhD) October 17, 2024 Emmanuel Ali(PhD) 1st Semester October 17, 2024 1 / 47 Outline 1 Introduction to AI 2 Search Algorithms 3 Evolutionary Algorithms 4 Knowledge representation and i...

Artificial Intelligence Emmanuel Ali(PhD) October 17, 2024 Emmanuel Ali(PhD) 1st Semester October 17, 2024 1 / 47 Outline 1 Introduction to AI 2 Search Algorithms 3 Evolutionary Algorithms 4 Knowledge representation and inference 5 Machine Learning Fundamentals Emmanuel Ali(PhD) 1st Semester October 17, 2024 2 / 47 Search algorithms Search algorithms are universal problem-solvers and are applied in a lot of aspects of AI. Search algorithms aids AI agent to explore environments and scenario as the set to achieve their goals. Search algorithms are deplored in a search area, with a start goal and updating the state goal. We consider two types of search algorithms namely: uninformed (blind) and informed(heuristic) search. Emmanuel Ali(PhD) 1st Semester October 17, 2024 3 / 47 Examples Route Planning and Navigation: GPS Navigation: GPS navigation systems use search algorithms to find the most efficient route from a starting point to a destination. Algorithms like A* or Dijkstra’s algorithm are commonly used to calculate the optimal path based on real-time traffic information. Emmanuel Ali(PhD) 1st Semester October 17, 2024 4 / 47 Examples Game Playing: Chess and Go: In board games like chess and Go, AI agents employ search algorithms to explore possible moves and their outcomes. The minimax algorithm, along with techniques like alpha-beta pruning, is used to determine the best move and anticipate the opponent’s responses. Emmanuel Ali(PhD) 1st Semester October 17, 2024 5 / 47 Examples Robotics: Path Planning for Robots: In robotics, search algorithms help robots plan their path through an environment while avoiding obstacles. Algorithms like Rapidly-exploring Random Trees (RRT) are used to create collision-free paths. Emmanuel Ali(PhD) 1st Semester October 17, 2024 6 / 47 Examples Natural Language Processing (NLP): Information Retrieval: In NLP, search algorithms are used in information retrieval systems like search engines. They match user queries with documents in a vast corpus and return relevant results, with algorithms like inverted indexing and ranking based on relevance. Emmanuel Ali(PhD) 1st Semester October 17, 2024 7 / 47 Examples AI for Healthcare: Diagnosis and Treatment Planning: In healthcare, AI systems use search algorithms to explore a patient’s medical history and symptoms to generate a diagnosis or suggest treatment plans. These algorithms consider various diagnostic possibilities and treatment options. Emmanuel Ali(PhD) 1st Semester October 17, 2024 8 / 47 Examples Machine Learning: Hyperparameter Optimization: In machine learning, search algorithms are used to find the best hyperparameters for models. Techniques like grid search and random search explore a parameter space to optimize a model’s performance. Emmanuel Ali(PhD) 1st Semester October 17, 2024 9 / 47 Examples Recommendation Systems: Content Recommendations: Recommendation systems use search algorithms to find and recommend content (e.g., movies, products, or articles) to users. Collaborative filtering and content-based filtering are search-based techniques used for this purpose. Emmanuel Ali(PhD) 1st Semester October 17, 2024 10 / 47 Examples Scheduling and Planning: Job Scheduling: In logistics and manufacturing, search algorithms are used for job scheduling to optimize resource utilization and minimize costs. Algorithms like the Traveling Salesman Problem (TSP) solver help plan efficient routes for deliveries. Emmanuel Ali(PhD) 1st Semester October 17, 2024 11 / 47 Examples Puzzle Solving: Rubik’s Cube Solving: AI algorithms can solve puzzles like the Rubik’s Cube by exploring different sequences of moves to reach the solved state. These algorithms can be based on search techniques. Emmanuel Ali(PhD) 1st Semester October 17, 2024 12 / 47 Examples Internet of Things (IoT): Energy Management: In IoT applications, search algorithms can optimize energy usage in smart homes or industrial settings. They can determine when to turn devices on or off to save energy while meeting user preferences. Emmanuel Ali(PhD) 1st Semester October 17, 2024 13 / 47 Search algorithm terminologies Search Space: The search space represents all the possible states, configurations, or actions that can be explored to find a solution. It typically consists of an initial state, a set of operators or actions that can be applied to the current state, a goal state, and a set of constraints. Initial State: This is the starting point of the search, representing the current situation or configuration from which the search begins. Operators or Actions: Operators are the actions that can be applied to a state to transition from one state to another. These actions may have associated costs or constraints. Goal State: The goal state represents the desired outcome of the search. The primary objective of the search algorithm is to find a sequence of actions that transform the initial state into the goal state. Search Tree or Graph: Search algorithms often construct a search tree or graph, where nodes represent states, and edges represent transitions between states. The search algorithm explores this tree or graph to find a solution. Emmanuel Ali(PhD) 1st Semester October 17, 2024 14 / 47 Search Algorithm Terms Completeness – Is a solution guaranteed to be found if at least one solution exists? Optimality – Is the solution found guaranteed to be the best (or lowest cost) solution if there exists more than one solution? Time Complexity – The upper bound on the time required to find a solution, as a function of the complexity of the problem. Space Complexity – The upper bound on the storage space (memory) required at any point during the search, as a function of the complexity of the problem. Emmanuel Ali(PhD) 1st Semester October 17, 2024 15 / 47 Uninformed search algorithms Uninformed search algorithms, also known as blind search algorithms, are a class of search algorithms used to explore a search space or state space without using any domain-specific knowledge or heuristics. These algorithms make decisions about which nodes to expand and explore based solely on the structure of the search space and without considering the cost or quality of the solutions. Uninformed search algorithms are typically used when little or no information about the problem is available, and their primary goal is to exhaustively search the space to find a solution if it exists. Uninformed search algorithms are useful when the search space is small and the computational resources available are limited. However, they may be inefficient for large, complex search spaces and are not always guaranteed to find optimal solutions. Some examples of informed search algorithms include: Breadth first search, Depth first search, Uniform cost search, Iterative deepening search, and Bidirectional search. Emmanuel Ali(PhD) 1st Semester October 17, 2024 16 / 47 Breadth first search Breadth-First Search (BFS) explores all nodes at the current level before moving to the next level. It guarantees the shortest path to the goal state in terms of the number of actions but can be memory-intensive. Breadth-First Search is a graph traversal and search algorithm used to explore all the vertices (nodes) of a graph in a breadthward motion, starting from a given source vertex. It is used for graph traversal, solving puzzles, shortest path finding, exploring hierarchical data structures like trees, and checking connectivity. BFS guarantees that it finds the shortest path in an unweighted graph because it explores vertices level by level, and the first occurrence of a vertex in the queue corresponds to the shortest path to that vertex. BFS is less suitable for graphs with very high branching factors or when you’re interested in other properties of the graph, like the maximum depth. Emmanuel Ali(PhD) 1st Semester October 17, 2024 17 / 47 Breadth First Search Data Structure: BFS typically employs a queue data structure to keep track of the vertices to be explored. The queue follows the First-In-First-Out (FIFO) principle, which ensures vertices are processed in the order they are added. Initialization: Begin by selecting a starting vertex (the source) from which the search will commence. Place this vertex in the queue and mark it as visited to prevent revisiting. Exploration: While the queue is not empty, perform the following steps: Dequeue the front vertex from the queue. This vertex is the one currently under exploration. Explore all the unvisited neighboring vertices of the current vertex. Add them to the queue and mark them as visited. Continue this process until all reachable vertices from the source have been visited. Termination: The algorithm continues until the queue is empty, meaning all vertices reachable from the source have been visited. If the graph consists of multiple disconnected components, the process may need to be repeated for each unvisited component. Path and Distance: While BFS explores the graph, it also constructs a tree called the BFS tree. This tree represents the shortest paths from the source to all visited vertices. It allows you to find the shortest path from the source to any other vertex by tracing the path back from the destination vertex to the source using parent pointers. Emmanuel Ali(PhD) 1st Semester October 17, 2024 18 / 47 Breadth First Search BFS(graph, start_vertex): Initialize an empty queue. Mark the start_vertex as visited and enqueue it. while the queue is not empty: Dequeue a vertex from the queue. Process the dequeued vertex (e.g., print it). for each adjacent vertex of the dequeued vertex: if it is not visited: Mark it as visited and enqueue it. Emmanuel Ali(PhD) 1st Semester October 17, 2024 19 / 47 Breadth First Search Figure 1: An example of breadth first search Emmanuel Ali(PhD) 1st Semester October 17, 2024 20 / 47 Depth first search Depth-First Search(DFS) explores a branch of the search tree as deeply as possible before backtracking. It’s often implemented using a stack. This often means it will visit nodes farther from the source before visiting nodes closer to the source. DFS may not always find the optimal solution and can get stuck in infinite loops if not properly controlled. It’s not guaranteed to find the shortest path in a graph. It can be applied to various AI tasks, such as searching for solutions in state spaces, maze solving, puzzle solving, game-playing, and any solution that requires exhaustive exploration. Emmanuel Ali(PhD) 1st Semester October 17, 2024 21 / 47 Depth first search Data Structure: DFS is typically implemented using a stack data structure, which can be implemented using a real stack or a recursive function call stack. Initialization: Start at a source vertex or node in the graph or tree. Mark this node as visited. Exploration: Perform the following steps: Explore an adjacent, unvisited node connected to the current node. Move to this new node. Mark the new node as visited. If you cannot move any further (i.e., all adjacent nodes are visited), backtrack to the previous node in the path and continue exploring unvisited neighbors from there. Termination: Continue the process until you have visited all nodes or until you find the target node or a solution to the problem. Emmanuel Ali(PhD) 1st Semester October 17, 2024 22 / 47 Depth first search DFS(graph, start_vertex): Create an empty stack. Push the start_vertex onto the stack. Mark the start_vertex as visited. while the stack is not empty: current_vertex = Pop the top vertex from the stack. Process the current_vertex (e.g., print it). for each unvisited neighbor of current_vertex: Mark the neighbor as visited. Push the neighbor onto the stack. Emmanuel Ali(PhD) 1st Semester October 17, 2024 23 / 47 Depth first search Figure 2: An example of using depth first search Emmanuel Ali(PhD) 1st Semester October 17, 2024 24 / 47 Uniform cost search Uniform Cost Search explores the node with the lowest path cost from the initial state. It’s used when the cost of actions varies and the goal is to find the least costly path. UCS guarantees that it finds the shortest path in a graph, even when edge costs are not uniform. It explores nodes with lower cumulative path costs first, effectively performing a ”greedy” search based on path cost. UCS can be implemented using a priority queue or a data structure that allows efficient retrieval of the vertex with the minimum cost. It can be applied to both directed and undirected graphs. Uniform Cost Search is a vital algorithm for finding the least-cost path in various AI applications, such as route planning, network routing, and navigation systems. It is efficient and ensures the optimal solution, making it a valuable tool for pathfinding in weighted graphs. Emmanuel Ali(PhD) 1st Semester October 17, 2024 25 / 47 Uniform cost search Data Structure: UCS uses a priority queue (or a min-heap) to keep track of the vertices to be explored. The priority queue orders vertices based on their path costs, with the lowest-cost vertices having the highest priority. Initialization: Start with the source vertex and initialize its cost to 0. Place the source vertex in the priority queue with a priority of 0. Exploration: Perform the following steps: i) Dequeue the vertex with the lowest-cost from the priority queue. This vertex is the current node being explored. ii) If the current node is the goal node, the algorithm terminates, and you have found the least-cost path. iii) Otherwise, explore all unvisited neighboring vertices of the current node and calculate their path costs from the source. Enqueue them in the priority queue with their respective priorities (i.e., path costs). Termination: The algorithm continues until you find the goal node, and you have found the least-cost path. If the priority queue becomes empty and the goal node is not reached, it means there is no path from the source to the goal. Emmanuel Ali(PhD) 1st Semester October 17, 2024 26 / 47 Uniform cost search UniformCostSearch(graph, start_vertex, goal_vertex): Initialize a priority queue with the start_vertex and its cost (initialized to 0). Create an empty set to track visited vertices. while the priority queue is not empty: current_vertex, current_cost = Dequeue the vertex with the lowest cost. if current_vertex is the goal_vertex: Return the path to the goal. if current_vertex is not in the visited set: Add current_vertex to the visited set. for each neighbor, cost to neighbor in graph from current_vertex: if neighbor is not in the visited set: Calculate the new_cost as current_cost + cost to neighbor. Enqueue neighbor with new_cost into the priority queue. Return "No path found" (goal is unreachable). Emmanuel Ali(PhD) 1st Semester October 17, 2024 27 / 47 Uniform cost search Figure 3: An example of using Uniform cost search Emmanuel Ali(PhD) 1st Semester October 17, 2024 28 / 47 Iterative deepening search Iterative Deepening Search (IDS) is a search algorithm used in artificial intelligence for solving problems, particularly in scenarios where the depth of the search space (e.g., tree or graph) is unknown. IDS is a combination of two other search techniques: depth-first search (DFS) and breadth-first search (BFS). It aims to find a solution while controlling memory usage and maintaining completeness, which means that it can find a solution if one exists. It guarantees that it will find a solution if one exists (completeness) and will return the optimal solution when the path cost is uniform. Also, it performs a series of depth-limited DFS searches, incrementally increasing the depth limit, which allows it to control memory usage. The time complexity of IDS is slightly higher than BFS but much lower than regular DFS, making it an efficient choice for solving problems in which the depth of the search space is unknown. Iterative Deepening Search is commonly used in game-playing algorithms (e.g., chess and checkers) and other scenarios with large state spaces. Emmanuel Ali(PhD) 1st Semester October 17, 2024 29 / 47 Iterative deepening search Initialization: Start by setting the initial depth limit to 0. Depth-Limited DFS: Perform a depth-limited DFS search with the current depth limit. This means you explore the search space, going as deep as the current limit allows. If the goal state is found at the current depth, the search stops, and a solution is returned. Increment Depth Limit: If the goal is not found within the depth limit, increment the depth limit by 1 and repeat step 2. The search continues to explore deeper levels of the search space. Termination: The search process continues until it finds a solution, at which point it stops and returns the solution path, or until the depth limit becomes greater than the maximum depth of the search space. In this case, the algorithm terminates without finding a solution. Emmanuel Ali(PhD) 1st Semester October 17, 2024 30 / 47 Iterative deepening search IterativeDeepeningSearch(problem): depth_limit = 0 while True: result = DepthLimitedSearch(problem, depth_limit) if result is a solution: return result depth_limit += 1 DepthLimitedSearch(problem, limit): return RecursiveDFS(problem, limit, [], 0) RecursiveDFS(problem, limit, path, depth): if depth > limit: return "cutoff" if problem is a goal: return path for each action in problem.actions(): child = problem.result(action) result = RecursiveDFS(child, limit, path + [action], depth + 1) if result is a solution: return result if result == "cutoff": cutoff = True if cutoff: return "cutoff" else: return "failure" Emmanuel Ali(PhD) 1st Semester October 17, 2024 31 / 47 Iterative deepening search Figure 4: An example of Iterative deepening search Emmanuel Ali(PhD) 1st Semester October 17, 2024 32 / 47 Bidirectional search Bidirectional search is a graph search algorithm used for finding the shortest path between two vertices (or nodes) in a graph. Unlike traditional search algorithms that start from a single source and expand outward, bidirectional search starts simultaneously from both the source and target nodes and seeks to meet in the middle. It is particularly useful when the graph is very large and the goal node is far from the source node, as it can reduce the search space. Bidirectional search aims to reduce the search space and, in some cases, find the shortest path more efficiently by searching from both ends toward the middle. It is particularly useful when the graph is large or when the branching factor (number of neighbors) is high, as it effectively cuts the search space in half. Bidirectional search is most effective when the cost of expanding a node is relatively constant across the graph. Bidirectional search is commonly used in route planning, network routing, and other applications where finding an efficient path between two points is essential. Emmanuel Ali(PhD) 1st Semester October 17, 2024 33 / 47 Bidirectional search Initialization: Start with two queues (one for the forward search from the source and one for the backward search from the target). Both queues start with their respective source and target nodes. Also, create two sets to track visited nodes for both searches. Search Process: Alternately, perform the following steps for the forward and backward searches: i) Dequeue the front node from the queue. ii) Check if this node is in the other search’s visited set. If it is, a meeting point is found, and you can construct the path from the source to the target through this node. iii) If there’s no meeting point, expand the node by generating its neighbors (adjacent nodes) and add them to the queue if they haven’t been visited. Mark them as visited. Termination: The algorithm terminates when you find a meeting point or when both queues are empty, indicating that there’s no path between the source and target. Emmanuel Ali(PhD) 1st Semester October 17, 2024 34 / 47 Bidirectional search BidirectionalSearch(graph, source, target): Create a set for visited nodes in the forward search. Create a set for visited nodes in the backward search. Create a queue for the forward search and enqueue the source. Create a queue for the backward search and enqueue the target. while both forward queue and backward queue are not empty: current_forward_node = Dequeue from the forward queue. current_backward_node = Dequeue from the backward queue. Check if current_forward_node is in the backward search’s visited set. If yes, a meeting point is found; construct the path and return it. Check if current_backward_node is in the forward search’s visited set. If yes, a meeting point is found; construct the path and return it. Expand current_forward_node and enqueue its neighbors (if not visited) to the forward queue. Expand current_backward_node and enqueue its neighbors (if not visited) to the backward queue. Return "No path found" (source and target are not connected). Emmanuel Ali(PhD) 1st Semester October 17, 2024 35 / 47 Informed search Informed search algorithms, also known as heuristic search algorithms, are a class of search algorithms that make use of domain-specific knowledge or heuristics to guide the search for a solution in a more efficient and informed manner. These algorithms consider additional information about the problem, such as estimates of the cost to reach the goal, and use this information to make decisions about which nodes to expand and explore. Informed search algorithms aim to find solutions more quickly and often focus on the most promising areas of the search space. Informed search algorithms are particularly useful when domain-specific knowledge is available and can help guide the search toward promising areas of the search space. They often find solutions more efficiently and quickly compared to uninformed search algorithms but may require careful selection and tuning of heuristics to balance exploration and exploitation. Some examples of informed search algorithms include: A* search, Best first search, Hill climbing search, Beam search, Iterative deepening A*, RBMF, and Simulated Annealing Emmanuel Ali(PhD) 1st Semester October 17, 2024 36 / 47 Best first search Best-First Search is a search algorithm used to explore a search space based on an evaluation function that estimates the desirability of each node or state. It’s often used for tasks such as finding the most promising solutions in a state space, optimizing heuristics, and making informed decisions. This algorithm selects the node that appears closest to the goal state based on a heuristic function. It does not necessarily guarantee the optimal solution. Best-First Search prioritizes nodes based on the evaluation function, which can be tailored to the specific problem and often incorporates a heuristic value. The choice of the evaluation function is critical; it determines the search strategy and influences the algorithm’s effectiveness in finding the best solution. Best-First Search is not guaranteed to find the optimal solution, as it does not consider the cumulative path cost. Best-First Search is a versatile algorithm used in various AI applications, particularly when the objective is to quickly find high-quality solutions or when the search space is too large for exhaustive exploration. Emmanuel Ali(PhD) 1st Semester October 17, 2024 37 / 47 Best first search Data Structure: Best-First Search uses a priority queue (or a similar data structure) to keep track of the nodes or states to be explored. Nodes are prioritized based on an evaluation function, which assigns a value to each node, indicating how promising it is. Nodes with higher values are explored first. Initialization: Start with an initial state or node and enqueue it into the priority queue with its evaluation score. Search Process: Perform the following steps until the priority queue is empty: i)Dequeue the node with the highest evaluation score. This node is the current node being explored. ii)Check if the current node is the goal state. If it is, the search terminates, and the solution is found. iii)If the goal state is not reached, expand the current node by generating its neighbors (successor states) and calculate their evaluation scores using the evaluation function. Enqueue these neighbors into the priority queue. Termination: The algorithm stops when the goal state is reached, and a solution is found, or when the priority queue is empty, indicating that no path to the goal is possible. Emmanuel Ali(PhD) 1st Semester October 17, 2024 38 / 47 Best first search BestFirstSearch(problem): Initialize a priority queue with the initial state and its evaluation score. while the priority queue is not empty: current_state, current_score = Dequeue the state with the lowest score. if current_state is the goal state: Return the solution path. Expand the current state to generate successor states. for each successor_state in successors: Calculate the evaluation score for the successor_state. Enqueue the successor_state with its score into the priority queue. Return "No solution found" (goal state is unreachable). Emmanuel Ali(PhD) 1st Semester October 17, 2024 39 / 47 Best first search Figure 5: An example of using best first search Emmanuel Ali(PhD) 1st Semester October 17, 2024 40 / 47 A* search A* is an informed search algorithm that finds the shortest path from a starting node to a target node in a weighted graph. It is particularly useful when you want to find the optimal path in terms of minimizing the total cost while exploring as few nodes as possible. A* guarantees finding the optimal path with the lowest cost as long as the heuristic function h(n) is admissible (never overestimates the true cost) and the edge costs are non-negative. It combines the advantages of uniform cost search (guaranteed optimality) and greedy search (efficiency). A* search is a powerful algorithm for finding optimal paths in various AI applications, including pathfinding, robotics, game-playing, and route planning.. It efficiently combines information about both the cost to reach a node and an estimate of the cost to reach the goal, allowing it to make informed decisions during the search. Emmanuel Ali(PhD) 1st Semester October 17, 2024 41 / 47 A* search Data Structure: A* uses a priority queue (also known as an open list) to keep track of the nodes to be explored. The priority queue sorts nodes based on a cost function, which is a combination of two values: i)g(n): The cost of the path from the start node to node n. ii)h(n): The estimated cost from node n to the target node (heuristic value). This value is problem-dependent and helps guide the search toward the target efficiently. It should be admissible, meaning it never overestimates the true cost. Initialization: Start with the source node and add it to the priority queue with a cost of g(source) + h(source). Search Process: Continuously perform the following steps until the priority queue is empty: - Dequeue the node with the lowest cost from the priority queue. This node is the current node being explored. - If the current node is the target node, the algorithm terminates, and you have found the optimal path. - Otherwise, expand the current node by generating its neighbors (adjacent nodes) and calculating their costs as g(neighbor) = g(current) + cost(current, neighbor) and h(neighbor). Add these neighbors to the priority queue with their respective costs. Termination: The algorithm stops when you find the target node, and you have the optimal path, or when the priority queue is empty, indicating that there is no path to the target node. Emmanuel Ali(PhD) 1st Semester October 17, 2024 42 / 47 A* Search AStarSearch(graph, start, target): Initialize an empty priority queue with the start node and cost = 0. Create a set to track visited nodes. while the priority queue is not empty: current_node, current_cost = Dequeue the node with the lowest cost. if current_node is the target node: Return the path to the target. if current_node is not in the visited set: Add current_node to the visited set. for each neighbor, edge_cost from current_node: g(neighbor) = current_cost + edge_cost h(neighbor) = calculate_heuristic(neighbor, target) f(neighbor) = g(neighbor) + h(neighbor) Enqueue neighbor with cost f(neighbor) into the priority queue. Return "No path found" (target is unreachable). Emmanuel Ali(PhD) 1st Semester October 17, 2024 43 / 47 A* Search Figure 6: An example using A* search Emmanuel Ali(PhD) 1st Semester October 17, 2024 44 / 47 Frame Title Emmanuel Ali(PhD) 1st Semester October 17, 2024 45 / 47 Frame Title Emmanuel Ali(PhD) 1st Semester October 17, 2024 46 / 47 Frame Title Emmanuel Ali(PhD) 1st Semester October 17, 2024 47 / 47