Lecture5 - Constraint Satisfaction Problems.pptx

Introduction to Artificial Intelligence Constraint Satisfaction Problems University of Technology and Applied Sciences Computing and Information Sciences Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS...

Introduction to Artificial Intelligence Constraint Satisfaction Problems University of Technology and Applied Sciences Computing and Information Sciences Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS CSDS3203 2 Defining Constraint Satisfaction Problems A constraint satisfaction problem (CSP) consists of three components: ◦ is a set of variables, ◦ is a set of domains, , one for each variable. ◦ is a set of constraints that specify the allowable combination of values. CCIS@UTAS CSDS3203 3 Solving CSPs CSPs deal with assignment of values to variables. A solution to a CSP is a consistent, complete assignment. A complete assignment is one in which every variable is assigned a value. A consistent assignment is one in which no constraints are violated. A partial assignment is one that leaves some variables unassigned. A partial solution is a partial assignment that is consistent. CCIS@UTAS CSDS3203 4 CSPs vs Search Problems Standard search problem: CSP: A state is a “black box” – any old data A state is a factored representation – support goal test, successor, cost. defined by variables with values from The order of the sequence of actions to domain. reach the goal is important. A goal test is a set of constraints specifying allowable values for variables. We don’t often care about how do we get to the goal, as long as we reach the goal in a reasonable amount of time CCIS@UTAS CSDS3203 5 Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS CSDS3203 6 Example: Map Coloring Variables: Northern Domains: Western Territory Queensland Australia Constraints: adjacent regions must have different colors South Australia ◦ E.g., , or N e w S o u Tasmania t h W a l CCIS@UTAS CSDS3203 e 7 Exercise Find a solution to the Australia’s color Northern Territory mapping problem, i.e., color each province Western Australia Queensland with either red, green or blue so that no two neighboring provinces have the same South Australia color. N e w S o u Tasmania t h W a l CCIS@UTAS CSDS3203 e 8 Map Coloring Solution Northern Territory Western Queensland Australia South Australia N e w S o u t h Solutions are assignments satisfying all constraints, W e.g., a l e s CCIS@UTAS CSDS3203 9 Constraint Graph NT Binary CSP: each constraint relates at Q most two variables. WA Constrain graph: nodes are variables, arcs show constraints. SA NSW General purpose CSP algorithms can use the graph structure to speed up V search. E.g. Tasmania is an independent sub-problem! T CCIS@UTAS CSDS3203 10 Varieties of Variables Discrete variables: ◦ Finite domains: − Assume variable, values, the number of complete assignment is. − E.g., map coloring, 8-queens, … ◦ Infinite domains (integers, strings, etc.) − Need a constraint language − E.g., job scheduling Continuous variables: ◦ Common in operations research: e.g., energy optimization in a production facility. ◦ Linear programming (LP) methods. CCIS@UTAS CSDS3203 11 Varieties of Constraints Unary constraints involve a single variable: ◦ E.g., Binary constraints involve pairs of variables, ◦ E.g., High-order constraints involve 3 or more variables, ◦ E.g., cryptarithmetic column constraints Preferences (soft constraints), e.g., is better then ◦ Often representable by cost for each variable assignment ◦ Constrained optimization problems. CCIS@UTAS CSDS3203 12 Example: Cryptarithmetic Find digit-letter mapping that solves the equation. Variables: X3 X 2 X1 Domains: T Constraints Hypergraph W O Constraints: + T WO F O U R F T U W O R X3 X2 X1 CCIS@UTAS CSDS3203 13 Real-World CSPs Assignment problems ◦ E.g., who teaches what Timetabling problems ◦ E.g., which class is offered when and where Hardware configurations Datacenter layout planning Factory scheduling Logistics and supply chain More example of such CSPs: https://www.csplib.org/ CCIS@UTAS CSDS3203 14 Exercise Formulate the following cryptarithmetic as a CSP. Draw the constraints hyper graph. X3 X2 X1 D I G + D I G H O L E CCIS@UTAS CSDS3203 15 Exercise: Solution Formulate the following cryptarithmetic as a CSP. Draw the constraints hyper graph. X3 X2 X1 D I G + D I G H O D L I E G H O L E Variables: Domains: Constraints: X3 X2 X1 CCIS@UTAS CSDS3203 16 Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS CSDS3203 17 Solving CSPs: Uninformed Search Approach Let us start with the straightforward, naive approach, then fix it. States are defined by values assigned so far, i.e., partial assignments. Solving CSP as a search problem: ◦ Initial state: the empty assignment ◦ Successor function: assign a value to an unassigned variable that does not conflict with current assignment − Fail if no legal values (not fixable!) ◦ Goal test: the current assignment is complete Every solution appears at depth , where is the number of variables ◦ Same as games, use DFS Path is irrelevant, we care about the solution at the end. CCIS@UTAS CSDS3203 18 Backtracking Search Backtracking search (BTS) is a basic uninformed search for CSPs. It uses DFS such that: 1. Assign one variable at a time: assignments are commutative, e.g., is the same as. 2. Check constraints on the go: consider values that don’t conflict with previous assignment. 3. If an assignment fails, backtrack to upper level, and assign other value (expensive). CCIS@UTAS CSDS3203 19 BTS Example NT Q WA Use the BTS to color the shown map. Pick SA NSW variable according to the following order: , and. The order of values is , , and. V T CCIS@UTAS CSDS3203 20 BTS Example NT Q WA SA NSW V T Initialize variables and domains. Nothing is assigned yet. CCIS@UTAS CSDS3203 21 BTS Example NT Q WA SA NSW V T Assign R to WA. Assignment consistent. Recuse. CCIS@UTAS CSDS3203 22 BTS Example NT Q WA SA NSW V T Assign R to NT. Assignment inconsistent. Continue next value. CCIS@UTAS CSDS3203 23 BTS Example NT Q WA SA NSW V T Assign G to NT. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 24 BTS Example NT Q WA SA NSW V T Assign R to SA. Assignment inconsistent. Continue next value. CCIS@UTAS CSDS3203 25 BTS Example NT Q WA SA NSW V T Assign G to SA. Assignment inconsistent. Continue next value. CCIS@UTAS CSDS3203 26 BTS Example NT Q WA SA NSW V T Assign B to SA. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 27 BTS Example NT Q WA SA NSW V T Assign R to Q. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 28 BTS Example NT Q WA SA NSW V T Assign R to V. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 29 BTS Example NT Q WA SA NSW V T Assign R to NSW. Assignment inconsistent. Continue next value. CCIS@UTAS CSDS3203 30 BTS Example NT Q WA SA NSW V T Assign G to NSW. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 31 BTS Example NT Q WA SA NSW V T Assign R to T. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 32 BTS Example NT Q WA SA NSW V T Assignment complete. Return  CCIS@UTAS CSDS3203 33  BTS Solution Tree NT Q WA SA NSW V T BTS Exercise NT Q WA Use the BTS to color the shown map. Pick SA NSW variable according to the following order: , and. The order of values is , , and. V T CCIS@UTAS CSDS3203 35 Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS CSDS3203 36 Improving BTS BTS can fail to assign value to a variable. Hence need to backtrack. ◦ Expensive in large or complex CSPs Can we detect inevitable failure early? ◦ Identify potential conflicts before they arise. ◦ Help prune unnecessary explorations of doomed paths. The order of picking variables and values makes a difference ◦ How to prioritize? CCIS@UTAS CSDS3203 37 Improving BTS Heuristics are back! We will look at two approaches: 1. Domain Update: detect inevitable failure early. ◦ Forward checking (FC) ◦ Arc consistency (AC3 Algorithm) - see the book ◦ Also called constraint propagation. 2. Dynamic Ordering: ◦ Which variables should be assigned next? Most Constrained Variable (MCV) heuristic. ◦ In what order should it’s values be tried? Least Constraining Value (LCV). CCIS@UTAS CSDS3203 38 Backtracking Search Algorithm def backtracking_search(csp): return backtrack({}, csp) def backtrack(assignment, csp): if is_complete(assignment, csp): # If the assignment is complete return assignment # return the assignment var = select_unassigned_variable(assignment, csp) # Select next unassigned variable for value in order_domain_values(var, assignment, csp): # Iterate over each value if is_consistent(var, value, assignment, csp): # value is consistent assignment[var] = value # assign it result = backtrack(assignment.copy(), csp) # recurse if result != None: # If result is not a None, return result # return it # remove value form it from the assignment del assignment[var] CCIS@UTAS CSDS3203 39 return None Domain Update Key idea: Forward checking provides one-step lookahead. After assigning a variable , eliminate inconsistent values from the domains of ’s neighbors. If any domain becomes empty, don’t recurse, continue with next values. NT NT Q Q Assign Q value B WA WA SA NSW SA NSW V V T T CCIS@UTAS CSDS3203 40 BTS+FC Example NT Q WA Use the BTS+FC to color the shown map. SA NSW Pick variable according to the following order: , and. The order of values is , , and. V T CCIS@UTAS CSDS3203 41 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Initialize variables and domains. Nothing is assigned yet. CCIS@UTAS CSDS3203 42 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign R to WA. Assignment consistent. Update NT, SA domains. Recurse. CCIS@UTAS CSDS3203 43 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign G to NT. Assignment consistent. Update SA, Q domains. Recurse. CCIS@UTAS CSDS3203 44 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign B to SA. Assignment consistent. Update Q, NSW, V domains. Recurse. CCIS@UTAS CSDS3203 45 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign R to Q. Assignment consistent. Update NSW domain. Recurse. CCIS@UTAS CSDS3203 46 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign R to V. Assignment consistent. Update NSW domain. Recurse. CCIS@UTAS CSDS3203 47 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign G to NSW. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 48 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assign R to T. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 49 Backtracking Search (With Forward Checking) NT Q WA SA NSW V T Assignment complete. Return  CCIS@UTAS CSDS3203 50 BTS +FC Solution Tree NT Q WA SA NSW V T BTS+FC Exercise NT Q WA Use the BTS with Forward Checking (FC) to color the shown map. Pick variable SA NSW according to the following order: , and. The order of values is , , and. V T CCIS@UTAS CSDS3203 52 Outline 1. Definition of CSPs 2. CSP Examples 3. Backtracking Search 4. Domain Update 5. Dynamic Ordering CCIS@UTAS CSDS3203 53 Variable and Value Ordering The algorithms we considered so far require the order in which variables or values picked to be specified. Choosing the right order of variables (and values) can noticeably improve the efficiency of the CSP algorithm. Dynamic ordering: let the algorithm decide the next variable or value to be picked for consideration. To implement dynamic ordering, we use two heuristics: ◦ Most constraint variable (MCV) for variable ordering − Also called Minimum Remaining Value (MRV). ◦ Least constraining value (LCV) for value ordering. Note, these heuristics are useful iff domain update is used. Why? CCIS@UTAS CSDS3203 54 Most Constrained Variable (MCV) Key Idea: Choose the variable with the fewest legal values in its domain. NT Q Which variable should WA be assigned first? SA NSW V T CCIS@UTAS CSDS3203 55 BTS + Variable Ordering Example NT Q WA Use the BTS, with variable ordering to color the shown map. Pick variables SA NSW according to the MCV heuristic. Assign WA first. The order of values is , , and. V T CCIS@UTAS CSDS3203 56 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Initialize variables and domains. Nothing is assigned yet. CCIS@UTAS CSDS3203 57 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign R to WA. Assignment consistent. Update NT, SA domains. Recurse. CCIS@UTAS CSDS3203 58 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign G to NT. Assignment consistent. Update SA, Q domains. Recurse. CCIS@UTAS CSDS3203 59 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign B to SA. Assignment consistent. Update Q, NSW, V domains. Recurse. CCIS@UTAS CSDS3203 60 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign R to Q. Assignment consistent. Update NSW domain. Recurse. CCIS@UTAS CSDS3203 61 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign G to NSW. Assignment consistent. Update V domain. Recurse. CCIS@UTAS CSDS3203 62 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign R to V. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 63 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assign R to T. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 64 Backtracking Search (With Variable Ordering) NT Q WA SA NSW V T Assignment complete. Return  CCIS@UTAS CSDS3203 65 BTS + Variable Ordering, Exercise NT Q WA Use the BTS, with variable ordering to color the shown map. Pick variables SA NSW according to the MCV heuristic. Assign SA first. The order of values is , , and. V T CCIS@UTAS CSDS3203 66 Least Constraining Value (LCV) NT Q WA SA NSW What value should we assign to Q? V T CCIS@UTAS CSDS3203 67 Least Constraining Value (LCV) NT Q WA Assigning R to Q leaves 2 + 2 + 2 = 6 SA NSW consistent values. V T CCIS@UTAS CSDS3203 68 Least Constraining Value (LCV) NT Q WA Assigning B to Q leaves 1 + 1 + 2 = 4 SA NSW consistent values. V Which value is better for Q? R or B? Why? T CCIS@UTAS CSDS3203 69 Least Constraining Value (LCV) Key Idea: Given a variable , pick the value that rules out the fewest choices for the neighboring variables. So, this heuristic allows you to pick the value which gives maximum flexibility for subsequent variable assignment. CCIS@UTAS CSDS3203 70 BTS + Value Ordering Example NT Q Use the BTS, with variable ordering to WA color the shown map. Pick the variables SA NSW according to the following order: WA, NT, SA, NSW, Q, V, and T. The order of values should be according to the LCV heuristic. V T CCIS@UTAS CSDS3203 71 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: WA V Consistent values: R=4, G=4, B=4 T Initialize variables and domains. Nothing is assigned yet. CCIS@UTAS CSDS3203 72 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: NT V Consistent values: G=3, B=3 T Assign R to WA. Assignment consistent. Update NT, SA domains. Recurse. CCIS@UTAS CSDS3203 73 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: SA V Consistent values: B=5 T Assign G to NT. Assignment consistent. Update SA, Q domains. Recurse. CCIS@UTAS CSDS3203 74 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: NSW V Consistent values: R=1, G=2 T Assign B to SA. Assignment consistent. Update Q, NSW,V domains. Recurse. CCIS@UTAS CSDS3203 75 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: Q V Consistent values: R=0 T Assign G to NSW. Assignment consistent. Update Q, V domains. Recurse. CCIS@UTAS CSDS3203 76 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: V V Consistent values: R=0 T Assign R to Q. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 77 Backtracking Search (With Value Ordering) NT Q WA SA NSW Next variable: T V Consistent values: R=0, G=0, B=0 T Assign R to V. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 78 Backtracking Search (With Value Ordering) NT Q WA SA NSW V T Assign R to T. Assignment consistent. Recurse. CCIS@UTAS CSDS3203 79 Backtracking Search (With Value Ordering) NT Q WA SA NSW V T Assignment complete. Return  CCIS@UTAS CSDS3203 80 Homework NT Q WA Use the BTS with forward checking (FC). Pick the variables according to the MCV SA NSW heuristic. The order of the values should be to the LCV heuristic. V T CCIS@UTAS CSDS3203 81 Recommended Reading For more on the topics covered in this lecture please refer to the following sources: Russell-Norvig Book (Russell & Norvig, 2020): Sections 5.1 – 5.3. CCIS@UTAS CSDS3203 82 References Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach (4th Edition). Pearson. http://aima.cs.berkeley.edu/ CCIS@UTAS CSDS3203 83