Lecture 6: Introduction to Graphs PDF

Analysis of Algorithms Lecture 6: Introduction to Graphs 11/26/2022 Eng. Aboulfateh Al-Dhabei 1  Uniform-profit Restaurant location problem Goal: maximize number of restaurants open Subject to: distance constraint (min-separation >= 10)...

Analysis of Algorithms Lecture 6: Introduction to Graphs 11/26/2022 Eng. Aboulfateh Al-Dhabei 1  Uniform-profit Restaurant location problem Goal: maximize number of restaurants open Subject to: distance constraint (min-separation >= 10) 5 2 2 6 6 3 6 10 7 d 0 5 7 9 15 6 9 15 10 7 0 0 0 11/26/2022 Eng. Aboulfateh Al-Dhabei 2  Events scheduling problem Goal: maximize number of non-conflicting events e6 e8 e3 e1 e4 e5 e7 e9 e2 Time 11/26/2022 Eng. Aboulfateh Al-Dhabei 3  Fractional knapsack problem item Weight Value $ / LB Goal: maximize value (LB) ($) without exceeding 1 2 2 1 bag capacity 2 4 3 0.75 Weight limit: 10LB 3 3 3 1 4 5 6 1.2 5 2 4 2 6 6 9 1.5 11/26/2022 Eng. Aboulfateh Al-Dhabei 4  Example item Weight Value $ / LB Goal: maximize value (LB) ($) without exceeding 5 2 4 2 bag capacity 6 6 9 1.5 Weight limit: 10LB 4 5 6 1.2 1 2 2 1 2 + 6 + 2 = 10 LB 3 3 3 1 4 + 9 + 2.4 = 15.4 2 4 3 0.75 11/26/2022 Eng. Aboulfateh Al-Dhabei 5  The remaining lectures Graph algorithms Very important in practice – Tons of computational problems can be defined in terms of graphs – We’ll study a few interesting ones Minimum spanning tree Shortest path Graph search Topological sort, connected components 11/26/2022 Eng. Aboulfateh Al-Dhabei 6  Graphs A graph G = (V, E) – V = set of vertices – E = set of edges = subset of V  V – Thus |E| = O(|V|2) 1 Vertices: {1, 2, 3, 4} Edges: {(1, 2), (2, 3), (1, 3), (4, 3)} 2 4 3 11/26/2022 Eng. Aboulfateh Al-Dhabei 7  Graph Variations (1) Directed / undirected: – In an undirected graph: Edge (u,v)  E implies edge (v,u)  E Road networks between cities – In a directed graph: Edge (u,v): uv does not imply vu Street networks in downtown – Degree of vertex v: The number of edges adjacency to v For directed graph, there are in-degree and out-degree 11/26/2022 Eng. Aboulfateh Al-Dhabei 8  1 1 2 4 2 4 3 In-degree = 3 3 Degree = 3 Out-degree = 0 Directed Undirected 11/26/2022 Eng. Aboulfateh Al-Dhabei 9  Graph Variations (2) Weighted / unweighted: – In a weighted graph, each edge or vertex has an associated weight (numerical value) E.g., a road map: edges might be weighted w/ distance 1 1 0.3 2 4 2 1.2 4 0.4 1.9 3 3 11/26/2022 Unweighted Eng. Aboulfateh Al-DhabeiWeighted 10  Graph Variations (3) Connected / disconnected: – A connected graph has a path from every vertex to every other – A directed graph is strongly connected if there is a directed path between any two vertices 1 2 4 Connected but not strongly connected 11/26/2022 Eng. Aboulfateh Al-Dhabei 11 3  Graph Variations (4) Dense / sparse: – Graphs are sparse when the number of edges is linear to the number of vertices |E|  O(|V|) – Graphs are dense when the number of edges is quadratic to the number of vertices |E|  O(|V|2) – Most graphs of interest are sparse – If you know you are dealing with dense or sparse graphs, different data structures may make sense 11/26/2022 Eng. Aboulfateh Al-Dhabei 12  Representing Graphs Assume V = {1, 2, …, n} An adjacency matrix represents the graph as a n x n matrix A: – A[i, j] = 1 if edge (i, j)  E = 0 if edge (i, j)  E For weighted graph – A[i, j] = wij if edge (i, j)  E = 0 if edge (i, j)  E For undirected graph – Matrix is symmetric: A[i, j] = A[j, i] 11/26/2022 Eng. Aboulfateh Al-Dhabei 13  Graphs: Adjacency Matrix Example: A 1 2 3 4 1 1 2 4 2 3 ?? 3 4 11/26/2022 Eng. Aboulfateh Al-Dhabei 14  Graphs: Adjacency Matrix Example: A 1 2 3 4 1 1 0 1 1 0 2 4 2 0 0 1 0 3 0 0 0 0 3 4 0 0 1 0 How much storage does the adjacency matrix require? 11/26/2022 A: O(V2) Eng. Aboulfateh Al-Dhabei 15  Graphs: Adjacency Matrix Example: A 1 2 3 4 1 1 0 1 1 0 2 4 2 1 0 1 0 3 1 1 0 1 3 4 0 0 1 0 Undirected graph 11/26/2022 Eng. Aboulfateh Al-Dhabei 16  Graphs: Adjacency Matrix Example: A 1 2 3 4 1 5 1 0 5 6 0 2 6 4 2 5 0 9 0 9 4 3 6 9 0 4 3 4 0 0 4 0 Weighted graph 11/26/2022 Eng. Aboulfateh Al-Dhabei 17  Graphs: Adjacency Matrix Time to answer if there is an edge between vertex u and v: Θ(1) Memory required: Θ(n2) regardless of |E| – Usually too much storage for large graphs – But can be very efficient for small graphs Most large interesting graphs are sparse – E.g., road networks (due to limit on junctions) – For this reason the adjacency list is often a more appropriate representation 11/26/2022 Eng. Aboulfateh Al-Dhabei 18  Graphs: Adjacency List Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: – Adj = {2,3} 1 – Adj = {3} – Adj = {} 2 4 – Adj = {3} Variation: can also keep 3 a list of edges coming into vertex 11/26/2022 Eng. Aboulfateh Al-Dhabei 19  Graph representations Adjacency list 1 2 3 3 2 4 3 3 How much storage does the adjacency list require? A: O(V+E) 11/26/2022 Eng. Aboulfateh Al-Dhabei 20  Graph representations Undirected graph 1 A 1 2 3 4 1 0 1 1 0 2 4 2 1 0 1 0 3 1 1 0 1 3 4 0 0 1 0 2 3 1 3 1 2 4 11/26/2022 Eng. Aboulfateh Al-Dhabei 21 3  Graph representations Weighted graph A 1 2 3 4 1 1 0 5 6 0 5 2 5 0 9 0 6 2 4 3 6 9 0 4 9 4 4 0 0 4 0 3 2,5 3,6 1,5 3,9 1,6 2,9 4,4 11/26/2022 Eng. Aboulfateh Al-Dhabei 22 3,4  Graphs: Adjacency List How much storage is required? For directed graphs – |adj[v]| = out-degree(v) – Total # of items in adjacency lists is  out-degree(v) = |E| For undirected graphs – |adj[v]| = degree(v) – # items in adjacency lists is  degree(v) = 2 |E| So: Adjacency lists take (V+E) storage Time needed to test if edge (u, v)  E is O(n) 11/26/2022 Eng. Aboulfateh Al-Dhabei 23  Tradeoffs between the two representations |V| = n, |E| = m Adj Matrix Adj List test (u, v)  E Θ(1) O(n) Degree(u) Θ(n) O(n) Memory Θ(n2) Θ(n+m) Edge insertion Θ(1) Θ(1) Edge deletion Θ(1) O(n) Graph traversal Θ(n2) Θ(n+m) Both representations are very useful and have different properties. 11/26/2022 Eng. Aboulfateh Al-Dhabei 24  Minimum Spanning Tree Problem: given a connected, undirected, weighted graph: 6 4 5 9 14 2 10 15 3 8 11/26/2022 Eng. Aboulfateh Al-Dhabei 25  Minimum Spanning Tree Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 6 4 5 9 A spanning tree is a tree that connects all 14 2 vertices 10 Number of edges = ? 15 A spanning tree has no designated root. 3 8 11/26/2022 Eng. Aboulfateh Al-Dhabei 26  How to find MST? Connect every node to the closest node? – Does not guarantee a spanning tree 11/26/2022 Eng. Aboulfateh Al-Dhabei 27  Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees – Let T be an MST of G with an edge (u,v) in the middle – Removing (u,v) partitions T into two trees T1 and T2 – w(T) = w(u,v) + w(T1) + w(T2) Claim 1: T1 is an MST of G1 = (V1, E1), and T2 is an MST of G2 = (V2, E2) Proof by contradiction: T1 T2 if T1 is not optimal, we can replace T1 with a better spanning tree, T1’ T1’ u v T1’, T2 and (u, v) form a new spanning tree T’ 11/26/2022 W(T’) < W(T). Contradiction. 28 Eng. Aboulfateh Al-Dhabei  Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees – Let T be an MST of G with an edge (u,v) in the middle – Removing (u,v) partitions T into two trees T1 and T2 – w(T) = w(u,v) + w(T1) + w(T2) Claim 2: (u, v) is the lightest edge connecting G1 = (V1, E1) and G2 = (V2, E2) Proof by contradiction: if (u, v) is not the lightest edge, we T1 T2 can remove it, and reconnect T1 x y and T2 with a lighter edge (x, y) u v T1, T2 and (x, y) form a new spanning tree T’ 11/26/2022 W(T’) < W(T). Contradiction. 29 Eng. Aboulfateh Al-Dhabei  Algorithms Generic idea: – Compute MSTs for sub-graphs – Connect two MSTs for sub-graphs with the lightest edge Two of the most well-known algorithms – Prim’s algorithm – Kruskal’s algorithm – Let’s first talk about the ideas behind the algorithms without worrying about the implementation and analysis 11/26/2022 Eng. Aboulfateh Al-Dhabei 30  Prim’s algorithm Basic idea: – Start from an arbitrary single node A MST for a single node has no edge – Gradually build up a single larger and larger MST 6 5 Not yet discovered 7 Fully explored nodes 11/26/2022 Eng. Aboulfateh Al-Dhabei 31 Discovered but not fully explored nodes  Prim’s algorithm Basic idea: – Start from an arbitrary single node A MST for a single node has no edge – Gradually build up a single larger and larger MST 6 2 5 9 4 Not yet discovered 7 Fully explored nodes 11/26/2022 Eng. Aboulfateh Al-Dhabei 32 Discovered but not fully explored nodes  Prim’s algorithm Basic idea: – Start from an arbitrary single node A MST for a single node has no edge – Gradually build up a single larger and larger MST 6 2 5 9 4 7 11/26/2022 Eng. Aboulfateh Al-Dhabei 33  Prim’s algorithm in words Randomly pick a vertex as the initial tree T Gradually expand into a MST: – For each vertex that is not in T but directly connected to some nodes in T Compute its minimum distance to any vertex in T – Select the vertex that is closest to T Add it to T 11/26/2022 Eng. Aboulfateh Al-Dhabei 34  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 35  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 36  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 37  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 38  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 39  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 40  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d 11/26/2022 Eng. Aboulfateh Al-Dhabei 41  Example a 6 12 5 9 b f g 14 7 8 15 c e h 3 10 d Total weight = 3 + 8 + 6 + 5 + 7 + 9 + 15 = 53 11/26/2022 Eng. Aboulfateh Al-Dhabei 42  Kruskal’s algorithm Basic idea: – Grow many small trees – Find two trees that are closest (i.e., connected with the lightest edge), join them with the lightest edge – Terminate when a single tree forms 11/26/2022 Eng. Aboulfateh Al-Dhabei 43  Claim If edge (u, v) is the lightest among all edges, (u, v) is in a MST Proof by contradiction: – Suppose that (u, v) is not in any MST – Given a MST T, if we connect (u, v), we create a cycle – Remove an edge in the cycle, have a new tree T’ – W(T’) < W(T) By the same argument, the second, third, …, lightest u v edges, if they do not create a cycle, must be in MST 11/26/2022 Eng. Aboulfateh Al-Dhabei 44  Kruskal’s algorithm in words Procedure: – Sort all edges into non-decreasing order – Initially each node is in its own tree – For each edge in the sorted list If the edge connects two separate trees, then – join the two trees together with that edge 11/26/2022 Eng. Aboulfateh Al-Dhabei 45  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 46  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 47  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 48  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 49  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 50  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 51  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 52  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 53  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 54  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 55  Example c-d: 3 b-f: 5 a b-a: 6 6 12 5 9 f-e: 7 b f g b-d: 8 14 7 8 15 f-g: 9 c e h d-e: 10 3 10 a-f: 12 d b-c: 14 e-h: 15 11/26/2022 Eng. Aboulfateh Al-Dhabei 56