lecture 22

CS 332: Algorithms Dijkstra’s Algorithm Disjoint-Set Union

Homework 5 Check web page this afternoon…

Review: Bellman-Ford Algorithm BellmanFord() for each v  V d[v] =  ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v)  E Relax(u,v, w(u,v)); for each edge (u,v)  E if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w Initialize d[], which will converge to shortest-path value  Relaxation: Make |V|-1 passes, relaxing each edge Test for solution: have we converged yet? Ie,  negative cycle?

Review: DAG Shortest Paths Problem: finding shortest paths in DAG Bellman-Ford takes O(VE) time. Do better using topological sort. Idea: if were lucky and processes vertices on each shortest path in order, B-F would be done in one pass Every path in a dag is subsequence of topologically sorted vertex order, so processing verts in that order, we will do each path in forward order (will never relax edges out of vert before doing all edges into vert). Thus: just one pass. Running time: O(V+E)

Review: Dijkstra’s Algorithm If no negative edge weights, we can beat BF Similar to breadth-first search Grow a tree gradually, advancing from vertices taken from a queue Also similar to Prim’s algorithm for MST Use a priority queue keyed on d[v]

Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S U {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); Relaxation Step Note: this is really a call to Q->DecreaseKey() B C D A 10 4 3 2 1 5 Ex: run the algorithm

Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S U {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); How many times is ExtractMin() called? How many times is DecreaseKey() called? What will be the total running time?

Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S U {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); How many times is ExtractMin() called? How many times is DecraseKey() called? A: O(E lg V) using binary heap for Q Can acheive O(V lg V + E) with Fibonacci heaps

Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S U {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); Correctness: we must show that when u is removed from Q, it has already converged

Correctness Of Dijkstra's Algorithm Note that d[v]   (s,v)  v Let u be first vertex picked s.t.  shorter path than d[u]  d[u] >  (s,u) Let y be first vertex  V-S on actual shortest path from s  u  d[y] =  (s,y) Because d[x] is set correctly for y's predecessor x  S on the shortest path, and When we put x into S, we relaxed (x,y), giving d[y] the correct value s x y u p 2 p 2

Correctness Of Dijkstra's Algorithm Note that d[v]   (s,v)  v Let u be first vertex picked s.t.  shorter path than d[u]  d[u] >  (s,u) Let y be first vertex  V-S on actual shortest path from s  u  d[y] =  (s,y) d[u] >  (s,u) =  (s,y) +  (y,u) ( Why? ) = d[y] +  (y,u)  d[y] But if d[u] > d[y], wouldn't have chosen u. Contradiction. s x y u p 2 p 2

Disjoint-Set Union Problem Want a data structure to support disjoint sets Collection of disjoint sets S = {S i }, S i ∩ S j =  Need to support following operations: MakeSet(x): S = S U {{x}} Union(S i , S j ): S = S - {S i , S j } U {S i U S j } FindSet(X): return S i  S such that x  S i Before discussing implementation details, we look at example application: MSTs

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); }

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1? 5 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2? 19 9 1 5 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5? 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17 25 14 8? 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9? 1 5 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13? 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17 25 14? 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17? 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19? 9 1 5 13 17 25 14 8 21 Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17 25 14 8 21? Run the algorithm:

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 1 5 13 17 25? 14 8 21 Run the algorithm:

Correctness Of Kruskal’s Algorithm Sketch of a proof that this algorithm produces an MST for T : Assume algorithm is wrong: result is not an MST Then algorithm adds a wrong edge at some point If it adds a wrong edge, there must be a lower weight edge (cut and paste argument) But algorithm chooses lowest weight edge at each step. Contradiction Again, important to be comfortable with cut and paste arguments

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } What will affect the running time?

Kruskal’s Algorithm Kruskal() { T =  ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } What will affect the running time? 1 Sort O(V) MakeSet() calls O(E) FindSet() calls O(V) Union() calls ( Exactly how many Union()s? )

Kruskal’s Algorithm: Running Time To summarize: Sort edges: O(E lg E) O(V) MakeSet()’s O(E) FindSet()’s O(V) Union()’s Upshot: Best disjoint-set union algorithm makes above 3 operations take O(E  (E,V)),  almost constant Overall thus O(E lg E), almost linear w/o sorting

Disjoint Set Union So how do we implement disjoint-set union? Naïve implementation: use a linked list to represent each set: MakeSet(): ??? time FindSet(): ??? time Union(A,B): “copy” elements of A into B: ??? time

Disjoint Set Union So how do we implement disjoint-set union? Naïve implementation: use a linked list to represent each set: MakeSet(): O(1) time FindSet(): O(1) time Union(A,B): “copy” elements of A into B: O(A) time How long can a single Union() take? How long will n Union()’s take?

Disjoint Set Union: Analysis Worst-case analysis: O(n 2 ) time for n Union’s Union(S 1 , S 2 ) “copy” 1 element Union(S 2 , S 3 ) “copy” 2 elements … Union(S n-1 , S n ) “copy” n-1 elements O(n 2 ) Improvement: always copy smaller into larger Why will this make things better? What is the worst-case time of Union()? But now n Union’s take only O(n lg n) time!

Amortized Analysis of Disjoint Sets Amortized analysis computes average times without using probability With our new Union(), any individual element is copied at most lg n times when forming the complete set from 1-element sets Worst case: Each time copied, element in smaller set 1st time resulting set size  2 2nd time  4 … (lg n)th time  n

Amortized Analysis of Disjoint Sets Since we have n elements each copied at most lg n times, n Union()’s takes O(n lg n) time We say that each Union() takes O(lg n) amortized time Financial term: imagine paying $(lg n) per Union At first we are overpaying; initial Union $O(1) But we accumulate enough $ in bank to pay for later expensive O(n) operation. Important: amount in bank never goes negative

lecture 22

Recommended

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to lecture 22 (20)

More from sajinsc (20)

Recently uploaded (20)

lecture 22