Editing Kruskal's algorithm

'''Kruskal's algorithm''' is a general-purpose algorithm for the [[minimum spanning tree]] problem, based on the [[disjoint sets data structure]]. The existence of very simple algorithms to maintain disjoint sets in almost [[Asymptotic analysis|constant time]] gives rise to simple implementations of Kruskal's algorithm whose running times are close to linear, usually outperforming [[Prim's algorithm]] in sparse graphs.

=Theory of the algorithm=
Kruskal's may be characterized as a [[greedy algorithm]], which builds the MST one edge at a time. As befits a MST algorithm, the greedy strategy is to continually add the remaining edge of lowest weight. Unlike Prim's, however, Kruskal's adds edges without regard to the connectivity of the partially built MST. We shall assume that a spanning tree exists for the following sections. (If you find them too difficult, skip them.)

==Lemma==
<p>Suppose that a spanning tree <math>T</math> is given of some graph <math>G</math>. Then, the addition of any edge <math>\notin E(T)</math> to <math>E(T)</math>, followed by the removal of any edge from the resulting cycle, yields a spanning tree of <math>G</math>.</p>

<p>''Proof'': Before the operation, the number of vertices is one more than the number of edges. After the operation, this is again true. As the addition of the new edge generates exactly one simple cycle, there are no longer any cycles after an edge on this cycle is removed. So the new <math>T</math> has a vertex count which exceeds its edge count by one and contains no cycles; it must therefore be a tree.</p>

==The algorithm==
<p>We are now ready to present the algorithm. We begin with no knowledge of the edges of the MST, and add them one by one until the MST is complete. To do so we consider edges in increasing order of weight. When considering an edge, if adding it would create a cycle, we skip it; otherwise we add it. Once <math>V-1</math> edges have been added, we have constructed a MST. Kruskal's is both <i>correct</i> (Theorem 2) and <i>complete</i> (Theorem 1).</p>

===Remark===
If adding an edge to the partially built MST generates a cycle, it will also do so if the edge is added to the partially built MST later on in the algorithm, since we only add edges and never remove them. The contrapositive is also true: if adding an edge does not generate a cycle, it wouldn't have generated a cycle if it were added earlier, either.

===Theorem 1===
<p>Kruskal's algorithm will never fail to find a spanning tree in a connected graph.</p>
<p>''Proof'': By contradiction. Assume that all edges have been considered and the partially built tree is still not complete. Then, there exists some edge that connects two vertices in different connected components of the partially built tree. But this edge must have been considered at some point and discarded as adding it would have created a cycle. This is a contradiction, as adding it ''now'' would not create a cycle, ''per'' the Remark above.</p>

===Theorem 2===
<p>Kruskal's algorithm will never produce a non-minimal spanning tree.</p>
<p>''Proof'':<br/>
We first proceed by induction on the number of edges so far added.
* When no edges have yet been added, the set of edges so far added, being empty, is obviously a subset of the edges of some MST.
* Suppose that <math>k</math> edges have been added in this manner (<math> 1 \le k < V </math>), and we know that they form a subset <math>S</math> of the edges of some MST. All further edges to be considered are of greater or equal weight. If adding an edge generates a cycle at this point, it can be discarded, as it can never be added ''per'' the Remark above. According to our algorithm, then, the next edge to be added is the edge <math>e</math> with minimal weight which does not generate a cycle. Consider now any MST <math>T</math> containing the edges so far added as a subset of its edge set. It it contains <math>e</math>, we are done; we know that when <math>e</math> is added our edge set is still a subset of the edge set of some MST. Otherwise, add the edge <math>e</math> to <math>E(T)</math>. A cycle is now generated. This cycle must contain some edge not yet considered, because we know that the edge currently under consideration, in conjunction with the edges previously added, do not form a cycle. This edge not yet considered has a weight greater than or equal to that of <math>e</math>. Then, deleting this edge yields a new spanning tree (''per'' the Lemma above). If the edge deleted was of greater weight than <math>e</math>, the resulting spanning tree has lower weight than <math>T</math>, a contradiction as <math>T</math> was assumed to be minimal. Otherwise, it must have equal weight to <math>e</math>, and deleting it and adding <math>e</math> gives a new spanning tree of equal weight, that is, another MST. So we have proven that given that there is any MST at all containing <math>S</math>, we can add <math>e</math> and find some MST containing this new set as a subset of its own edge set.<br/></p>
Thus, if a spanning tree is built, then its edges form a subset of the edge set of some MST of the graph, and therefore the spanning tree must be that MST itself.

<!--
=Implementation=
As the previous sections are a bit heavy, here is some pseudocode for Prim's algorithm:
<pre>
input G
let u = some vertex ∈ V(G)
let T = ({u},∅)
while V(T) ≠ V(G)
     let (v,w) ∈ E(G) such that v ∈ V(T), w &notin; V(T), and wt(v,w) is minimal
     add w to V(T)
     add (v,w) to E(T)
</pre>
-->

=Implementation=
We have so far glossed over a crucial detail: we must have a means of efficiently deciding whether an edge can be added without generating a cycle, and of adding that edge if it can. To do so, we note that a cycle is created if and only if the two endpoints of the edge are in the same connected component. We could, of course, answer this query through any [[graph search]] algorithm such as [[Depth-first search|DFS]] or [[Breadth-first search|BFS]]. However, that would make the algorithm quadratic time overall, which is undesirable. Instead, we notice that a data structure [[Disjoint sets data structure|already exists]] which can efficiently identify the component containing a vertex and add new edges (joining together components). Assuming that we know how to implement it, then, here is Kruskal's algorithm:
<pre>
input G
let T = (V(G),∅)
for each {u,v} ∈ E(G) in increasing order of wt(u,v)
     if find(u) ≠ find(v)
          unite(u,v)
          add {u,v} to E(T)
</pre>
It would make sense to stop the loop after <math>V-1</math> edges have been added (instead of processing every edge, which is often unnecessary). Nevertheless, it does not affect the asymptotic complexity (see Analysis)

=Analysis=
The usual implementation of Kruskal's sorts the edges by weight first, which takes <math>\mathcal{O}(E\lg E)</math> time. Following this, <math>\mathcal{O}(E)</math> union-find operations are performed. As each can be done in almost-constant time (see [[Disjoint set data structure|the article itself]]), this step requires approximately <math>\mathcal{O}(E)</math> time to complete. The cost of the sort then dominates the running time. As typical implementations of [[quicksort]], usually used here, are faster than those of [[heapsort]], which may be said to operate implicitly in [[Prim's algorithm]], a well-coded Kruskal's will outperform Prim's in sparse graphs. In dense graphs, Prim's non-heap implementation, taking <math>\mathcal{O}(E+V^2)</math> time, is likely to outperform Kruskals, with runtime close to <math>\mathcal{O}(V^2 \lg V)</math>. In-place quicksort takes expected linear stack space, and the disjoint set data structure takes linear extra space, so Kruskal's has a linear memory complexity.

= References =
* Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. ''Introduction to Algorithms'', Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0-262-03293-7. Section 23.2: The algorithms of Kruskal and Prim, pp.567&ndash;574.

[[Category:Algorithms]]
[[Category:Graph theory]]