Add Chapter on APSP. Start Chapter on MST.

57-apsp.tex

+%!tex root = ./skript.tex
+
+\chapter{All-Pairs-Shortest-Paths}
+
+The \emph{all-pairs-shortest-paths} problem is the
+variant of the shortest paths problem were we would
+like to find shortest paths for all pairs $v, w$
+of vertices in a given graph $G = (V, E)$.
+
+More precisely, the problem is as follows: given
+a directed graph $G = (V, E)$ with edge weights $\ell: E \rightarrow \mathbb{R}$
+such that $G$ contains no negative cycles. The goal is to find a shortest path
+tree for every vertex $v \in V$.
+
+Given the discussion in the previous chapters, there is a simple solution to
+this problem: simply run a SSSP-algorithm individually for each node $v \in V$.
+If all weights in $G$ are nonnegative, we can use Dijkstra's algorithm for this,
+and we obtain a total running time of 
+$O(|V| \cdot (|V| \log |V| + |E|) = O(|V|^2\log |V| + |V| \cdot |E|)$. If $G$ is connected,
+then we $|E|$ is between $\Theta (|V|)$ and $\Theta(|V|^2)$.
+Thus, the resulting running time is between $O(|V|^2\log |V|)$ and $O(|V|^3)$, depending
+on the number of edges.
+
+If $G$ has general weights, then we need to use the Bellman-Ford algorithm, resulting
+in a running time of $O(|V| \cdot (|V| \cdot |E|) ) = O(|V|^2|E|)$. Depending on the
+number of edges, this is between $O(|V|^3)$ and $O(|V|^4)$.
+
+We will now consider an algorithm that always requires $O(|V|^3)$ steps, independent
+of the number of edges. This is never worse than running Bellman-Ford for every vertex,
+and it is comparable to using Dijkstra, if the graph is \emph{dense} (i.e., if the graph
+has $\Theta(|V|^2)$ edges. The algorithm is very simple and, unlike Dijkstra's algorithm,
+it does not need any sophisticated data structures.
+
+\paragraph{The algorithm of Floyd-Warshall.}
+The algorithm of Floyd-Warshall uses \emph{dynamic programming} in order to
+solve the APSP-problem in a directed graph $G = (V, E)$ with weights
+$\ell: E \rightarrow \mathbb{R}$.
+The key to a dynamic-programming-solution is to find a suitable recursive
+formulation of the problem. In the Floyd-Warshall algorithm, this is done as
+follows: let $n = |V|$ and suppose that $V = \{1, \dots, n\}$.\footnote{In the
+chapter on graph representation, we have already
+discussed that we can assume that our vertex set consists of numbers; otherwise,
+can simply fix an arbitrary numbering of the vertices and use this to identify them.}
+Then, we define the following subproblems: for any $v, w \in V$, and for
+any $k \in \{0, \dots, n\}$, set 
+\[
+	d_{vw}^{(k)} = \begin{array}{c}
+		\text{the shortest length of a path from $v$ to $w$ that} \\
+	 \text{ uses only intermediate vertices from $\{1, \dots, k\}$}.
+	\end{array}
+\]
+The \emph{intermediate} vertices of a path $\pi$ are all vertices of the path
+except for the first and the last vertex.
+In particular, if $k = 0$, we have $\{1, \dots, k\} = \emptyset$,
+and no intermediate vertices are allowed. 
+If no path from $v$ to $w$ with the desired property exists,
+we define $d_{vw}^{(k)}$ to be $\infty$, 
+Now, we can write down a recursive formula for $d_{vw}^{(k)}$, using recursion
+on $k$. We begin with $k = 0$. In this case, no intermediate vertices are allowed.
+If $v = w$, then $\pi: v$ is a path from $v$ to $v$ of length $0$.
+If $v \neq w$ and if $(v, w) \in V$, then the only path from $v$ to $w$ without
+intermediate vertices consists of the edge $(v, w)$. The length is $\ell(v, w)$.
+Otherwise, if $v \neq w$ and $(v, w) \not\in V$, no such path exists, and the
+length is $\infty$.
+Thus, we have, for all $v, w \in E$:
+\[
+	d_{vw}^{(0)} = \begin{cases}
+      0, & \text{if $v = w$, }\\
+		\ell(v, w), & \text{if $v \neq w$ and $(v, w) \in E$, }\\
+      \infty, & \text{otherwise}.
+    \end{cases}       
+\]
+Next, we discuss how to go from $k - 1$ to $k$. Let $v, w \in V$, and let
+$\pi$ be a shortest path from $v$ to $w$ among all paths that use only
+intermediate vertices from $\{1, \dots, k \}$. Now, there are two
+possibilities: (i) the path $\pi$ does not use the vertex $k$.
+Then, $\pi$ is also a shortest path from 
+$v$ to $w$ among all paths  with
+intermediate vertices from $\{1, \dots, k - 1\}$. Thus, the length of $\pi$ is 
+$d_{vw}^{(k - 1)}$; (ii) the path $\pi$ uses the vertex $k + 1$. Then, $\pi$
+uses $k$ exactly once. By the subpath-optimality-property, we know that
+$\pi$ consists of a shortest path $\pi_1$ from $v$ to $k$, followed by a shortest
+path $\pi_2$ from $k$ to $w$. Crucially, $\pi_1$ and $\pi_2$ do not have $k$
+as an intermediate vertex, but only vertices from $\{1, \dots, k - 1\}$. 
+Thus, we conclude that the length of $\pi_1$ is $d_{vk}^{(k - 1)}$, and the
+length of $\pi_2$ is $d_{kw}^{(k - 1)}$. In total, the length of $\pi$ is
+$d_{vk}^{(k - 1)} + d_{kw}^{(k - 1)}$.
+
+In conclusion, we now have two possibilities for what $d_{vw}^{(k)}$ could be.
+Since we are looking for a shortest path, the true value is the smaller of the two.
+Thus, we have, for all $v, w \in V$, and for all $k \in \{1, \dots n\}$.
+\[
+	d_{vw}^{(k)} = \min \left\{d_{vw}^{(k - 1)}, d_{vk}^{(k - 1)} + d_{kw}^{(k - 1)}\right\}.
+\]
+Once $k = n$, all vertices are allowed as intermediate vertices, Thus, the
+values $d_{vw}^{(n)}$ represent the true distances between the vertices in $G$.
+
+As is common with dynamic programming algorithms, once the recurrence is known, it
+is quite straightforward to write the corresponding code. We just need a program
+that systematically computes all the $d_{vw}^{(k)}$, for $k = 0, \dots, n$.
+We can use a small observation to save space: since $d_{vw}^{(k + 1)}$ relies only on
+the values for $k$, we do not need to store all the previous values. It suffices to
+have an $n \times n$ array that represents the values from the previous round.
+Now, the pseudocode is as follows:
+\begin{verbatim}
+    // initialize an (n x n)-array A with the 
+    // distances for k = 0
+    for i := 1 to n do
+      for j := 1 to n do
+        if i = j then
+          A[i][j] <- 0
+        else if (i, j) is an edge then
+          A[i][j] <- length(i, j)
+        else
+          A[i][j] <- infty
+    for k := 1 to n do
+      // B contains the distances for k - 1,
+      // and our goal is to fill out A with the
+      // distances for k 
+      copy A to B 
+      for i := 1 to n do
+        for j := 1 to n do
+          if B[i][j] <= B[i][k] + B[k][j] then
+            A[i][j] <- B[i][j]
+          else
+            A[i][j] <- B[i][k] + B[k][j]
+    return A
+\end{verbatim}
+The running time of the algorithm is $O(n^3)$, since the main
+work is done in three nested \texttt{for}-loops, each with $n$ iterations.
+Since we use only two $(n \times n)$-arrays, the space is 
+$O(n^2)$.
+
+To obtain the shortest path trees for all the vertices $v \in V$, we can use an analogous
+recursion: for any $v, w \in V$, and for
+any $k \in \{0, \dots, n\}$, set 
+\[
+	\pi_{vw}^{(k)} = \begin{array}{c}
+		\text{the predecessor of $w$ on a shortest from $v$ to $w$ that} \\
+	 \text{uses only intermediate vertices from $\{1, \dots, k\}$}.
+	\end{array}
+\]
+For $k = 0$, there are three cases for two vertices $v, w \in V$: (i) if $v = w$, 
+then the shortest path from $v$ to $v$ consists only of $v$, and there is no predecessor;
+(ii) if $v \neq w$ and $(v, w) \in E$, then the shortest path from $v$ to $w$ with no intermediate vertices
+consists only of the edge $(v, w)$, and the predecessor is $v$; and (iii) if $v \neq w$ and
+$(v, w) \not\in E$, then there is no shortest path with no intermediate vertices. In particular, 
+no predecessor exists. This gives:
+\[
+	\pi_{vw}^{(0)} = \begin{cases}
+		v, & \text{if $v \neq w$ and $(v, w) \in E$, }\\
+      \perp, & \text{otherwise}.
+    \end{cases}       
+\]
+To go from $k - 1$ to $k$, we need to distinguish whether the shortest path from $v$ to
+$w$ with intermediate vertices from $\{1, \dots, k\}$ is obtained by (i) taking the shortest 
+path from $v$ to $w$ with intermediate vertices from $\{1, \dots, k - 1\}$; or by (ii)
+concatenating a shortest path from $v$ to $k$ with a shortest path from $k$ to $w$, both
+with intermediate vertices from $\{1, \dots, k - 1\}$. In  both cases, the predecessor of $w$
+is the same as the predecessor on the shortest path that ends in $w$. We can distinguish
+which case occurs by looking at the $d$-values for $k - 1$. We thus have
+\[
+	\pi_{vw}^{(k)} = \begin{cases}
+		\pi_{vw}^{(k - 1)}, & 
+		\text{if $d_{uv}^{(k - 1)} \leq d_{vk}^{(k - 1)} + d_{kw}^{(k - 1)}$,}\\
+      \pi_{kw}^{(k - 1)}, & \text{otherwise}.
+    \end{cases}       
+\]
+It is straightforward to extend the pseudocode for computing the $\pi$-values:
+\begin{verbatim}
+    // NEW: Add an (n x n)-array C for the predecessors
+    for i := 1 to n do
+      for j := 1 to n do
+        if i = j then
+          A[i][j] <- 0
+          C[i][j] <- NULL
+        else if (i, j) is an edge then
+          A[i][j] <- length(i, j)
+          C[i][j] <- i 
+        else
+          A[i][j] <- infty
+          C[i][j] <- NULL 
+    for k := 1 to n do
+      // D contains the predecessors for k - 1,
+      // and our goal is to fill out C with the
+      // predecessors for k 
+      copy A to B 
+      copy C to D 
+      for i := 1 to n do
+        for j := 1 to n do
+          if B[i][j] <= B[i][k] + B[k][j] then
+            A[i][j] <- B[i][j]
+            C[i][j] <- D[i][j]
+          else
+            A[i][j] <- B[i][k] + B[k][j]
+            C[i][j] <- D[k][j]
+    return A, C
+\end{verbatim}
+The running time is still $O(n^3)$, and the space is still $O(n^2)$.
+
+\textbf{Note}: In \emph{Grundlagen der Theoretischen Informatik}, we will
+see the Algorithm of Kleene. It is closely related to the algorithm of Floyd-Warshall
+and uses a very similar idea to construct an equivalent regular expression for a 
+given deterministic finite automaton (we will learn in GTI what these words mean).
+
+\textbf{TODO}: Add discussion of matrix multiplication, four Russians, fine-grained
+complexity.

58-mst.tex

+%!tex root = ./skript.tex
+
+\chapter{Minimum Spanning Trees}
+

skript.tex

@@ -81,6 +81,8 @@
 \contentUnit{54-sp}
 \contentUnit{55-dijkstra}
 \contentUnit{56-bellmanford}
+\contentUnit{57-apsp}
+\contentUnit{58-mst}
 
 \end{contentPart}