Advanced Data Structures and Algorithms - Chapter 9
Format: Detailed Markdown Teaching Material
Target Audience: Computer Science Students (Undergraduate to Graduate level)
Duration: 3-4 hours of comprehensive instruction
Table of Contents
- Introduction to Graphs
- Graph Representations
- Graph Traversal Algorithms
- Topological Sorting
- Shortest Path Algorithms
- Minimum Spanning Trees
- Strongly Connected Components
- Network Flow Algorithms
- Advanced Graph Topics
- Implementation Examples
- Practice Problems and Solutions
- Assessment and Evaluation
1. Introduction to Graphs
1.1 What is a Graph?
A graph G = (V, E) is a mathematical structure consisting of: - V (Vertices/Nodes): A finite set of elements - E (Edges): A finite set of pairs of vertices
Visual Representation
A --- B
| |
C --- D
In this example: - V = {A, B, C, D} - E = {(A,B), (A,C), (B,D), (C,D)}
1.2 Types of Graphs
1.2.1 Directed vs Undirected Graphs
Undirected Graph: - Edges have no direction - (A,B) is the same as (B,A) - Example: Social networks (friendship is mutual)
Directed Graph (Digraph): - Edges have direction - (A,B) ≠ (B,A) - Example: Web pages linking to each other
1.2.2 Weighted vs Unweighted Graphs
Unweighted Graph: - All edges have equal importance - Example: Simple road connections
Weighted Graph: - Edges have associated weights (costs, distances, etc.) - Example: Road network with travel times
1.3 Real-World Applications
- Social Networks: Facebook friends, Twitter followers
- Transportation: Flight routes, road networks
- Computer Networks: Internet topology, routing
- Dependencies: Course prerequisites, software dependencies
- Web Structure: Page links, search engines
- Biology: Protein interactions, neural networks
- Economics: Trade relationships, supply chains
1.4 Graph Terminology
- Path: Sequence of vertices connected by edges
- Cycle: Path that starts and ends at the same vertex
- Connected Graph: Path exists between any two vertices
- Strongly Connected: Directed graph where path exists in both directions
- Tree: Connected acyclic graph
- Forest: Collection of trees
- DAG (Directed Acyclic Graph): Directed graph with no cycles
2. Graph Representations
2.1 Adjacency Matrix
A 2D array where A[i][j] = 1 if edge exists between vertices i and j.
Example:
Graph: A-B, A-C, B-D, C-D
A B C D
A [ 0 1 1 0 ]
B [ 1 0 0 1 ]
C [ 1 0 0 1 ]
D [ 0 1 1 0 ]
Advantages:
- O(1) edge lookup
- Simple to implement
- Good for dense graphs
Disadvantages:
- O(V²) space complexity
- Inefficient for sparse graphs
- Wastes space for unconnected vertices
2.2 Adjacency List
Each vertex maintains a list of its neighbors.
Example:
A: [B, C]
B: [A, D]
C: [A, D]
D: [B, C]
Advantages:
- O(V + E) space complexity
- Efficient for sparse graphs
- Easy to iterate over neighbors
Disadvantages:
- O(degree) edge lookup
- More complex to implement
- Not suitable for dense graphs
2.3 Edge List
Simple list of all edges.
Example:
[(A,B), (A,C), (B,D), (C,D)]
When to Use Each Representation:
| Scenario | Best Representation | Reason |
|---|---|---|
| Dense graph, frequent edge queries | Adjacency Matrix | O(1) lookup |
| Sparse graph, memory constrained | Adjacency List | Space efficient |
| Edge-based algorithms (Kruskal's MST) | Edge List | Direct edge access |
| Dynamic graph (frequent additions/deletions) | Adjacency List | Easy modifications |
3. Graph Traversal Algorithms
3.1 Breadth-First Search (BFS)
BFS explores vertices level by level, like ripples in water.
Algorithm Steps:
- Start from source vertex
- Visit all neighbors at distance 1
- Visit all neighbors at distance 2
- Continue until all reachable vertices visited
Detailed Pseudocode:
BFS(G, s):
for each vertex v in G.V:
v.color = WHITE
v.distance = INFINITY
v.parent = NIL
s.color = GRAY
s.distance = 0
s.parent = NIL
Q = empty queue
ENQUEUE(Q, s)
while Q is not empty:
u = DEQUEUE(Q)
for each v in G.Adj[u]:
if v.color == WHITE:
v.color = GRAY
v.distance = u.distance + 1
v.parent = u
ENQUEUE(Q, v)
u.color = BLACK
Time Complexity: O(V + E)
Space Complexity: O(V)
Applications:
- Shortest path in unweighted graphs
- Level-order tree traversal
- Social network analysis
- Web crawling
Example Walkthrough:
Graph: A-B, A-C, B-D, C-D
Starting from A:
Level 0: A
Level 1: B, C
Level 2: D
BFS Tree:
A
/ \
B C
| |
D D
3.2 Depth-First Search (DFS)
DFS explores as far as possible along each branch before backtracking.
Algorithm Steps:
- Start from source vertex
- Visit first unvisited neighbor
- Recursively explore from that neighbor
- Backtrack when no more unvisited neighbors
Detailed Pseudocode:
DFS(G):
for each vertex u in G.V:
u.color = WHITE
u.parent = NIL
time = 0
for each vertex u in G.V:
if u.color == WHITE:
DFS-VISIT(G, u)
DFS-VISIT(G, u):
time = time + 1
u.discover_time = time
u.color = GRAY
for each v in G.Adj[u]:
if v.color == WHITE:
v.parent = u
DFS-VISIT(G, v)
u.color = BLACK
time = time + 1
u.finish_time = time
Time Complexity: O(V + E)
Space Complexity: O(V) for recursion stack
Applications:
- Cycle detection
- Topological sorting
- Finding strongly connected components
- Maze solving
Example Walkthrough:
Graph: A-B, A-C, B-D, C-D
Starting from A:
A (discover: 1) → B (discover: 2) → D (discover: 3)
D (finish: 4) → B (finish: 5) → A
A → C (discover: 6) → D (already visited)
C (finish: 7) → A (finish: 8)
DFS Tree:
A
/ \
B C
|
D
3.3 BFS vs DFS Comparison
| Aspect | BFS | DFS |
|---|---|---|
| Data Structure | Queue | Stack (recursion) |
| Memory Usage | O(V) | O(V) for recursion |
| Shortest Path | Yes (unweighted) | No |
| Cycle Detection | No | Yes |
| Topological Sort | No | Yes |
| Space Complexity | O(V) | O(V) |
4. Topological Sorting
4.1 Definition and Applications
Topological sort is a linear ordering of vertices in a DAG such that for every directed edge (u,v), vertex u comes before v in the ordering.
Real-World Applications:
- Course prerequisites
- Build system dependencies
- Task scheduling
- Package installation order
4.2 Kahn's Algorithm
Algorithm Steps:
- Calculate in-degree for each vertex
- Add all vertices with in-degree 0 to queue
- While queue is not empty: - Remove vertex from queue - Add to topological order - Decrease in-degree of neighbors - Add neighbors with in-degree 0 to queue
Detailed Pseudocode:
TOPOLOGICAL-SORT(G):
// Calculate in-degrees
for each vertex v in G.V:
v.in_degree = 0
for each edge (u,v) in G.E:
v.in_degree = v.in_degree + 1
// Initialize queue with vertices of in-degree 0
Q = empty queue
for each vertex v in G.V:
if v.in_degree == 0:
ENQUEUE(Q, v)
// Process vertices
result = empty list
while Q is not empty:
u = DEQUEUE(Q)
APPEND(result, u)
for each vertex v in G.Adj[u]:
v.in_degree = v.in_degree - 1
if v.in_degree == 0:
ENQUEUE(Q, v)
// Check for cycle
if length(result) != |V|:
return "Graph contains a cycle"
return result
Time Complexity: O(V + E)
Space Complexity: O(V)
4.3 DFS-based Topological Sort
Algorithm:
- Perform DFS on the graph
- When a vertex finishes (turns black), add it to front of result
- Reverse the result to get topological order
Detailed Pseudocode:
DFS-TOPOLOGICAL-SORT(G):
for each vertex u in G.V:
u.color = WHITE
u.parent = NIL
time = 0
result = empty list
for each vertex u in G.V:
if u.color == WHITE:
DFS-VISIT-TOPOLOGICAL(G, u, result)
return REVERSE(result)
DFS-VISIT-TOPOLOGICAL(G, u, result):
u.color = GRAY
for each v in G.Adj[u]:
if v.color == WHITE:
DFS-VISIT-TOPOLOGICAL(G, v, result)
else if v.color == GRAY:
// Back edge found - cycle detected
return "Graph contains a cycle"
u.color = BLACK
PREPEND(result, u)
4.4 Example Walkthrough
Course Prerequisites Example:
Courses: CS101, CS102, CS201, CS301, CS401
Prerequisites:
- CS102 requires CS101
- CS201 requires CS102
- CS301 requires CS201
- CS401 requires CS301
Graph:
CS101 → CS102 → CS201 → CS301 → CS401
Topological Order: [CS101, CS102, CS201, CS301, CS401]
5. Shortest Path Algorithms
5.1 Single-Source Shortest Path Problem
Given a weighted graph G and a source vertex s, find shortest paths from s to all other vertices.
5.2 Dijkstra's Algorithm
When to Use:
- Non-negative edge weights
- Single-source shortest path
- Dense graphs
Algorithm Steps:
- Initialize distances: dist[s] = 0, others = ∞
- Use priority queue (min-heap) to process vertices
- Extract minimum distance vertex
- Relax all edges from that vertex
- Update distances and add to queue
Detailed Pseudocode:
DIJKSTRA(G, w, s):
INITIALIZE-SINGLE-SOURCE(G, s)
S = empty set // vertices whose shortest paths are known
Q = G.V // priority queue
while Q is not empty:
u = EXTRACT-MIN(Q)
S = S ∪ {u}
for each vertex v in G.Adj[u]:
RELAX(u, v, w)
INITIALIZE-SINGLE-SOURCE(G, s):
for each vertex v in G.V:
v.distance = ∞
v.parent = NIL
s.distance = 0
RELAX(u, v, w):
if v.distance > u.distance + w(u,v):
v.distance = u.distance + w(u,v)
v.parent = u
Time Complexity: O((V + E) log V) with binary heap
Space Complexity: O(V)
Example Walkthrough:
Graph with weights:
A --3-- B
| |
1 4
| |
C --2-- D
Starting from A:
Step 1: A (dist=0) → neighbors B(3), C(1)
Step 2: C (dist=1) → neighbor D(3)
Step 3: B (dist=3) → neighbor D(7, but 3+4=7 > 3, so no update)
Step 4: D (dist=3) → no unvisited neighbors
Final distances: A=0, B=3, C=1, D=3
5.3 Bellman-Ford Algorithm
When to Use:
- Negative edge weights allowed
- Detects negative cycles
- Sparse graphs
Algorithm Steps:
- Initialize distances: dist[s] = 0, others = ∞
- Relax all edges |V| - 1 times
- Check for negative cycles
Detailed Pseudocode:
BELLMAN-FORD(G, w, s):
INITIALIZE-SINGLE-SOURCE(G, s)
for i = 1 to |V| - 1:
for each edge (u,v) in G.E:
RELAX(u, v, w)
// Check for negative cycles
for each edge (u,v) in G.E:
if v.distance > u.distance + w(u,v):
return FALSE // Negative cycle detected
return TRUE
Time Complexity: O(VE)
Space Complexity: O(V)
Example with Negative Weights:
Graph:
A --2-- B
| |
-1 3
| |
C --1-- D
After |V|-1 = 3 iterations:
A=0, B=2, C=-1, D=0
No negative cycle detected.
5.4 Floyd-Warshall Algorithm (All-Pairs Shortest Path)
When to Use:
- All-pairs shortest path
- Small graphs (V ≤ 1000)
- Dense graphs
Algorithm:
FLOYD-WARSHALL(W):
n = W.rows
D^(0) = W
for k = 1 to n:
for i = 1 to n:
for j = 1 to n:
D^(k)[i,j] = min(D^(k-1)[i,j], D^(k-1)[i,k] + D^(k-1)[k,j])
return D^(n)
Time Complexity: O(V³)
Space Complexity: O(V²)
6. Minimum Spanning Trees (MST)
6.1 Definition and Properties
A Minimum Spanning Tree of a connected, undirected, weighted graph is a spanning tree with minimum total weight.
Properties:
- MST has exactly V-1 edges
- MST is unique if all edge weights are distinct
- MST connects all vertices with minimum total weight
6.2 Kruskal's Algorithm
Algorithm Steps:
- Sort all edges by weight
- Use Union-Find data structure
- Process edges in order of increasing weight
- Add edge if it doesn't create cycle
Detailed Pseudocode:
KRUSKAL(G, w):
A = empty set
for each vertex v in G.V:
MAKE-SET(v)
sort edges by weight
for each edge (u,v) in sorted order:
if FIND-SET(u) != FIND-SET(v):
A = A ∪ {(u,v)}
UNION(u, v)
return A
Time Complexity: O(E log E) = O(E log V)
Space Complexity: O(V)
6.3 Prim's Algorithm
Algorithm Steps:
- Start with arbitrary vertex
- Grow MST by adding minimum weight edge
- Use priority queue to find minimum edge
- Continue until all vertices included
Detailed Pseudocode:
PRIM(G, w, r):
for each vertex u in G.V:
u.key = ∞
u.parent = NIL
r.key = 0
Q = G.V // priority queue
while Q is not empty:
u = EXTRACT-MIN(Q)
for each v in G.Adj[u]:
if v in Q and w(u,v) < v.key:
v.parent = u
v.key = w(u,v)
Time Complexity: O(E log V) with binary heap
Space Complexity: O(V)
6.4 Example Walkthrough
Kruskal's Algorithm:
Graph:
A --4-- B
| \ |
2 3 1
| \ |
C --2-- D
Edges sorted by weight:
1. (B,D) weight 1
2. (A,C) weight 2
3. (C,D) weight 2
4. (A,B) weight 4
MST: (B,D), (A,C), (C,D)
Total weight: 1 + 2 + 2 = 5
7. Strongly Connected Components (SCCs)
7.1 Definition
A Strongly Connected Component of a directed graph is a maximal set of vertices where every vertex can reach every other vertex.
7.2 Kosaraju's Algorithm
Algorithm Steps:
- Perform DFS and record finish times
- Reverse the graph
- Perform DFS on reversed graph in decreasing finish time order
- Each DFS tree is an SCC
Detailed Pseudocode:
KOSARAJU(G):
// Step 1: First DFS pass
for each vertex v in G.V:
v.color = WHITE
v.parent = NIL
time = 0
finish_order = empty list
for each vertex v in G.V:
if v.color == WHITE:
DFS-FIRST-PASS(G, v, finish_order)
// Step 2: Reverse graph
G_rev = REVERSE-GRAPH(G)
// Step 3: Second DFS pass
for each vertex v in G.V:
v.color = WHITE
v.parent = NIL
sccs = empty list
for each vertex v in REVERSE(finish_order):
if v.color == WHITE:
current_scc = empty list
DFS-SECOND-PASS(G_rev, v, current_scc)
APPEND(sccs, current_scc)
return sccs
Time Complexity: O(V + E)
Space Complexity: O(V)
7.3 Tarjan's Algorithm
Algorithm Steps:
- Use DFS with stack
- Track discovery time and low link
- Identify SCCs when low link equals discovery time
Detailed Pseudocode:
TARJAN(G):
for each vertex v in G.V:
v.index = undefined
v.lowlink = undefined
v.on_stack = false
index = 0
stack = empty stack
sccs = empty list
for each vertex v in G.V:
if v.index == undefined:
STRONGLY-CONNECTED(v)
STRONGLY-CONNECTED(v):
v.index = index
v.lowlink = index
index = index + 1
PUSH(stack, v)
v.on_stack = true
for each w in G.Adj[v]:
if w.index == undefined:
STRONGLY-CONNECTED(w)
v.lowlink = min(v.lowlink, w.lowlink)
else if w.on_stack:
v.lowlink = min(v.lowlink, w.index)
if v.lowlink == v.index:
scc = empty list
repeat:
w = POP(stack)
w.on_stack = false
APPEND(scc, w)
until w == v
APPEND(sccs, scc)
8. Network Flow Algorithms
8.1 Maximum Flow Problem
Given a directed graph with capacities on edges, find the maximum flow from source to sink.
8.2 Ford-Fulkerson Algorithm
Algorithm Steps:
- Initialize flow to 0
- While augmenting path exists: - Find augmenting path using DFS/BFS - Calculate residual capacity - Update flow along path
Detailed Pseudocode:
FORD-FULKERSON(G, s, t):
for each edge (u,v) in G.E:
(u,v).flow = 0
while there exists augmenting path p from s to t:
c_f(p) = min{c_f(u,v) : (u,v) in p}
for each edge (u,v) in p:
(u,v).flow = (u,v).flow + c_f(p)
(v,u).flow = (v,u).flow - c_f(p)
return sum of flows from s
Time Complexity: O(E × max_flow)
Space Complexity: O(V)
8.3 Edmonds-Karp Algorithm
Uses BFS to find augmenting paths, ensuring polynomial time complexity.
Time Complexity: O(VE²)
Space Complexity: O(V)
9. Advanced Graph Topics
9.1 Eulerian Paths and Circuits
Eulerian Circuit:
- Visits every edge exactly once
- Starts and ends at same vertex
- Exists if and only if all vertices have even degree
Eulerian Path:
- Visits every edge exactly once
- May start and end at different vertices
- Exists if and only if exactly 0 or 2 vertices have odd degree
9.2 Hamiltonian Paths and Cycles
Hamiltonian Cycle:
- Visits every vertex exactly once
- Starts and ends at same vertex
- NP-Complete problem
9.3 Graph Coloring
Chromatic Number:
- Minimum number of colors needed to color vertices
- No two adjacent vertices have same color
- NP-Complete for general graphs
10. Implementation Examples
10.1 Graph Class in Python
class Graph:
def __init__(self, directed=False):
self.vertices = {}
self.edges = []
self.directed = directed
def add_vertex(self, vertex):
if vertex not in self.vertices:
self.vertices[vertex] = []
def add_edge(self, u, v, weight=1):
self.add_vertex(u)
self.add_vertex(v)
self.vertices[u].append((v, weight))
self.edges.append((u, v, weight))
if not self.directed:
self.vertices[v].append((u, weight))
def get_neighbors(self, vertex):
return self.vertices.get(vertex, [])
def get_all_vertices(self):
return list(self.vertices.keys())
def get_all_edges(self):
return self.edges
10.2 BFS Implementation
from collections import deque
def bfs(graph, start):
visited = set()
queue = deque([start])
visited.add(start)
result = []
while queue:
vertex = queue.popleft()
result.append(vertex)
for neighbor, _ in graph.get_neighbors(vertex):
if neighbor not in visited:
visited.add(neighbor)
queue.append(neighbor)
return result
10.3 DFS Implementation
def dfs(graph, start, visited=None):
if visited is None:
visited = set()
visited.add(start)
result = [start]
for neighbor, _ in graph.get_neighbors(start):
if neighbor not in visited:
result.extend(dfs(graph, neighbor, visited))
return result
10.4 Dijkstra's Algorithm Implementation
import heapq
def dijkstra(graph, start):
distances = {vertex: float('infinity') for vertex in graph.get_all_vertices()}
distances[start] = 0
previous = {vertex: None for vertex in graph.get_all_vertices()}
pq = [(0, start)]
visited = set()
while pq:
current_distance, current_vertex = heapq.heappop(pq)
if current_vertex in visited:
continue
visited.add(current_vertex)
for neighbor, weight in graph.get_neighbors(current_vertex):
distance = current_distance + weight
if distance < distances[neighbor]:
distances[neighbor] = distance
previous[neighbor] = current_vertex
heapq.heappush(pq, (distance, neighbor))
return distances, previous
10.5 Union-Find for Kruskal's Algorithm
class UnionFind:
def __init__(self, vertices):
self.parent = {v: v for v in vertices}
self.rank = {v: 0 for v in vertices}
def find(self, x):
if self.parent[x] != x:
self.parent[x] = self.find(self.parent[x])
return self.parent[x]
def union(self, x, y):
px, py = self.find(x), self.find(y)
if px == py:
return False
if self.rank[px] < self.rank[py]:
self.parent[px] = py
elif self.rank[px] > self.rank[py]:
self.parent[py] = px
else:
self.parent[py] = px
self.rank[px] += 1
return True
def kruskal(graph):
edges = sorted(graph.get_all_edges(), key=lambda x: x[2])
uf = UnionFind(graph.get_all_vertices())
mst = []
for u, v, weight in edges:
if uf.union(u, v):
mst.append((u, v, weight))
return mst
11. Practice Problems and Solutions
11.1 Basic Problems
Problem 1: Number of Islands
Given a 2D grid of '1's (land) and '0's (water), count the number of islands.
Solution Approach: - Use DFS to explore connected components - Mark visited cells to avoid counting twice
Problem 2: Course Schedule
Given course prerequisites, determine if all courses can be completed.
Solution Approach: - Build directed graph from prerequisites - Use topological sort or cycle detection - Return false if cycle exists
Problem 3: Word Ladder
Find shortest transformation sequence from start word to end word.
Solution Approach: - Model as graph where words are vertices - Edges connect words differing by one character - Use BFS to find shortest path
11.2 Intermediate Problems
Problem 4: Network Delay Time
Find minimum time for signal to reach all nodes in network.
Solution Approach: - Use Dijkstra's algorithm - Return maximum distance from source
Problem 5: Critical Connections
Find edges whose removal increases number of connected components.
Solution Approach: - Use Tarjan's algorithm for bridges - Identify edges not in any cycle
11.3 Advanced Problems
Problem 6: Alien Dictionary
Given sorted words in alien language, determine character order.
Solution Approach: - Build graph from character relationships - Use topological sort
Problem 7: Cheapest Flights Within K Stops
Find cheapest flight path with at most K stops.
Solution Approach: - Modified Dijkstra with stop count constraint - Use dynamic programming approach
12. Assessment and Evaluation
12.1 Learning Objectives Checklist
- [ ] Can define graph terminology and properties
- [ ] Can choose appropriate graph representation
- [ ] Can implement BFS and DFS algorithms
- [ ] Can perform topological sorting
- [ ] Can implement shortest path algorithms
- [ ] Can find minimum spanning trees
- [ ] Can identify strongly connected components
- [ ] Can solve basic network flow problems
12.2 Assessment Methods
Written Exams:
- Algorithm trace problems
- Complexity analysis questions
- Implementation tasks
Programming Assignments:
- Graph algorithm implementations
- Real-world problem solving
- Performance optimization
Oral Presentations:
- Algorithm explanation
- Problem-solving walkthrough
- Complexity analysis
12.3 Common Misconceptions
- BFS vs DFS: Students often confuse when to use each
- Graph Representation: Choosing wrong representation for problem
- Algorithm Complexity: Incorrect analysis of time/space complexity
- Edge Cases: Forgetting to handle disconnected graphs, cycles
- Implementation Details: Off-by-one errors, incorrect data structures
12.4 Teaching Tips
- Visual Learning: Use graph visualizers and animations
- Step-by-Step: Walk through algorithms with small examples
- Real-World Connections: Relate algorithms to familiar applications
- Practice Problems: Provide graduated difficulty levels
- Peer Learning: Encourage algorithm explanation to classmates
13. Resources and References
13.1 Textbooks
- Cormen, Leiserson, Rivest, Stein: "Introduction to Algorithms"
- Weiss: "Data Structures and Algorithm Analysis"
- Sedgewick: "Algorithms in Java/C++"
13.2 Online Resources
- VisuAlgo: https://visualgo.net/
- GeeksforGeeks: Graph algorithms section
- LeetCode: Graph problems
- Khan Academy: Graph theory
13.3 Visualization Tools
- Graphviz: Graph visualization software
- D3.js: Interactive graph visualizations
- Cytoscape: Network analysis platform
14. Conclusion
Graph algorithms form the foundation of many computer science applications. Mastery of these algorithms requires:
- Understanding: Clear grasp of algorithm principles
- Practice: Regular implementation and problem-solving
- Analysis: Ability to analyze time and space complexity
- Application: Recognizing when to apply specific algorithms
This comprehensive guide provides the theoretical foundation, practical implementations, and assessment tools necessary for effective teaching and learning of graph algorithms.
Remember: The key to mastering graph algorithms is consistent practice and understanding the underlying principles rather than memorizing code. Encourage students to think about the problem, choose the right algorithm, and understand why it works.
This teaching guide is designed to be comprehensive yet accessible, providing both theoretical depth and practical implementation guidance for effective graph algorithm education.