Barrier Synchronization

Making sure everyone finishes Phase 1 before anyone starts Phase 2.

The idea

Imagine a MapReduce job or a parallel physics simulation. You split the work among 4 threads. But Phase 2 of the calculation requires the combined results of Phase 1. If Thread A finishes Phase 1 really fast, it cannot simply jump into Phase 2—it would be calculating with incomplete data! A Barrier is a concurrency primitive that forces all threads to wait at a designated checkpoint. Only when the last thread arrives does the barrier drop, letting everyone proceed to the next round.

Step 1: Three threads are processing Phase 1 at different speeds.

How it works (CyclicBarrier / WaitGroup)

You initialize a Barrier with a count (e.g., N=3 threads). When a thread calls barrier.wait(), it goes to sleep and the count decrements. When the count hits 0, the barrier wakes all sleeping threads up, and optionally resets itself for the next round (making it "Cyclic").

# Using a Barrier in Python

import threading

# Create a barrier for 3 threads
barrier = threading.Barrier(3)

def worker_thread(worker_id):
    print(f"Worker {worker_id} doing Phase 1...")
    do_heavy_math()
    
    # Wait for the other 2 workers to finish Phase 1
    barrier.wait() 
    
    print(f"Worker {worker_id} starting Phase 2!")
    do_phase_two()

# Start 3 threads. None will start Phase 2 until all 3 hit the wait().

Cost

Barriers are essential for phase-based parallel algorithms, but they introduce straggler problems. If one thread is stuck on a slow CPU core, the other N-1 threads sit entirely idle, wasting resources. The overall speed is strictly limited by the slowest worker in the pool.

Watch out for