Library Checkout (Distributed Locks)

How to guarantee that only one server can process a job at a time.

The idea

Imagine you have a background job that charges a user's credit card for their monthly subscription. You run 3 backend servers for redundancy. At 12:00 AM, the cron job triggers. Without coordination, Server A, Server B, and Server C will ALL try to charge the user's card simultaneously. The user gets billed $30 instead of $10! To prevent this, the servers must use a Distributed Lock (like checking out a library book). Server A grabs the lock, does the job, and returns the lock. Servers B and C see the lock is taken and do nothing.

Step 1: Uncoordinated chaos. 3 servers all run the billing job at exactly 12:00 AM. The user is charged 3 times.

How it works (Redis SET NX)

Distributed locks are usually implemented using Redis. Server A tries to create a key (e.g., lock:billing:alice). It uses the NX flag (Not eXists), meaning Redis will ONLY create the key if it doesn't already exist. This is an atomic operation. Server A succeeds (returns OK). A millisecond later, Server B tries to create the same key, but Redis rejects it (returns NULL). Server B knows someone else is doing the job and aborts.

// 1. Try to grab the lock. 
// "NX" = Only set if Not Exists. 
// "PX 5000" = Auto-expire the lock after 5 seconds!
const lockAcquired = await redis.set("lock:billing:alice", "server_A", "NX", "PX", 5000);

if (!lockAcquired) {
    console.log("Someone else is billing Alice. Skipping.");
    return;
}

try {
    // 2. We have the lock! Charge the credit card.
    await stripe.charge(alice, 10);
} finally {
    // 3. Always release the lock when finished
    await redis.del("lock:billing:alice");
}

Cost

Distributed locks add network latency (talking to Redis) before you can start your actual work. Additionally, if you rely entirely on a single Redis instance for locking, and that Redis instance crashes, all your servers will fail to acquire locks and your system halts. For mission-critical locking, algorithms like Redlock (using 5 independent Redis nodes) are required.

Watch out for