Atomic writes: temp file then rename

The idea

When you overwrite a file directly, the new bytes go down a little at a time. If the machine loses power halfway through, the file is left as a mangled mix of old and new — neither version is intact, and a reader sees garbage.

The fix is to never touch the live file while you build its replacement. Write the complete new version to a temporary file alongside it, make sure it's safely on disk, then rename the temp file over the destination. On POSIX systems rename() is atomic, so a reader always sees either the old whole file or the new whole file — never a torn one.

See it work

How it works

Write the replacement to a temp file in the same directory as the destination (so the final rename stays on one filesystem), flush it all the way to disk with fsync, then call os.replace to swap it in. os.replace maps to an atomic rename, so the swap happens as a single indivisible step.

import os, tempfile

def atomic_write(path, data):
    d = os.path.dirname(os.path.abspath(path))
    # Temp file MUST live in the same directory / filesystem
    # as the destination, or the rename can't be atomic.
    fd, tmp = tempfile.mkstemp(dir=d, prefix=".tmp-")
    try:
        with os.fdopen(fd, "w") as f:
            f.write(data)
            f.flush()
            os.fsync(f.fileno())   # force bytes to disk before renaming
        os.replace(tmp, path)      # atomic rename over the destination
        # Optional: fsync the directory so the rename itself survives a crash.
        dirfd = os.open(d, os.O_RDONLY)
        try:
            os.fsync(dirfd)
        finally:
            os.close(dirfd)
    except BaseException:
        os.unlink(tmp)             # clean up the temp file on failure
        raise

A reader opening path at any instant sees the old complete file before the rename and the new complete file after it. There is no moment where the file is half-written.

Cost / trade-offs

Watch out for

• Putting the temp file in /tmp or another mount: a cross-filesystem rename silently falls back to copy-then-delete, which is not atomic. Always create the temp in the destination’s own directory.
• Skipping the fsync on the temp file before the rename — the rename can land on disk while the new contents are still in the OS cache, so a crash gives you an empty or stale file.
• Not fsyncing the directory after the rename — the directory entry can be lost on a crash, undoing the swap even though the temp data was durable.
• Leaving orphaned .tmp files behind when a write fails or crashes — clean them up, or you slowly leak disk.
• Forgetting to copy the original file’s permissions and owner — a fresh temp file often defaults to restrictive or wrong modes, so the replacement can become unreadable to others.

Worked example

Picture a settings saver that must update config.json. It writes the new JSON to config.json.tmp in the same folder, calls fsync on it, then runs os.replace("config.json.tmp", "config.json").

Now suppose the power dies at the worst possible instant. If it dies during the temp write, config.json was never touched — the app boots with the old, intact config and the leftover .tmp is just discarded. If it dies right at the rename, the rename either fully happened or fully didn’t, so the app sees the old config or the new config — never a half-parsed file that crashes the loader. Either way, the user’s settings survive.

Check yourself