Socket short writes

write() may accept fewer bytes than you gave it — you must loop until everything is sent, advancing past what was already written.

The idea

When you call write() on a TCP socket, the kernel copies your bytes into a fixed-size send buffer and returns how many it accepted. If that buffer is nearly full, it accepts only some of your bytes — a short write — and returns a count smaller than the length you passed.

The return value is not optional. The correct pattern keeps an offset into your buffer, writes the remaining slice, advances the offset by what was accepted, and repeats until offset == len. Assuming write() always sends the whole buffer silently truncates the stream.

toggle the naive “assume full write” path

An 11-byte message waiting to go out. The kernel send buffer holds 5 bytes and drains slowly to the network. Press Play, or step through the write loop.

Green bytes are sent; the bold outline is the slice the next write() attempts. In the naive mode, the unwritten tail is left behind in warm orange — those bytes never reach the network.

How it works

Wrap the call in a loop. Track how far you have written with an offset, and on each pass hand write() only the remaining slice buf[offset:]. Add the returned count to offset and keep going until offset == len. On a non-blocking socket, a return of -1 with errno == EAGAIN (a.k.a. EWOULDBLOCK) is not an error — it means the send buffer is full, so wait for the socket to become writable, then retry.

def write_all(sock, buf):
    offset = 0
    n = len(buf)
    while offset < n:
        try:
            sent = sock.send(buf[offset:])   # may accept fewer than n - offset
        except BlockingIOError:              # errno == EAGAIN / EWOULDBLOCK
            wait_until_writable(sock)         # e.g. select / poll / epoll for EPOLLOUT
            continue                          # buffer was full — retry, do NOT drop bytes
        if sent == 0:
            raise ConnectionError("peer closed the connection")
        offset += sent                        # advance past the bytes that went out
    # loop exits only when offset == n: every byte is in the kernel's hands

The key invariant: offset only ever moves forward by the exact count write() reported. You never assume a byte was sent unless the kernel said so, and you never re-send a byte you already advanced past.

Signals

When short writes happen	What the symptom looks like
Send buffer fills (fast producer, slow consumer)	Late bytes are dropped; the receiver sees a message cut off mid-stream
Non-blocking socket, buffer momentarily full	`write()` returns `-1`/`EAGAIN`; mistaken for an error, the connection is closed
Large payload exceeds free buffer space	One `write()` sends a prefix only; the tail silently vanishes
Receiver applies backpressure (stops reading)	The window closes, `write()` makes no progress, the sender appears to hang
Length-prefixed or delimited framing on top	A truncated body desynchronises the parser — framing errors on every later message

A short write is normal and expected, not a malfunction. The bug is always in the caller that ignored the return value.

Watch out for

Assuming write() sends the whole buffer. It is allowed to return any count from 1 up to the length you passed. On a blocking socket the count can still be short once the send buffer is nearly full. Always loop on the offset.
Ignoring the return value. Calling write(fd, buf, len) and discarding the result drops every byte past the accepted count. The tail never reaches the wire and no error is raised.
Treating EAGAIN as a fatal error. On a non-blocking socket, -1 with EAGAIN/EWOULDBLOCK means “buffer full, retry later,” not “connection broken.” Closing the socket here drops a healthy connection.
Busy-looping on EAGAIN. Spinning while (send() == -1) pegs a CPU core while the buffer stays full. Wait for writability with select/poll/epoll (EPOLLOUT) before retrying.
Losing framing across a partial write. If you do not track the offset, a half-sent length-prefixed message corrupts the stream boundary, so every later message is misparsed — including splitting multi-byte UTF-8 mid-character.

Worked example

Send an 11-byte message through a 5-byte send window. The buffer starts empty; the network drains it between attempts. Watch the offset advance by exactly what each write() reports.

buf = b"HELLO-WORLD"     # len = 11, offset = 0

write(buf[0:11]) -> 5    # buffer had 5 free; accepted 5.   offset = 5
                         # ... a burst refills the buffer to 5/5 ...
write(buf[5:11]) -> -1   # EAGAIN: buffer is full right now.  offset stays 5
                         # ... wait for writable; the buffer drains ...
write(buf[5:11]) -> 5    # accepted the next 5.              offset = 10
                         # ... one byte still pending ...
write(buf[10:11]) -> 1   # accepted the last byte.           offset = 11 == len -> done

Three successful writes (5, 5, 1) with one EAGAIN in between deliver all 11 bytes. The naive version stops after the first write() returns 5 and leaves -WORLD unsent — the receiver gets HELLO and a desynchronised stream.

Check yourself

You call write() with 10 bytes and it returns 3. What should the next call pass?

On a non-blocking socket, write() returns -1 with errno == EAGAIN. What is the right response?