TCP socket framing

The idea

It is tempting to think that each time you call send(), the other side gets exactly one matching recv(). It does not. TCP is a stream, not a packet protocol. The kernel is free to split, merge, and re-time your bytes however it likes on the way across the network.

So a single recv() can hand you half a message, exactly one message, or several messages glued end to end. The bytes always arrive in order and are never lost — but the boundaries between your messages are gone. TCP never promised to keep them.

To get discrete messages back, you add framing: a small rule that says where each message stops. The most common rule is length-prefixing — put a fixed-size header in front of every message that says how many payload bytes follow. The receiver reads the length, then reads exactly that many bytes, emits one complete frame, and keeps any leftover bytes in a buffer for the next frame.

How it works

Keep one growing buffer across calls. Every time bytes arrive, append them, then drain as many complete frames as the buffer currently holds. The inner while loop is what makes coalesced reads work — it keeps slicing frames until the buffer can’t yield another whole one.

import struct

buffer = bytearray()

def on_bytes(chunk):                # called for every TCP recv()
    buffer.extend(chunk)
    while True:
        if len(buffer) < 4:         # not even a full length header yet
            return
        (length,) = struct.unpack(">I", buffer[:4])   # 4-byte big-endian length
        if len(buffer) < 4 + length:
            return                  # header present but payload incomplete
        frame = bytes(buffer[4:4 + length])
        del buffer[:4 + length]     # consume this frame; keep the rest
        handle(frame)               # a complete application message

def send(sock, payload):
    sock.sendall(struct.pack(">I", len(payload)) + payload)

Two returns, two waiting conditions: not enough bytes for the header, and header is here but the payload is short. In both cases you stop and wait for the next recv(). The leftover bytes stay in buffer and become the start of the next frame.

Cost

A 4-byte big-endian length holds up to ~4 GB. That ceiling is also a liability — see the watch-out below.

Watch out for

• Assuming one recv() equals one message. This is the number-one framing bug. A read can return a partial message, or several messages stuck together. Never treat a single read as a single message.
• Forgetting to loop on coalesced reads. If one chunk carries two whole frames, a single pass emits only the first and silently strands the second in the buffer. You must loop until the buffer can’t produce another full frame.
• Mismatched byte order. The length must use a fixed endianness on both sides. Pick network byte order (big-endian, >I); a sender and receiver that disagree on endianness will read garbage lengths.
• Trusting the length field. An unbounded length lets a hostile or buggy peer claim a 4 GB frame and exhaust your memory. Always cap the maximum frame size and drop the connection if it’s exceeded.
• Dropping leftover bytes — or partial sends. Keep the buffer’s tail for the next frame; don’t reset it between reads. And use sendall, not send — send may write fewer bytes than you handed it, truncating your frame on the wire.

Worked example

Three messages go out: "abc" (length 3), "hi" (length 2), and "ping" (length 4). To keep it readable the visual uses a one-byte length header. They arrive in four chunks that deliberately ignore the frame boundaries:

No chunk lined up with a frame. Between reads the buffer quietly held the dangling partial frame, and the length header told the receiver exactly how many more bytes to wait for. That is the whole trick.

Check yourself