Python Timeout Puzzles

Transmission Control Protocol (TCP) implements reliable delivery of messages atop the unreliable Internet Protocol (IP). IP is best-effort – when you send an IP packet it may not reach the destination, and there's no way to know if it did. TCP adds mechanisms to ensure that your data reaches the other side. One such mechanism is retransmission.

Every byte in a TCP stream gets a sequence number: its index in the stream plus a base value. Each message can contain an acknowledgement number, the sequence number of the next byte the host is expecting² from the stream it is receiving. These ACK messages are sent roughly³ for every data message received.

                    Next expected byte              Possible to receive data 
                Value sent in ACKs, B+N+1           out of order.  Note that
                                \                 ACK still points before gap.
                                 |                               |
                                 v                               v
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |      Acknowleged Data     |XXXXXXXXXXXXXXXXXXXXXXXXX|           |XXX..|
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ^                          ^                 ^                       ^
    |                          |                 |                       |
     \                        /                   \                     /
Base value, B        Last received byte, B+N       Data not yet received

When TCP sends a packet, it waits to get the corresponding ACK⁴ before marking that data as reliably sent. If no ACK comes within a certain window, TCP will retransmit that packet. This window is the TCP Retransmission Window, a timeout determining how long to wait before sending a packet again.

The window is determined by an algorithm outlined in RFC 2988 (2), originally from Van Jacobson. The core idea is to continuously measure the round-trip time between the sender and receiver and to set the timeout based on the observed distribution. This can't apply for the initial window used for connecting as no round trips have happened yet. As such the RFC sets a default of 3 seconds.

What happens once we retransmit? RFC 1122 (4.2.3.1) specifies that a TCP implementation must include exponential back-off on the retransmit timer when it expires. RFC 2988 (5.5) further requires that this use a doubling operation with an upper bound. As such we can see the retransmit window for the initial segment as being

\[ \min\{3 \times 2^n, \textrm{RTO_MAX}\} \, \textrm{seconds}, \]

where \(n\) is the number of times the segment has been retransmitted and \(\textrm{RTO_MAX}\) is a constant determined by the TCP implementation. \(\textrm{RTO_MAX}\) must be at least 60 seconds. With this in mind, if we wanted to line up our connect timeouts with the TCP retransmission window's expiry, we should really be setting timeouts to be just over a power of two multiplied by 3, not any multiple of 3.

RFC 2988 is actually obsoleted by the more recent RFC 6298, creatively titled Computing TCP's Retransmission Timer. This RFC reduces the default initial window down to 1 second, so the retransmit window for the initial segment becomes

\[ \min\{2^n, \textrm{RTO_MAX}\} \, \textrm{seconds}. \]

Fans of the number 3 will be relieved to know that it is still used as a magic value in the RFC. If the initial segment was retransmitted, and the timeout is less than 3 seconds when a connection is established, then the timeout must be set to 3 seconds once data transfer begins. In short, if you need to retransmit the first packet exactly once, then set the timeout to 3 seconds for the next packet. This doesn't have any effect on the initial packet, though, which is what a connect timeout should be concerned with.

Since this RFC came out, the advice in the Requests documentation seems irrelevant, as the number 3 should have no bearing whatsoever on timeouts for a call to connect(2).

Pedantry aside, this still got us no closer to understanding why this advice is given. Considering how retransmissions work, I don't see any reason why one would do this. We could receive the response at any point after sending the initial request, regardless of when we retransmit. In fact, just longer than a retransmission window would mean we retransmit our segment and then immediately give up, which seems wasteful.

I've tracked down the original PR for adding these docs (and connect timeouts), but there's no mention of this statement at all. There were several older PRs attempting to do the same thing, but they had no answers either. I've been unable to track down this practice anywhere else. I suspect it's apocryphal. If anybody out there knows where this comes from, I'd love to hear it!

During this search I found a more recent issue that made this completely irrelevant.

I wanted to see if this were true. It's such a long departure from the documented behaviour of just applying to connect(2) that it had to be seen to be believed. Let's write a little reproducer together.

First, let's use the Python standard library's http.server module⁵ to quickly whip up a test server: we want to implement a method such that we read all the request, send success, and exit.

class TestHandler(http.server.SimpleHTTPRequestHandler):
    # Implementing HTTP methods can be done simply by creating a do_VERB()
    # method!
    def do_POST(self):
        self.rfile.read()
        self.send_response(http.HTTPStatus.OK)
        self.end_headers()

For convenience, let's add a context manager that will run a server for us in the background, and clean it up once we exit a with block.

@contextmanager
def test_server() -> Generator[str, None, None]:
    """Run a simple test server and clean up afterwards.

    We yield the base URL for the server, which is handy because we're serving
    on an ephemeral port.
    """
    httpd = http.server.HTTPServer(("127.0.0.1", 0), TestHandler)
    t = threading.Thread(target=httpd.serve_forever)
    t.start()
    try:
        yield f"http://{httpd.server_name}:{httpd.server_port}/"
    finally:
        httpd.shutdown()
        t.join()

Now let's try and induce a timeout once a connection has been made, but before it's time to read. We should be able to achieve this by using a short timeout window, and trying to send a lot of data in the request. We can use a much longer timeout length for the read timeout, and time our request to check which timeout we're hitting.

def send_request(url: str) -> None:
    before = time.clock_gettime(time.CLOCK_MONOTONIC)
    try:
        requests.post(url, data="A" * 1024 * 1024 * 512, timeout=(0.1, 100))
    except Exception as e:
        print(f"Requests raised an exception: {type(e).__name__}({e})")
    after = time.clock_gettime(time.CLOCK_MONOTONIC)
    print(f"Took {after - before}s")

Finally, let's put things all together so we have a single script we can run.

def main():
    with test_server() as url:
        send_request(url)

if __name__ == "__main__":
    main()

Let's give it a try, and see if we hit a timeout, and if so which kind.

./src/timeouts/test-timeout.py

Requests raised an exception: ConnectionError(('Connection aborted.', TimeoutError('timed out')))
Took 0.22552246099985496s

Well, it claims it was a TimeoutError, and the error talks about connections, which seems hopeful. More hopeful still is that it took less than 100 seconds to hit the timeout, and as such it must have been the connect timeout in use.

How can we know that the timeout applied for the entire request? It's feasible that we timed out on a single send(2) call. The easiest way to confirm this is to use strace. strace is a very handy tool you can use to trace the system calls a program makes on Linux.

strace -T -e connect,send,sendto,sendmsg,poll,select ./src/timeouts/test-timeout.py 2>&1

The -T flag will ensure that we get times for each syscall displayed in angle brackets <.> at the end of each line, and the -e flag allows us to limit to just the syscalls we're interested in.

connect(4, {sa_family=AF_INET, sin_port=htons(40337), sin_addr=inet_addr("127.0.0.1")}, 16) = 0 <0.000014>
sendto(4, "POST / HTTP/1.1\r\nHost: localhost"..., 174, 0, NULL, 0) = 174 <0.000042>
poll([{fd=4, events=POLLOUT}], 1, 100)  = 1 ([{fd=4, revents=POLLOUT}]) <0.000016>
sendto(4, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 536870912, 0, NULL, 0) = 2968064 <0.000712>
poll([{fd=4, events=POLLOUT}], 1, 100)  = 1 ([{fd=4, revents=POLLOUT}]) <0.000624>
sendto(4, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 533902848, 0, NULL, 0) = 1047552 <0.000233>
<snip 149 very similar lines>
poll([{fd=4, events=POLLOUT}], 1, 2)    = 1 ([{fd=4, revents=POLLOUT}]) <0.000695>
sendto(4, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 454669824, 0, NULL, 0) = 1047552 <0.000202>
poll([{fd=4, events=POLLOUT}], 1, 1)    = 1 ([{fd=4, revents=POLLOUT}]) <0.000750>
sendto(4, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 453622272, 0, NULL, 0) = 1047552 <0.000198>
127.0.0.1 - - [10/Jul/2026 06:56:24] "POST / HTTP/1.1" 200 -
Requests raised an exception: ConnectionError(('Connection aborted.', TimeoutError('timed out')))
Took 0.8319506759999058s
+++ exited with 0 +++

As we can see, no individual call is taking longer than the timeout. It must be the case that this limit is applied to the entire request!

Requests provides a more friendly interface to urllib3, which does all of the heavy lifting of HTTP requests. urllib3 handles management of connections, sending requests over these connections, and reading responses. It also provides the timeout functionality used by Requests.

# src/requests/adapters.py

class HTTPAdapter(BaseAdapter):
    ...

    def send(self,
             request: PreparedRequest,
             stream: bool = False,
             timeout: _t.TimeoutType = None,
             verify: _t.VerifyType = True,
             cert: _t.CertType = None,
             proxies: dict[str, str] | None = None,
        ) -> Response:
            ...
            try:
                # conn is a urllib3 ConnectionPool
                resp = conn.urlopen(
                    method=request.method,
                    url=url,
                    body=request.body,
                    headers=request.headers,
                    redirect=False,
                    assert_same_host=False,
                    preload_content=False,
                    decode_content=False,
                    retries=self.max_retries,
                    timeout=resolved_timeout,
                    chunked=chunked,

                )
            except (ProtocolError, OSError) as err:
                raise ConnectionError(err, request=request)
            ...

urllib3 implements timeouts by passing them on to the socket module of the Python standard library. Before establishing a connection urllib3 calls socket.settimeout with the connect timeout. It then creates the connection and sends the request before updating the socket's timeout to the read timeout.

# src/urllib3/connectionpool.py

class HTTPConnectionPool(ConnectionPool, RequestMethods):
    ...
    def _make_request(
        self,
        conn: BaseHTTPConnection,
        method: str,
        url: str,
        body: _TYPE_BODY | None = None,
        headers: typing.Mapping[str, str] | None = None,
        retries: Retry | None = None,
        timeout: _TYPE_TIMEOUT = _DEFAULT_TIMEOUT,
        chunked: bool = False,
        response_conn: BaseHTTPConnection | None = None,
        preload_content: bool = True,
        decode_content: bool = True,
        enforce_content_length: bool = True,
    ) -> BaseHTTPResponse:
        ...
         self.num_requests += 1

        timeout_obj = self._get_timeout(timeout)
        timeout_obj.start_connect()
        conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout)

        ...

        # conn.request() calls http.client.*.request, not the method in
        # urllib3.request. It also calls makefile (recv) on the socket.
        try:
            conn.request(
                method,
                url,
                body=body,
                headers=headers,
                chunked=chunked,
                preload_content=preload_content,
                decode_content=decode_content,
                enforce_content_length=enforce_content_length,
            )

        ...

        # Reset the timeout for the recv() on the socket
        read_timeout = timeout_obj.read_timeout

        if not conn.is_closed:
            # In Python 3 socket.py will catch EAGAIN and return None when you
            # try and read into the file pointer created by http.client, which
            # instead raises a BadStatusLine exception. Instead of catching
            # the exception and assuming all BadStatusLine exceptions are read
            # timeouts, check for a zero timeout before making the request.
            if read_timeout == 0:
                raise ReadTimeoutError(
                    self, url, f"Read timed out. (read timeout={read_timeout})"
                )
            conn.timeout = read_timeout

        ...

To apply a timeout, Python sets the underlying socket to use non-blocking I/O, and uses select(2) (or poll(2) if available) to apply the given timeout on network calls. For example, if using socket.send() with a timeout, then Python will use poll() to wait for the socket to be writable up until the timeout. If not writable by then, you get a TimeoutError. If the socket is writable then send() is called, and you (hopefully) get a success.

socket.send() is a simple wrapper of the send() system call. If using it, you'll need to account for short writes to be sure that you have actually sent all your data. Python provides a handy socket.sendall() function, which handles the short writes on your behalf. Unsurprisingly urllib3 uses socket.sendall() to send its requests⁶.

Python 3.5 subtly changed the timeout semantics of socket.sendall(). Prior to this release, the socket's timeout was applied to each send() call. The timeout value was the maximum time to wait to perform a single write. In Python 3.5 the timeout applies to the entire socket.sendall() call – it's now the maximum time to wait to perform all the writes!

As such, since Python 3.5 connect timeouts in Requests and urllib3 exhibit the surprising behaviour of applying to your entire request upload.