webDAV door closes the network connection preventing reuse

[airnandez]

Reusing established TLS connection between a dCache client and the webDAV door is beneficial, in particular when a great number of requests that are typically served directly by the door (e.g. HEAD, MKCOL, MOVE, etc.) are issued by the client, as is our use case.

We have been working on chasing inefficiencies in our dCache client and noticed that, contrary to what we have thought, the webDAV door closes the network connection with the client after serving each request, so the client cannot reuse the connection for subsequent requests. I am aware this is intended behavior for PUT requests which include the Expect: 100-continue header but I thought that the door left the door open for other kind of requests.

I failed to find something in the documentation to understand whether this behavior is intended. However, I noticed that the HTTP response does not include a Connection: close header, which makes me think this behavior may not be intended, hence this issue. I also note that the behavior of the server of dCache frontend API is the same: it closes the network connection after serving each request, even if the client explicitly adds a Connection: keep-alive header, which is the default for HTTP 1.1 anyway.

To understand if this was related to the Python client we are using, we did the tests below:

• implemented two independent clients, one uses urllib3 and the other httpx
• used those two clients to send a few HEAD requests to several servers, including https://prometheus.desy.de:2443/, https://home.cern/, https://www.desy.de/, https://httpbin.org:443/uuid.

All the endpoints above but the dCache prometheus.desy.de:2443 (dCache/11.1.0-SNAPSHOT) leave the connection open for the client to reuse it. The behavior of our local dCache instance (dCache/10.2.14) is the same as prometheus.desy.de: it closes the connection. We can reproduce that behavior with any of our two clients.

I wonder if there is any possibility to make the webDAV door to keep the network connection with its clients open: this would help us to avoid spending 30ms on average establishing the TCP+TLS connection for each request in the context of our local area network.

I am attaching:

• the execution trace of a client which sequentially sends three HEAD requests to https://home.cern/ which shows that in this case the client can reuse the connection,
• the execution trace of a client which sequentially sends three HEAD requests to https://prometheus.desy.de:2443/Users/paul/target/test-file which shows that the the network connection is closed after each request,
• the httpx-based Python script httpx_client.py which produces those traces

cern.log
prometheus.log
httpx_client.py

[airnandez]

Additional information. I did further tests with requests using other HTTP verbs and this is what I found:

When the request uses method...	can the client reuse the network connection?	Comments
MKCOL	Yes ✅
MOVE	Yes ✅
PROPFIND	Yes ✅
PUT	Yes ✅	Only when the door proxies the data to the pool. If the client includes Expect: 100-continue header the connection to the door cannot be reused.
DELETE	No ❌
GET	No ❌	The client cannot reuse the connexion to the door after receiving the redirect response. The network connection to the pool can be reused after the client receives the pool's response.
HEAD	No ❌
OPTIONS	No ❌