Connections

HTTPX abstract all the network/application connection level gory details from the user, so that, from the user’s perspective, it’s all HTTP. It’s a design decision, therefore a “feature not a bug”.

This document explains what’s the default behaviour, how to enable variants of it (and what are the implications), and alternative usages.

Default

HTTPX is optimized for concurrent request handling, which means, it will try to reuse the minimum of resources (sockets, TLS sessions) for the maximum number of requests and will interleave them (as much as the underlying protocol allows).

Transport (L4)

Currently the library only supports TCP sockets (in the future, when/if QUIC support is here, hopefully this will change).

If the server supports persistent connections (HTTP/2: persistent by default, HTTP/1: persistent when using Keep-Alive), it will send all requests through the same connection, otherwise it will reopen a connection to the server for each request.

If you’re using HTTP/1.1 and the server supports persistent connections, but you still want to use a different connection for each request, you can force this by setting the “connection” header to “close”:

HTTPX.headers("connection" => "close").get("http://example.com/1", "http://example.com/2")

Session (L5)

If the request is done using an “https://” URI, it will try to perform ALPN negotiation and choose “h2” (the recommended and preferred protocol to use for interactions). If the server doesn’t support “h2” or ALPN negotiation at all, it will fallback to HTTP/1.1 over TLS.

If the request is done using an “http://” URI, it will use the fallback protocol (if you don’t change it, it’s “HTTP/1.1”).

(if using the “h2c” plugin, this can vary).

Presentation (L6)

If the selected protocol is HTTP/2, requests will be interleaved in the same connection. Requests will use the same connection if:

  • They are sent to the same origin;
  • They are sent to different origins, but both resolve to the same IP and the server TLS certificate matches against both origins (wilcard or hostnames lists, if TLS is being used);

(This technique is called “unsharding”, which I picked up from cURL’s blog).

If the selected protocol is HTTP/1, HTTPX will try to use HTTP pipelining to send multiple requests to the server (the number of requests is controlled by the :max_concurrent_requests option). If the server doesn’t support pipelining, it will fallback to sending one request at a time in the same connection; if the peer closes the connection however, it falls back again, this time to one request per connection.

Caveat: Some servers/proxies may not fail gracefully when receiving the pipelined requests and may abruptly abort the connection. There is no recovery strategy implemented for that. However, you can disable pipelining by setting :max_concurrent_requests to 1.

Connection Reuse

Sometimes users may want to manage the connections by themselves, and want to reuse them to do the requests. You can do this by using the :io option:

require "httpx"

tls = fetch_tls_socket_to_that_server

response = HTTPX.get("https://myserver.org", io: tls)

tls.closed? #=> false

When this option is used, HTTPX will treat the io like a “dump pipe” and will not do anything to it besides reading from and writing to it. It will also send all requests through that dump pipe, so beware of the connection reuse implications discussed above.

You can also pass an hash, where the IOs are indexed by the IP:PORT pair (depending of your network stack, don’t forget to add both entries for IPv4 and IPv6 addresses if you want to support both). In such a case, the IP to which the hostname of the request URI will be resolved, will determine whether the already open socket is picked up:

require "httpx"

server_ipv4_address = "172.125.2.255:8080"
server_ipv6_address = "[2001:db8:a0b:12f0::1]:8080"
tls = fetch_tls_socket_to_that_server
ios = { server_ipv4_address => tls,
        server_ipv6_address => tls }

response = HTTPX.get("https://myserver.org", io: ios)

tls.closed? #=> false

External Event Loop integration

TODO: fill this up when this is possible

Next: Thread Safety