Home Wiki

Faq

Do I need to specify any params to ensure requests are concurrent?

No additional params are required, but you have to do them all in the same call, just like it’s documented.

Whether the requests will be performed concurrently is dependent on the server, i.e. HTTP/2 enabled servers can receive multiple requests at the same time, and HTTP/1.1 servers supporting pipeline as well.

Concurrency is therefore protocol-dependent, and httpx will not support distributing M requests over N connections on the same pool. If the server you’re requesting to doesn’t fit the criteria above, check Connection Pools.

Will each thread need its own static Session object with the persistent plugin in order to leverage the persistent connections?

No. httpx handles the complexity for you, so you just need to assign HTTPX.plugin(:persistent) to a variable, and this can be used in any context.

httpx does maintain per-session connection pools however, meaning that, if you’re using multiple threads, connections to the same origin server will be managed in a thread-safe manner.

How can I tell if connections are being reused?

You can troubleshoot that by enabling the logs, where information of DNS requests, TLS handshake and connection state transitions will be displayed.

I have an issue with `httpx`, how can I report it in a useful manner?

The first thing is to describe how you’re using httpx, i.e. what options/plugins are enabled:

# example: "I'm using this in production"
HTTPX.plugin(:persistent).with(timeout: { request_timeout: 10 }) # etc etc ....

If possible, the best way to report an issue is to reproduce it against some public server, and provide the script in the issue description.

However, sometimes that’s not possible, because the issue manifests over time on a long running workload. And since you most likely want allow server access to an httpx maintainer, the next best thing is to provide debug logs:

http = initiate_httpx_session
# add log infrastructure, check the wiki entry for logs:
# https://gitlab.com/os85/httpx/-/wikis/Debugging
http = http.with(debug: your_application_logger, debug_level: 3, debug_redact: true)

collect a decent amount of log messages prior to the occurrence being reported (the amount of logs may vary here) and post them in the issue tracker as a file or text.

my application has too much traffic / collecting debug logs is too expensive

For such situations, it’s best if you then provide a custom logger where messages are buffered and only written to the logger when the outstanding event occurs. The example below of a circular buffer has been used for such systems:

 module ErrorLogger
    extend self

    MAX_LOG_SIZE = 1024 * 1024 * 2 # 2mb, add your own limit

    @mut = Thread::Mutex.new
    @size = 0
    @buffer = []

    def dup
      @mut.synchronize { @buffer.dup }
    end

    def <<(str)
      return unless feature_flag_system.enabled?('httpx-enable-logs')

      @mut.synchronize do
        @buffer << str
        @size += str.size


        while @size > MAX_LOG_SIZE
          chunk = @buffer.shift
          @size -= chunk.size
        end
      end
    end

    module Plugin
      module RequestMethods
        # logs should be written here, as usually the errors are identifiable here, and if you're using
        # tracing, this ensures that the logs are emitted with trace ids which help find them
        def response=(response)
          super

          return unless feature_flag_system.enabled?('httpx-enable-logs')

          return unless @response

          # add your own condition of what an issue may look like here
          unless @response.is_a?(HTTPX::ErrorResponse) || @response.status.between?(400, 599)
            return
          end

          your_logger.error { 'Error performing HTTPX request' }

          ErrorLogger.dup.each do |chunk|
            your_logger.error { chunk }
          end
        end
      end
    end
  end

How do I force HTTP/1.1 even if the server supports HTTP/2?

You must disable it in the ALPN negotiation:

HTTPX.with(ssl: { alpn_protocols: %w[http/1.1] }).get("https://example.com")

Will httpx ever support other protocols?

This library does HTTP only, and will not support other protocols. Ever. There is only one cURL :) .

Next: home

honeyryder

Faq