Wiki

HTTPX design goals are to be simple and intuitive to use, to not reinvent the wheel, and to be forwards-compatible with the protocols of tomorrow.

It was made in order to provide a ruby http test library that could support both HTTP/2 and HTTP/1 (and its unique features) using the same API. Beyond that, it tried to take the best features of existing ruby HTTP libraries, and avoiding its inconsistencies.

Basic Usage

• Make Requests: learn how to make requests
• Pass Parameters: learn how to send requests using query, form (“www-form-uriencoded”), raw data, and multipart files in requests
• Pass Headers: learn how to set headers, and trailers
• Response Handling: learn how to handle responses from requests

Plugins

HTTPX ships with a plugin system “inherited” from sequel, roda or shrine. It allows for controlled extension of the core functionality, i.e. it can be plugged-in at the instance level.

require "httpx"

# sends request with the "Authentication" header
auth_client = HTTPX.plugin(:authentication)
auth_client.authentication("SECRET-TOKEN").get("http://example.com") #=> ...Authentication: "SECRET-TOKEN"\r\n...

# does NOT send the "Authentication" header
HTTPX.get("http://example.come")

• Proxy: send requests through HTTP, HTTPS, Socks4, Socks4a, Socks5 or SSH proxies.
• Auth: authenticate requests using HTTP Basic Auth or HTTP Digest Auth.
• Follow Redirects: requests which automatically follow HTTP 3XX Redirect responses.
• Retries: retry requests on certain errors.
• Cookies: manage Cookies in requests for the duration of a session.
• Server Push: (HTTP/2 only) support server push, receive responses before initiating requests.
• Upgrade: (HTTP/1.1 only) negotiate an HTTP/1 connection into a new protocol via the Upgrade header
  • H2C: upgrade connection to cleartext HTTP/2 via h2c negotiation.
  • H2: upgrade connection to HTTP/2 via h2 prior knowledge when ALPN negotiation can’t be used.
• Persistent: keep the connections alive and reuse them.
• Stream: handles streaming requests (i.e. the Twitter Streaming API).
• Expect: Supports Expect: 100-continue header in requests with a body.
• Rate Limiter: handle rate limiting responses from throttling server automatically;
• AWS Sigv4: authenticate requests using AWS Sigv4 request signatures;
• GRPC: DSL for declaring and performing GRPC requests;
• Response Cache: Use HTTP caching to minimize bandwidth usage, leveraging etag and last-modified strategies;
• Circuit Breaker: “fail fast and recover after a while” when peer error rate increases;
• WebDav: convenience methods to handle common webdav operations;
• OAuth: OAuth session handling capabilities;
• SSRF Filter: Filters internal network requests with user-input URLs;
• Callbacks: how to react on specific events of an HTTP request lifecycle.
• Content Digest: Encode and validate integrity of HTTP payloads via the “Content-Digest” header.
• Custom Plugins: learn how to create your custom plugin and extend the API.

Integrations

httpx ships with custom third-party integrations for the following gems:

• Faraday: backend for the faraday HTTP client library.
• Datadog: opentracing auto-instrumentation for the datadog ruby SDK.
• Sentry: breadrumbs and tracing for the Sentry ruby SDK.
• Webmock: Integration for mocking httpx requests for webmock.

External

This is a non-exhaustive list of tools which ship with their own httpx integration:

• new-relic is able to trace HTTP requests performed with httpx.
• open-telemetry integrates with httpx to trace HTTP requests.
• openapi-generator is able to generate HTTP clients on top of httpx.

Considerations

• Compression: handle compression and decompression of requests and responses (gzip, deflate, brotli).
• Error Handling: how httpx handles errors when using multiple requests.
• IDN: how httpx handles IDNA.
• DNS Resolvers: how you can choose how to perform DNS resolving.
• TLS: How to configure TLS options.
• Timeouts: how you can leverage network timeouts.
• Connections: how HTTPX manages connections.
• Connection Pools: what makes the library thread-safe.
• Debugging: how to debug and inspect the HTTP interactions.
• JRuby/Truffleruby/Other Rubies: notes about support for non-CRuby implementations.
• FAQ
• Why?

Contributing

If you want to contribute, here’s what you’ll need to know.

Ruby or docker

In order to contribute, you can choose to develop in your machine, or use the docker-compose setup we use for the CI builds.

If you choose to develop locally, you’ll have to have ruby installed. It’s recommended you use a ruby version manager for this. If you never worked with one, my personal recommendation is to use chruby and ruby-install. You’ll also need gcc (or clang) and make installed (some development dependencies require the installation of C-extensions). All test runs will hit publicly available peers (i.e.: nghttp2.org).

If you choose to develop using docker:

# this example is for ruby 2.7 specifically, there's a compose file for each supported version
> docker-compose -f docker-compose.yml -f docker-compose-ruby-2.7.yml run --entrypoint sh httpx
> test/support/ci/build.sh

and you’re good to go. All tests will run against the containerized services.

assert

When writing tests, you have to focus on minimalism. If you look at the test suite, only one assertion method from minitest is being used in most cases: assert. There will be exceptions, but you’ll be asked to use assert whenever possible.

Integration test first

All HTTP features have an integration test using httpbin (with a few exceptions). If you want to add a specific HTTP feature, test it using an endpoint from httpbin which can validate it.