Advanced techniques for calling HTTP APIs in Ruby

It is deceptively straightforward make HTTP calls in Ruby, using the built-in Net::HTTP library or one of many popular gems. However, I’ve found that there is a large gap between an initial implementation and a robust one that stands up to production use.

This post covers several advanced topics related to making HTTP API calls in Ruby. Each adds a bit of complexity, and so I wouldn’t necessarily recommend using all of them on every project. But I think knowledge of these patterns is useful; I’ve found them all to come in handy at some point.

Although the patterns and techniques I will describe in this post are broadly applicable to any technology stack, I’ll be using Faraday 2.x as the basis of my recommendations below. Faraday has been my choice of HTTP library for a while, and it is popular and well-maintained by most measures.

Let’s begin!

Table of contents

• Errors
• Timeouts
• Persistent connections
• Logging
• Aside: Faraday middleware configuration pitfalls
• Rate limits
• Circuit breakers
• Caching
• A complete example
• Further reading

Errors

When consuming HTTP APIs, error handling is an easy thing to overlook. Many common HTTP libraries – Net::HTTP, Faraday, and JavaScript’s fetch, among others – will not raise an exception if an HTTP API returns an error response.

require "net/http"
Net::HTTP.get("https://api.github.com/gist")

require "faraday"
Faraday.get("https://api.github.com/gist")

When consuming APIs, I almost always want to raise an exception when I don’t get an expected response. A 4xx response could tell me that my API credentials are wrong, or that a rate limit has been exceeded. A 5xx response indicates the API itself is broken or temporarily offline. An exception is usually the best way to get my attention in these cases.

Example: using the :raise_error middleware

In Faraday, this is accomplished by adding the :raise_error middleware to the connection config:

require "faraday"

conn = Faraday.new do |config|
  config.response :raise_error
end

# raises Faraday::ResourceNotFound
conn.get("https://api.github.com/gist")

↑ Table of contents

Timeouts

When you depend on an external HTTP API, this introduces a lot of failure modes into your application. A particularly serious form of failure comes when an API is slow to respond.

Let’s say that an API used in a highly-trafficked part of your app suddenly becomes very slow. This could lead to the following consequences:

• Users run out of patience and refresh or retry their requests, potentially compounding the problem. Or they simply abandon your app.
• Your cloud provider drops connections when your app waits for too long, resulting in users seeing 5xx errors. For example, Heroku kills requests after 30 seconds.
• The more Ruby threads that are busy waiting for slow API calls, the fewer that are available to accept new requests. Your app might eventually become unreachable until you forcefully restart it.

Specify aggressive timeouts

To prevent problems like these from spiraling out of control, I recommend always specifying explicit timeout values when using HTTP APIs. The default timeouts for HTTP libraries are usually way too high (Net::HTTP defaults to 60 seconds).

Timeouts should be large enough to allow requests to succeed when the system is operating normally, but low enough that requests are quickly aborted during signs of strain.

Example: Faraday timeout options

Faraday allows you to set timeouts when configuring the connection object:

require "faraday"

faraday_options = {
  request: {
    open_timeout: 1,
    read_timeout: 5,
    write_timeout: 5
  }
}

conn = Faraday.new(**faraday_options) do |config|
  # Middleware can be declared here
end

↑ Table of contents

Persistent connections

On a recent healthcare project, my team built a Rails app that queried a REST API of an established practice management system. During periods of high traffic in production, we were sending so many simultaneous queries to the API that its web server ran out of file descriptors and began intermittently dropping connections and issuing 503 errors. How could this happen?

In Ruby, when you make an HTTP call using Net::HTTP or using Faraday’s default :net_http adapter, every GET or POST requires setting up a new TCP socket, sending the request, and then closing the socket. Allocating a socket uses finite resources on the client and the server. In my team’s case we were issuing so many HTTP calls that we essentially created a denial-of-service attack.

A smarter way to use limited resources is by using persistent HTTP connections. Instead of opening a new TCP socket for every request to the same server, you can reuse a single longer-lived socket. This can also improve performance, because you skip the TCP setup process for each HTTP call.

Example: using faraday-net_http_persistent

A nice feature of Faraday is that it supports pluggable HTTP backends, using its adapter setting. One of these adapters is provided by the faraday-net_http_persistent gem, which seamlessly implements persistent connection management. Here’s an example:

require "faraday"
require "faraday/net_http_persistent"

def conn
  @conn ||= Faraday.new do |config|
    # Other middleware are configured here
    # ...

    # Use :net_http_persistent instead of the default :net_http
    config.adapter :net_http_persistent
  end
end

Behind the scenes, the net-http-persistent library maintains a connection pool. Whenever you make an HTTP call, it checks to see if there is an existing TCP socket already in the connection pool and reuses it if possible. All this happens transparently just by swapping out the Faraday adapter.

When my team switched from :net_http to :net_http_persistent, the production errors we were seeing during high-traffic periods quickly went away.

↑ Table of contents

Logging

When developing against an API, especially an unfamiliar one, I like being able to see the exact request and response data that is being sent back and forth so that I can build an understanding of how the API works. Likewise, once I get to production and encounter a bug, being able to see the HTTP traffic between my app and the API can be a helpful troubleshooting tool.

This is where logging comes into play. Out of the box, Rails automatically logs controller requests and database queries. It’s a good idea to log your HTTP calls as well.

Example: using the :logger middleware

Faraday has a simple :logger middleware that can log information about every HTTP call to a log destination of your choice. Minimally, it will log the HTTP verb, URL, and response status, like this:

[2023-04-22T03:02:36.756969 INFO] POST https://api.github.com/graphql
[2023-04-22T03:02:37.267885 INFO] Status 200

The logger can also be configured to include the full headers and bodies of the request and response. Here’s an example:

Faraday.new do |config|
  config.response :logger,
    Rails.logger,
    headers: true,
    bodies: true,
    log_level: :debug

  config.adapter :net_http
end

Redacting sensitive data

It’s easy to leak sensitive information into log files, especially if you are logging the headers and bodies of every HTTP request. Not all apps have customer PII, but nearly every app has API keys or other credentials. You should do your best to keep those secrets out of your log files.

Faraday has you covered here, too: the :logger middleware can be configured with a block that is used to redact sensitive data with a regular expression. Here’s an example:

Faraday.new do |config|
  config.response :logger, Rails.logger, headers: true, log_level: :debug do |formatter|
    # Redact X-API-Key so that only the last 4 characters are shown in the logs
    formatter.filter(/(x-api-key:\s*)"\w+(\w{4})"/i, '\1"xxxx\2"')
  end
  config.adapter :net_http
end

↑ Table of contents

Aside: Faraday middleware configuration pitfalls

Faraday is a great Ruby HTTP client, but its middleware concept can lead to some pitfalls.

In Faraday, every HTTP request passes through a middleware stack that you define via Faraday.new. This gives you a huge amount of power and flexibility in handling HTTP requests and responses in a declarative fashion. For example:

Faraday.new do |config|
  config.request :authorization, :bearer, ENV["API_TOKEN"]
  config.response :raise_error
  config.request :retry
  config.response :logger
  config.adapter :net_http
end

Here’s how Faraday’s middleware stack works:

1. When you make an HTTP request, each middleware is first executed in the order listed in the config. This gives the middleware a chance to modify or log the request before it is actually performed.
2. Then the actual HTTP network call is made by the HTTP adapter. This is why the adapter is always listed last.
3. Once the response is received, each middleware is executed in reverse order. This gives the middleware a chance to parse or react to the headers and data in the response.

Every middleware is executed once on the request and again for the response. It doesn’t matter if the middleware is added as config.request or config.response. This is why the :logger middleware is able to log both requests and responses, even though it is confusingly declared as config.response. 😵‍💫

Pitfall: :raise_error and :logger in the wrong order

The ordering of the middleware is where things can go wrong. Consider this ordering:

config.response :logger
config.response :raise_error

What happens if you make an HTTP call and the server responds with a 404? Since responses are processed by the middleware stack in reverse order, that means :raise_error will be executed first. It sees the 404 status and raises an error: Faraday::ResourceNotFound.

Because an exception was raised, the Faraday call is immediately aborted. That means the :logger middleware never has a chance to see the response! So in the above config, errors will never get logged.

The correct order would be:

config.response :raise_error
config.response :logger

In short, when using Faraday, configure the middleware stack carefully. When it doubt, copy a known-good config and/or unit test your config with webmock to make sure it behaves as you expect. I’ve shared a real-world config at the end of this article.

↑ Table of contents

Rate limits

Sometimes an API will work great in development or in a small proof-of-concept, but start failing when put under heavy load in production. Maybe you are sending more traffic to the API than it was designed to handle. In many cases, APIs will have rate limits that cause requests to get rejected when those limits are exceeded.

A well-behaved API will enforce its rate limit with a 429 error and Retry-After header that tells you why the request was rejected, and how long you should wait before trying again in order to obey the rate limit. A poorly-behaved API might simply start timing out or responding with 5xx errors when traffic gets too high.

Example: using faraday-retry

Faraday has a solution for both of these scenarios: the faraday-retry middleware. It can automatically detect 429 errors, inspect the response headers, and use that information to issue the request again after the correct amount of delay.

For APIs that do not send the Retry-After header, faraday-retry can be configured to retry after successively larger delays (exponential backoff). Random “jitter” can also be injected into these delays to further smooth out the requests.

Here’s an example of how I typically configure faraday-retry:

require "faraday"
require "faraday-retry"

retry_options = {
  max: 5,                   # Retry a failed request up to 5 times
  interval: 0.5,            # First retry after 0.5s
  backoff_factor: 2,        # Double the delay for each subsequent retry
  interval_randomness: 0.5, # Specify "jitter" of up to 50% of interval
  retry_statuses: [429],    # Retry only when we get a 429 response
  methods: [:get]           # Retry only GET requests
}

conn = Faraday.new do |config|
  # Other middleware get specified here
  # ...

  # :retry should be specified after other middleware, but before :logger
  config.request :retry, retry_options
  config.response :logger
  config.adapter :net_http
end

Refer to the faraday-retry source code for the full list of retry options and their default values.

↑ Table of contents

Circuit breakers

There are cases where retrying failed requests can actually make a problem worse. If an API is failing because of a surge of traffic, retrying failed requests can sustain that traffic or magnify it. The API can get so burdened with requests that it is not able to recover to a stable, healthy state.

API failures can also create a cascade of slowdowns, especially in a service-oriented or microservice architecture where providing data to end-users requires orchestrating requests to many different APIs. For example, if your microservice relies on data from another team’s API, that means slowdowns or high failure rates from their API can make your microservice slower and less reliable as a result. Cascading errors and timeouts can quickly spread across a distributed system.

In scenarios like these, it’s better to turn off requests to the troublesome API altogether, wait, and then resume your requests once the API stabilizes. This is called the circuit breaker pattern, and it works like this:

1. Detect when an API connection is problematic. This is usually done by measuring the rate of failed requests.
2. When a certain failure rate threshold is reached, “open” the circuit breaker. Further attempts to make requests to the API are disallowed.
3. While the circuit breaker is open and access to the API is blocked, you can either “fail fast” or use a fallback. For example, if an advertising API you need is down, you could simply not show ads, or you could show a hard-coded promotion instead.
4. After a cool-down period, “half-open” the circuit breaker to let some requests to the API go through. If those requests succeed, you determine that the API has recovered, close the circuit breaker, and send all requests to the API as normal. If the requests still fail, you open the circuit breaker again and restart the cool-down period.

This is quite a complex state machine to maintain for each API you might be using. Thankfully there are off-the-shelf solutions.

Example: using the faulty gem

Ruby HTTP clients like Faraday typically do not have circuit breaker logic built-in. My go-to solution is the general-purpose faulty gem, which does an excellent job at implementing the circuit breaker pattern, offers plenty of configuration, and integrates nicely with Rails.

Here is a simple implementation:

require "faraday"
require "faulty"

# In a Rails app, this line is best placed in config/initializers/faulty.rb.
Faulty.init

class ExampleClientWithCircuitBreaker
  def get(path, params = {})
    # This will raise Faulty::OpenCircuitError if the circuit breaker is open.
    circuit.run do
      # Here's the actual HTTP call via Faraday, which is protected by the
      # circuit breaker and runs only when the circuit is closed. If this
      # code fails often enough, Faulty will open the circuit.
      conn.get(path, params).body
    end
  end

  private

  def circuit
    # Faulty can have multiple circuits, each with its own configuration.
    # Here we are configuring a circuit that makes sense for this particular
    # API and the needs of our application. The circuit will be opened if
    # >= 50% of requests fail with a Faraday::Error over a 1-minute period.
    @circuit ||= begin
      Faulty.circuit(
        "example",
        rate_threshold: 0.5,
        evaluation_window: 60,
        errors: [Faraday::Error]
      )
    end
  end

  def conn
    @conn ||= Faraday.new("https://api.example.com/") do |config|
      # It's important to raise an exception when a request fails.
      # That way Faulty knows when to trip the circuit breaker.
      config.response :raise_error

      config.adapter :net_http
    end
end

In practice, I often use Faulty in conjunction with its cache system, as described in the next section.

↑ Table of contents

Caching

HTTP API calls are often slower that we’d like (especially compared to local method calls), and can be good candidates for caching, in order to squeeze more performance out of an app. For example, consider a financial app that needs to look up the market closing price of a stock ticker symbol via an API. This information is read-only and changes infrequently, so it is an easy choice for caching.

Caching is usually not that straightforward, though. You’ll need to balance the performance gains against the additional complexity that caching brings, plus the potential downside of showing users stale data. Every app will have a unique set of trade-offs. It’s best to avoid caching until there is a clearly demonstrated need for it.

Example: using Rails.cache

If you do decide to cache, a simple solution is to use the excellent cache implementation provided by Rails. You can leverage it using the Rails.cache.fetch method, like this:

def get_with_cache(path, params: {}, expires_in: 5.minutes)
  # Compute a cache key that is unique for the API, path, and query params
  request_fingerprint = Digest::SHA256.hexdigest({ path:, params: }.inspect)
  key = "ExampleApi/get/#{request_fingerprint}"

  Rails.cache.fetch(key, expires_in:) do
    faraday.get(path, params).body
  end
end

Where to place the cache?

In a more complex real-world code base, it can be hard to decide where the cache fetch block should be placed. For example, if you are using the GitHub API to fetch repository information, there are at least a few different levels where you could insert a cache, in increasing layers of abstraction:

• Cache the raw data you get back from the GitHub API; or
• Cache the Faraday::Response object that contains the raw data, plus response headers, request metadata, etc; or
• Cache the array of Repository objects that you create after parsing the data returned from the API.

In practice, I find it best to cache the raw data. If you cache a Faraday::Response or array of Repository objects, and then later upgrade the faraday gem or change the definition of the Repository class, you could easily break backwards-compatibility with the data that resides in your cache.

Caching as a circuit breaker fallback

Another powerful use of caching is to make a system more fault-tolerant. If an API call fails with an error, or if the API is offline due to a open circuit breaker, you can return a cached response instead of raising an exception. Showing the user stale data is sometimes better than nothing.

For example, imagine you are building an e-commerce app that fetches product ratings from an external API. If that product rating API goes down, you can show a cached rating that you fetched a few hours ago rather than crashing or showing no rating information at all. Most users probably won’t even notice that the ratings are slightly out of sync.

The faulty gem, which I introduced in the circuit breaker section, has a cache option built exactly for this find of fallback scenario. Here is an example:

def get_with_cached_fallback(path, params: {})
  request_fingerprint = Digest::SHA256.hexdigest({ path:, params: }.inspect)
  key = "ExampleApi/get/#{request_fingerprint}"

  circuit.run(cache: key) do
    faraday.get(path, params).body
  end
end


def circuit
  @circuit ||= Faulty.circuit(
    "example",
    cache_expires_in: 24.hours,
    errors: [Faraday::Error]
  )
end

By default, Faulty uses Rails.cache behind the scenes. So if you’ve already done the work to set up Memcached or Redis for Rails caching, Faulty will leverage that infrastructure.

↑ Table of contents

A complete example

Here’s a real-world example of code I’ve used to interact with the GitHub GraphQL API. It uses many of the techniques explained in this article:

require "digest"
require "faraday"
require "faulty"

class GithubClient
  def fetch_graphql_query(query)
    cache = Digest::SHA256.hexdigest(query)
    circuit_breaker.try_run(cache:) do
      faraday.post("graphql", query:).body
    end.or_default({"data" => {}})
  end

  private

  def circuit_breaker
    @circuit_breaker ||= Faulty.circuit(
      :github,
      cache_expires_in: 1.week,
      cache_refreshes_after: 1.hour,
      evaluation_window: 10.minutes,
      errors: [Faraday::Error]
    )
  end

  def faraday
    @faraday ||= begin
      options = {
        request: {
          open_timeout: 1,
          read_timeout: 1,
          write_timeout: 1
        }
      }
      Faraday.new(url: "https://api.github.com/", **options) do |config|
        config.request :authorization, :bearer, ENV["GITHUB_ACCESS_TOKEN"]
        config.request :json
        config.response :json
        config.response :raise_error
        config.response :logger, Rails.logger, headers: true, bodies: true, log_level: :debug do |fmt|
          fmt.filter(/^(Authorization: ).*$/i, '\1[REDACTED]')
        end
        config.adapter :net_http
      end
    end
  end
end

↑ Table of contents

Further reading

• The Gateway Pattern
• Ruby HTTP clients compared (Ruby Toolbox)
• Faraday (GitHub)
• Faulty – Fault-tolerance tools for ruby based on circuit-breakers (GitHub)
• faraday-net_http_persistent (GitHub)
• faraday-retry (GitHub)
• Faraday’s official documentation
• Persistent HTTP connections (MDN)
• HTTP 429 status (MDN)
• HTTP Retry-After header (MDN)
• Exponential backoff (Wikipedia)
• Circuit breaker design pattern (Wikipedia)
• Caching with Rails (Rails Guides)
• ActiveSupport::Cache::Store (Rails API docs)