It is deceptively straightforward make HTTP calls in Ruby, using the built-in
Net::HTTP library or one of many popular gems. However, I’ve found that there is a large gap between an initial implementation and a robust one that stands up to production use.
This post covers several advanced topics related to making HTTP API calls in Ruby. Each adds a bit of complexity, and so I wouldn’t necessarily recommend using all of them on every project. But I think knowledge of these patterns is useful; I’ve found them all to come in handy at some point.
Although the patterns and techniques I will describe in this post are broadly applicable to any technology stack, I’ll be using Faraday 2.x as the basis of my recommendations below. Faraday has been my choice of HTTP library for a while, and it is popular and well-maintained by most measures.
- Table of contents
- Persistent connections
- Aside: Faraday middleware configuration pitfalls
- Rate limits
- Circuit breakers
- A complete example
- Further reading
When consuming HTTP APIs, error handling is an easy thing to overlook. Many common HTTP libraries –
fetch, among others – will not raise an exception if an HTTP API returns an error response.
When consuming APIs, I almost always want to raise an exception when I don’t get an expected response. A 4xx response could tell me that my API credentials are wrong, or that a rate limit has been exceeded. A 5xx response indicates the API itself is broken or temporarily offline. An exception is usually the best way to get my attention in these cases.
In Faraday, this is accomplished by adding the
:raise_error middleware to the connection config:
When you depend on an external HTTP API, this introduces a lot of failure modes into your application. A particularly serious form of failure comes when an API is slow to respond.
Let’s say that an API used in a highly-trafficked part of your app suddenly becomes very slow. This could lead to the following consequences:
- Users run out of patience and refresh or retry their requests, potentially compounding the problem. Or they simply abandon your app.
- Your cloud provider drops connections when your app waits for too long, resulting in users seeing 5xx errors. For example, Heroku kills requests after 30 seconds.
- The more Ruby threads that are busy waiting for slow API calls, the fewer that are available to accept new requests. Your app might eventually become unreachable until you forcefully restart it.
To prevent problems like these from spiraling out of control, I recommend always specifying explicit timeout values when using HTTP APIs. The default timeouts for HTTP libraries are usually way too high (
Net::HTTP defaults to 60 seconds).
Timeouts should be large enough to allow requests to succeed when the system is operating normally, but low enough that requests are quickly aborted during signs of strain.
Faraday allows you to set timeouts when configuring the connection object:
On a recent healthcare project, my team built a Rails app that queried a REST API of an established practice management system. During periods of high traffic in production, we were sending so many simultaneous queries to the API that its web server ran out of file descriptors and began intermittently dropping connections and issuing 503 errors. How could this happen?
In Ruby, when you make an HTTP call using
Net::HTTP or using Faraday’s default
:net_http adapter, every GET or POST requires setting up a new TCP socket, sending the request, and then closing the socket. Allocating a socket uses finite resources on the client and the server. In my team’s case we were issuing so many HTTP calls that we essentially created a denial-of-service attack.
A smarter way to use limited resources is by using persistent HTTP connections. Instead of opening a new TCP socket for every request to the same server, you can reuse a single longer-lived socket. This can also improve performance, because you skip the TCP setup process for each HTTP call.
A nice feature of Faraday is that it supports pluggable HTTP backends, using its
adapter setting. One of these adapters is provided by the faraday-net_http_persistent gem, which seamlessly implements persistent connection management. Here’s an example:
Behind the scenes, the net-http-persistent library maintains a connection pool. Whenever you make an HTTP call, it checks to see if there is an existing TCP socket already in the connection pool and reuses it if possible. All this happens transparently just by swapping out the Faraday adapter.
When my team switched from
:net_http_persistent, the production errors we were seeing during high-traffic periods quickly went away.
When developing against an API, especially an unfamiliar one, I like being able to see the exact request and response data that is being sent back and forth so that I can build an understanding of how the API works. Likewise, once I get to production and encounter a bug, being able to see the HTTP traffic between my app and the API can be a helpful troubleshooting tool.
This is where logging comes into play. Out of the box, Rails automatically logs controller requests and database queries. It’s a good idea to log your HTTP calls as well.
Faraday has a simple
:logger middleware that can log information about every HTTP call to a log destination of your choice. Minimally, it will log the HTTP verb, URL, and response status, like this:
[2023-04-22T03:02:36.756969 INFO] POST https://api.github.com/graphql [2023-04-22T03:02:37.267885 INFO] Status 200
The logger can also be configured to include the full headers and bodies of the request and response. Here’s an example:
It’s easy to leak sensitive information into log files, especially if you are logging the headers and bodies of every HTTP request. Not all apps have customer PII, but nearly every app has API keys or other credentials. You should do your best to keep those secrets out of your log files.
Faraday has you covered here, too: the
:logger middleware can be configured with a block that is used to redact sensitive data with a regular expression. Here’s an example:
Faraday is a great Ruby HTTP client, but its middleware concept can lead to some pitfalls.
In Faraday, every HTTP request passes through a middleware stack that you define via
Faraday.new. This gives you a huge amount of power and flexibility in handling HTTP requests and responses in a declarative fashion. For example:
Here’s how Faraday’s middleware stack works:
- When you make an HTTP request, each middleware is first executed in the order listed in the config. This gives the middleware a chance to modify or log the request before it is actually performed.
- Then the actual HTTP network call is made by the HTTP
adapter. This is why the adapter is always listed last.
- Once the response is received, each middleware is executed in reverse order. This gives the middleware a chance to parse or react to the headers and data in the response.
Every middleware is executed once on the request and again for the response. It doesn’t matter if the middleware is added as
config.response. This is why the
:logger middleware is able to log both requests and responses, even though it is confusingly declared as
The ordering of the middleware is where things can go wrong. Consider this ordering:
What happens if you make an HTTP call and the server responds with a 404? Since responses are processed by the middleware stack in reverse order, that means
:raise_error will be executed first. It sees the 404 status and raises an error:
Because an exception was raised, the Faraday call is immediately aborted. That means the
:logger middleware never has a chance to see the response! So in the above config, errors will never get logged.
The correct order would be:
In short, when using Faraday, configure the middleware stack carefully. When it doubt, copy a known-good config and/or unit test your config with webmock to make sure it behaves as you expect. I’ve shared a real-world config at the end of this article.
Sometimes an API will work great in development or in a small proof-of-concept, but start failing when put under heavy load in production. Maybe you are sending more traffic to the API than it was designed to handle. In many cases, APIs will have rate limits that cause requests to get rejected when those limits are exceeded.
A well-behaved API will enforce its rate limit with a 429 error and
Retry-After header that tells you why the request was rejected, and how long you should wait before trying again in order to obey the rate limit. A poorly-behaved API might simply start timing out or responding with 5xx errors when traffic gets too high.
Faraday has a solution for both of these scenarios: the faraday-retry middleware. It can automatically detect 429 errors, inspect the response headers, and use that information to issue the request again after the correct amount of delay.
For APIs that do not send the
Retry-After header, faraday-retry can be configured to retry after successively larger delays (exponential backoff). Random “jitter” can also be injected into these delays to further smooth out the requests.
Here’s an example of how I typically configure faraday-retry:
Refer to the faraday-retry source code for the full list of retry options and their default values.
There are cases where retrying failed requests can actually make a problem worse. If an API is failing because of a surge of traffic, retrying failed requests can sustain that traffic or magnify it. The API can get so burdened with requests that it is not able to recover to a stable, healthy state.
API failures can also create a cascade of slowdowns, especially in a service-oriented or microservice architecture where providing data to end-users requires orchestrating requests to many different APIs. For example, if your microservice relies on data from another team’s API, that means slowdowns or high failure rates from their API can make your microservice slower and less reliable as a result. Cascading errors and timeouts can quickly spread across a distributed system.
In scenarios like these, it’s better to turn off requests to the troublesome API altogether, wait, and then resume your requests once the API stabilizes. This is called the circuit breaker pattern, and it works like this:
- Detect when an API connection is problematic. This is usually done by measuring the rate of failed requests.
- When a certain failure rate threshold is reached, “open” the circuit breaker. Further attempts to make requests to the API are disallowed.
- While the circuit breaker is open and access to the API is blocked, you can either “fail fast” or use a fallback. For example, if an advertising API you need is down, you could simply not show ads, or you could show a hard-coded promotion instead.
- After a cool-down period, “half-open” the circuit breaker to let some requests to the API go through. If those requests succeed, you determine that the API has recovered, close the circuit breaker, and send all requests to the API as normal. If the requests still fail, you open the circuit breaker again and restart the cool-down period.
This is quite a complex state machine to maintain for each API you might be using. Thankfully there are off-the-shelf solutions.
Ruby HTTP clients like Faraday typically do not have circuit breaker logic built-in. My go-to solution is the general-purpose faulty gem, which does an excellent job at implementing the circuit breaker pattern, offers plenty of configuration, and integrates nicely with Rails.
Here is a simple implementation:
In practice, I often use Faulty in conjunction with its cache system, as described in the next section.
HTTP API calls are often slower that we’d like (especially compared to local method calls), and can be good candidates for caching, in order to squeeze more performance out of an app. For example, consider a financial app that needs to look up the market closing price of a stock ticker symbol via an API. This information is read-only and changes infrequently, so it is an easy choice for caching.
Caching is usually not that straightforward, though. You’ll need to balance the performance gains against the additional complexity that caching brings, plus the potential downside of showing users stale data. Every app will have a unique set of trade-offs. It’s best to avoid caching until there is a clearly demonstrated need for it.
If you do decide to cache, a simple solution is to use the excellent cache implementation provided by Rails. You can leverage it using the
Rails.cache.fetch method, like this:
In a more complex real-world code base, it can be hard to decide where the cache
fetch block should be placed. For example, if you are using the GitHub API to fetch repository information, there are at least a few different levels where you could insert a cache, in increasing layers of abstraction:
- Cache the raw data you get back from the GitHub API; or
- Cache the
Faraday::Responseobject that contains the raw data, plus response headers, request metadata, etc; or
- Cache the array of
Repositoryobjects that you create after parsing the data returned from the API.
In practice, I find it best to cache the raw data. If you cache a
Faraday::Response or array of
Repository objects, and then later upgrade the faraday gem or change the definition of the
Repository class, you could easily break backwards-compatibility with the data that resides in your cache.
Another powerful use of caching is to make a system more fault-tolerant. If an API call fails with an error, or if the API is offline due to a open circuit breaker, you can return a cached response instead of raising an exception. Showing the user stale data is sometimes better than nothing.
For example, imagine you are building an e-commerce app that fetches product ratings from an external API. If that product rating API goes down, you can show a cached rating that you fetched a few hours ago rather than crashing or showing no rating information at all. Most users probably won’t even notice that the ratings are slightly out of sync.
By default, Faulty uses
Rails.cache behind the scenes. So if you’ve already done the work to set up Memcached or Redis for Rails caching, Faulty will leverage that infrastructure.
Here’s a real-world example of code I’ve used to interact with the GitHub GraphQL API. It uses many of the techniques explained in this article:
- The Gateway Pattern
- Ruby HTTP clients compared (Ruby Toolbox)
- Faraday (GitHub)
- Faulty – Fault-tolerance tools for ruby based on circuit-breakers (GitHub)
- faraday-net_http_persistent (GitHub)
- faraday-retry (GitHub)
- Faraday’s official documentation
- Persistent HTTP connections (MDN)
- HTTP 429 status (MDN)
- HTTP Retry-After header (MDN)
- Exponential backoff (Wikipedia)
- Circuit breaker design pattern (Wikipedia)
- Caching with Rails (Rails Guides)
- ActiveSupport::Cache::Store (Rails API docs)