ruby
Advanced techniques for calling HTTP APIs in Ruby
How to use Faraday and other gems to implement circuit breakers, gracefully handle API rate limits, pool connections for better performance, and more.
It is deceptively straightforward make HTTP calls in Ruby, using the built-in Net::HTTP
library or one of many popular gems. However, I’ve found that there is a large gap between an initial implementation and a robust one that stands up to production use.
This post covers several advanced topics related to making HTTP API calls in Ruby. Each adds a bit of complexity, and so I wouldn’t necessarily recommend using all of them on every project. But I think knowledge of these patterns is useful; I’ve found them all to come in handy at some point.
Although the patterns and techniques I will describe in this post are broadly applicable to any technology stack, I’ll be using Faraday 2.x as the basis of my recommendations below. Faraday has been my choice of HTTP library for a while, and it is popular and well-maintained by most measures.
Let’s begin!
Table of contents
- Errors
- Timeouts
- Persistent connections
- Logging
- Aside: Faraday middleware configuration pitfalls
- Rate limits
- Circuit breakers
- Caching
- A complete example
- Further reading
Errors
When consuming HTTP APIs, error handling is an easy thing to overlook. Many common HTTP libraries – Net::HTTP
, Faraday, and JavaScript’s fetch
, among others – will not raise an exception if an HTTP API returns an error response.
require "net/http"
Net::HTTP.get("https://api.github.com/gist")
require "faraday"
Faraday.get("https://api.github.com/gist")
Net::HTTP
and Faraday both don’t raise an exception by default.
When consuming APIs, I almost always want to raise an exception when I don’t get an expected response. A 4xx response could tell me that my API credentials are wrong, or that a rate limit has been exceeded. A 5xx response indicates the API itself is broken or temporarily offline. An exception is usually the best way to get my attention in these cases.
Example: using the :raise_error
middleware
In Faraday, this is accomplished by adding the :raise_error
middleware to the connection config:
require "faraday"
conn = Faraday.new do |config|
config.response :raise_error
end
# raises Faraday::ResourceNotFound
conn.get("https://api.github.com/gist")
:raise_error
middleware raises an appropriate subclass of Faraday::Error
if the response has a 4xx or 5xx status.
Timeouts
When you depend on an external HTTP API, this introduces a lot of failure modes into your application. A particularly serious form of failure comes when an API is slow to respond.
Let’s say that an API used in a highly-trafficked part of your app suddenly becomes very slow. This could lead to the following consequences:
- Users run out of patience and refresh or retry their requests, potentially compounding the problem. Or they simply abandon your app.
- Your cloud provider drops connections when your app waits for too long, resulting in users seeing 5xx errors. For example, Heroku kills requests after 30 seconds.
- The more Ruby threads that are busy waiting for slow API calls, the fewer that are available to accept new requests. Your app might eventually become unreachable until you forcefully restart it.
Specify aggressive timeouts
To prevent problems like these from spiraling out of control, I recommend always specifying explicit timeout values when using HTTP APIs. The default timeouts for HTTP libraries are usually way too high (Net::HTTP
defaults to 60 seconds).
Timeouts should be large enough to allow requests to succeed when the system is operating normally, but low enough that requests are quickly aborted during signs of strain.
Example: Faraday timeout options
Faraday allows you to set timeouts when configuring the connection object:
require "faraday"
faraday_options = {
request: {
open_timeout: 1,
read_timeout: 5,
write_timeout: 5
}
}
conn = Faraday.new(**faraday_options) do |config|
# Middleware can be declared here
end
Faraday::TimeoutError
when these timeout thresholds are exceeded. Timeout values are in seconds.
Persistent connections
On a recent healthcare project, my team built a Rails app that queried a REST API of an established practice management system. During periods of high traffic in production, we were sending so many simultaneous queries to the API that its web server ran out of file descriptors and began intermittently dropping connections and issuing 503 errors. How could this happen?
In Ruby, when you make an HTTP call using Net::HTTP
or using Faraday’s default :net_http
adapter, every GET or POST requires setting up a new TCP socket, sending the request, and then closing the socket. Allocating a socket uses finite resources on the client and the server. In my team’s case we were issuing so many HTTP calls that we essentially created a denial-of-service attack.
A smarter way to use limited resources is by using persistent HTTP connections. Instead of opening a new TCP socket for every request to the same server, you can reuse a single longer-lived socket. This can also improve performance, because you skip the TCP setup process for each HTTP call.
Example: using faraday-net_http_persistent
A nice feature of Faraday is that it supports pluggable HTTP backends, using its adapter
setting. One of these adapters is provided by the faraday-net_http_persistent gem, which seamlessly implements persistent connection management. Here’s an example:
require "faraday"
require "faraday/net_http_persistent"
def conn
@conn ||= Faraday.new do |config|
# Other middleware are configured here
# ...
# Use :net_http_persistent instead of the default :net_http
config.adapter :net_http_persistent
end
end
:net_http_persistent
behavior.
Behind the scenes, the net-http-persistent library maintains a connection pool. Whenever you make an HTTP call, it checks to see if there is an existing TCP socket already in the connection pool and reuses it if possible. All this happens transparently just by swapping out the Faraday adapter.
When my team switched from :net_http
to :net_http_persistent
, the production errors we were seeing during high-traffic periods quickly went away.
Logging
When developing against an API, especially an unfamiliar one, I like being able to see the exact request and response data that is being sent back and forth so that I can build an understanding of how the API works. Likewise, once I get to production and encounter a bug, being able to see the HTTP traffic between my app and the API can be a helpful troubleshooting tool.
This is where logging comes into play. Out of the box, Rails automatically logs controller requests and database queries. It’s a good idea to log your HTTP calls as well.
Example: using the :logger
middleware
Faraday has a simple :logger
middleware that can log information about every HTTP call to a log destination of your choice. Minimally, it will log the HTTP verb, URL, and response status, like this:
[2023-04-22T03:02:36.756969 INFO] POST https://api.github.com/graphql
[2023-04-22T03:02:37.267885 INFO] Status 200
The logger can also be configured to include the full headers and bodies of the request and response. Here’s an example:
Faraday.new do |config|
config.response :logger,
Rails.logger,
headers: true,
bodies: true,
log_level: :debug
config.adapter :net_http
end
:debug
ensures that this amount of detail is only logged when Rails debug logging is turned on (i.e. when developing locally or when troubleshooting in production).
Redacting sensitive data
It’s easy to leak sensitive information into log files, especially if you are logging the headers and bodies of every HTTP request. Not all apps have customer PII, but nearly every app has API keys or other credentials. You should do your best to keep those secrets out of your log files.
Faraday has you covered here, too: the :logger
middleware can be configured with a block that is used to redact sensitive data with a regular expression. Here’s an example:
Faraday.new do |config|
config.response :logger, Rails.logger, headers: true, log_level: :debug do |formatter|
# Redact X-API-Key so that only the last 4 characters are shown in the logs
formatter.filter(/(x-api-key:\s*)"\w+(\w{4})"/i, '\1"xxxx\2"')
end
config.adapter :net_http
end
X-API-Key
header are redacted from the logs.
Aside: Faraday middleware configuration pitfalls
Faraday is a great Ruby HTTP client, but its middleware concept can lead to some pitfalls.
In Faraday, every HTTP request passes through a middleware stack that you define via Faraday.new
. This gives you a huge amount of power and flexibility in handling HTTP requests and responses in a declarative fashion. For example:
Faraday.new do |config|
config.request :authorization, :bearer, ENV["API_TOKEN"]
config.response :raise_error
config.request :retry
config.response :logger
config.adapter :net_http
end
config.
directive adds a middleware to the middleware stack. Order matters.
Here’s how Faraday’s middleware stack works:
- When you make an HTTP request, each middleware is first executed in the order listed in the config. This gives the middleware a chance to modify or log the request before it is actually performed.
- Then the actual HTTP network call is made by the HTTP
adapter
. This is why the adapter is always listed last. - Once the response is received, each middleware is executed in reverse order. This gives the middleware a chance to parse or react to the headers and data in the response.
Every middleware is executed once on the request and again for the response. It doesn’t matter if the middleware is added as config.request
or config.response
. This is why the :logger
middleware is able to log both requests and responses, even though it is confusingly declared as config.response
. 😵💫
Pitfall: :raise_error
and :logger
in the wrong order
The ordering of the middleware is where things can go wrong. Consider this ordering:
config.response :logger
config.response :raise_error
What happens if you make an HTTP call and the server responds with a 404? Since responses are processed by the middleware stack in reverse order, that means :raise_error
will be executed first. It sees the 404 status and raises an error: Faraday::ResourceNotFound
.
Because an exception was raised, the Faraday call is immediately aborted. That means the :logger
middleware never has a chance to see the response! So in the above config, errors will never get logged.
The correct order would be:
config.response :raise_error
config.response :logger
In short, when using Faraday, configure the middleware stack carefully. When it doubt, copy a known-good config and/or unit test your config with webmock to make sure it behaves as you expect. I’ve shared a real-world config at the end of this article.
Rate limits
Sometimes an API will work great in development or in a small proof-of-concept, but start failing when put under heavy load in production. Maybe you are sending more traffic to the API than it was designed to handle. In many cases, APIs will have rate limits that cause requests to get rejected when those limits are exceeded.
A well-behaved API will enforce its rate limit with a 429 error and Retry-After
header that tells you why the request was rejected, and how long you should wait before trying again in order to obey the rate limit. A poorly-behaved API might simply start timing out or responding with 5xx errors when traffic gets too high.
Example: using faraday-retry
Faraday has a solution for both of these scenarios: the faraday-retry middleware. It can automatically detect 429 errors, inspect the response headers, and use that information to issue the request again after the correct amount of delay.
For APIs that do not send the Retry-After
header, faraday-retry can be configured to retry after successively larger delays (exponential backoff). Random “jitter” can also be injected into these delays to further smooth out the requests.
Here’s an example of how I typically configure faraday-retry:
require "faraday"
require "faraday-retry"
retry_options = {
max: 5, # Retry a failed request up to 5 times
interval: 0.5, # First retry after 0.5s
backoff_factor: 2, # Double the delay for each subsequent retry
interval_randomness: 0.5, # Specify "jitter" of up to 50% of interval
retry_statuses: [429], # Retry only when we get a 429 response
methods: [:get] # Retry only GET requests
}
conn = Faraday.new do |config|
# Other middleware get specified here
# ...
# :retry should be specified after other middleware, but before :logger
config.request :retry, retry_options
config.response :logger
config.adapter :net_http
end
Retry-After
header, Faraday will automatically use that to determine the delay. Otherwise it uses exponential backoff.
Refer to the faraday-retry source code for the full list of retry options and their default values.
Circuit breakers
There are cases where retrying failed requests can actually make a problem worse. If an API is failing because of a surge of traffic, retrying failed requests can sustain that traffic or magnify it. The API can get so burdened with requests that it is not able to recover to a stable, healthy state.
API failures can also create a cascade of slowdowns, especially in a service-oriented or microservice architecture where providing data to end-users requires orchestrating requests to many different APIs. For example, if your microservice relies on data from another team’s API, that means slowdowns or high failure rates from their API can make your microservice slower and less reliable as a result. Cascading errors and timeouts can quickly spread across a distributed system.
In scenarios like these, it’s better to turn off requests to the troublesome API altogether, wait, and then resume your requests once the API stabilizes. This is called the circuit breaker pattern, and it works like this:
- Detect when an API connection is problematic. This is usually done by measuring the rate of failed requests.
- When a certain failure rate threshold is reached, “open” the circuit breaker. Further attempts to make requests to the API are disallowed.
- While the circuit breaker is open and access to the API is blocked, you can either “fail fast” or use a fallback. For example, if an advertising API you need is down, you could simply not show ads, or you could show a hard-coded promotion instead.
- After a cool-down period, “half-open” the circuit breaker to let some requests to the API go through. If those requests succeed, you determine that the API has recovered, close the circuit breaker, and send all requests to the API as normal. If the requests still fail, you open the circuit breaker again and restart the cool-down period.
This is quite a complex state machine to maintain for each API you might be using. Thankfully there are off-the-shelf solutions.
Example: using the faulty
gem
Ruby HTTP clients like Faraday typically do not have circuit breaker logic built-in. My go-to solution is the general-purpose faulty gem, which does an excellent job at implementing the circuit breaker pattern, offers plenty of configuration, and integrates nicely with Rails.
Here is a simple implementation:
require "faraday"
require "faulty"
# In a Rails app, this line is best placed in config/initializers/faulty.rb.
Faulty.init
class ExampleClientWithCircuitBreaker
def get(path, params = {})
# This will raise Faulty::OpenCircuitError if the circuit breaker is open.
circuit.run do
# Here's the actual HTTP call via Faraday, which is protected by the
# circuit breaker and runs only when the circuit is closed. If this
# code fails often enough, Faulty will open the circuit.
conn.get(path, params).body
end
end
private
def circuit
# Faulty can have multiple circuits, each with its own configuration.
# Here we are configuring a circuit that makes sense for this particular
# API and the needs of our application. The circuit will be opened if
# >= 50% of requests fail with a Faraday::Error over a 1-minute period.
@circuit ||= begin
Faulty.circuit(
"example",
rate_threshold: 0.5,
evaluation_window: 60,
errors: [Faraday::Error]
)
end
end
def conn
@conn ||= Faraday.new("https://api.example.com/") do |config|
# It's important to raise an exception when a request fails.
# That way Faulty knows when to trip the circuit breaker.
config.response :raise_error
config.adapter :net_http
end
end
circuit.run
can wrap arbitrary code, like a Faraday request, in a circuit breaker. The circuit automatically disables that code once a certain failure rate is met. It uses a similar heuristic to re-enable the code once the system has “healed”.
In practice, I often use Faulty in conjunction with its cache system, as described in the next section.
Caching
HTTP API calls are often slower that we’d like (especially compared to local method calls), and can be good candidates for caching, in order to squeeze more performance out of an app. For example, consider a financial app that needs to look up the market closing price of a stock ticker symbol via an API. This information is read-only and changes infrequently, so it is an easy choice for caching.
Caching is usually not that straightforward, though. You’ll need to balance the performance gains against the additional complexity that caching brings, plus the potential downside of showing users stale data. Every app will have a unique set of trade-offs. It’s best to avoid caching until there is a clearly demonstrated need for it.
Example: using Rails.cache
If you do decide to cache, a simple solution is to use the excellent cache implementation provided by Rails. You can leverage it using the Rails.cache.fetch
method, like this:
def get_with_cache(path, params: {}, expires_in: 5.minutes)
# Compute a cache key that is unique for the API, path, and query params
request_fingerprint = Digest::SHA256.hexdigest({ path:, params: }.inspect)
key = "ExampleApi/get/#{request_fingerprint}"
Rails.cache.fetch(key, expires_in:) do
faraday.get(path, params).body
end
end
Rails.cache.fetch
. If a previously cached result exists and has not expired, Rails will return it, skipping the HTTP call. Otherwise the HTTP call is executed and the result is stored in the cache for future use.
Where to place the cache?
In a more complex real-world code base, it can be hard to decide where the cache fetch
block should be placed. For example, if you are using the GitHub API to fetch repository information, there are at least a few different levels where you could insert a cache, in increasing layers of abstraction:
- Cache the raw data you get back from the GitHub API; or
- Cache the
Faraday::Response
object that contains the raw data, plus response headers, request metadata, etc; or - Cache the array of
Repository
objects that you create after parsing the data returned from the API.
In practice, I find it best to cache the raw data. If you cache a Faraday::Response
or array of Repository
objects, and then later upgrade the faraday gem or change the definition of the Repository
class, you could easily break backwards-compatibility with the data that resides in your cache.
Caching as a circuit breaker fallback
Another powerful use of caching is to make a system more fault-tolerant. If an API call fails with an error, or if the API is offline due to a open circuit breaker, you can return a cached response instead of raising an exception. Showing the user stale data is sometimes better than nothing.
For example, imagine you are building an e-commerce app that fetches product ratings from an external API. If that product rating API goes down, you can show a cached rating that you fetched a few hours ago rather than crashing or showing no rating information at all. Most users probably won’t even notice that the ratings are slightly out of sync.
The faulty gem, which I introduced in the circuit breaker section, has a cache option built exactly for this find of fallback scenario. Here is an example:
def get_with_cached_fallback(path, params: {})
request_fingerprint = Digest::SHA256.hexdigest({ path:, params: }.inspect)
key = "ExampleApi/get/#{request_fingerprint}"
circuit.run(cache: key) do
faraday.get(path, params).body
end
end
def circuit
@circuit ||= Faulty.circuit(
"example",
cache_expires_in: 24.hours,
errors: [Faraday::Error]
)
end
cache:
parameter specified in circuit.run
means that if a request fails, or if the API goes down and the circuit breaker opens, Faulty will automatically fall back to using a cached response. In this example the cached response is allowed to be up to 24 hours old.
By default, Faulty uses Rails.cache
behind the scenes. So if you’ve already done the work to set up Memcached or Redis for Rails caching, Faulty will leverage that infrastructure.
A complete example
Here’s a real-world example of code I’ve used to interact with the GitHub GraphQL API. It uses many of the techniques explained in this article:
require "digest"
require "faraday"
require "faulty"
class GithubClient
def fetch_graphql_query(query)
cache = Digest::SHA256.hexdigest(query)
circuit_breaker.try_run(cache:) do
faraday.post("graphql", query:).body
end.or_default({"data" => {}})
end
private
def circuit_breaker
@circuit_breaker ||= Faulty.circuit(
:github,
cache_expires_in: 1.week,
cache_refreshes_after: 1.hour,
evaluation_window: 10.minutes,
errors: [Faraday::Error]
)
end
def faraday
@faraday ||= begin
options = {
request: {
open_timeout: 1,
read_timeout: 1,
write_timeout: 1
}
}
Faraday.new(url: "https://api.github.com/", **options) do |config|
config.request :authorization, :bearer, ENV["GITHUB_ACCESS_TOKEN"]
config.request :json
config.response :json
config.response :raise_error
config.response :logger, Rails.logger, headers: true, bodies: true, log_level: :debug do |fmt|
fmt.filter(/^(Authorization: ).*$/i, '\1[REDACTED]')
end
config.adapter :net_http
end
end
end
end
Further reading
- The Gateway Pattern
- Ruby HTTP clients compared (Ruby Toolbox)
- Faraday (GitHub)
- Faulty – Fault-tolerance tools for ruby based on circuit-breakers (GitHub)
- faraday-net_http_persistent (GitHub)
- faraday-retry (GitHub)
- Faraday’s official documentation
- Persistent HTTP connections (MDN)
- HTTP 429 status (MDN)
- HTTP Retry-After header (MDN)
- Exponential backoff (Wikipedia)
- Circuit breaker design pattern (Wikipedia)
- Caching with Rails (Rails Guides)
- ActiveSupport::Cache::Store (Rails API docs)