The problem: sending large files efficiently
Scaling Rails is all about reducing the number of requests and making the rest as short as possible. Move long running requests to asynchronous background jobs. Offload asset requests to a CDN. Speed up requests with caches. And so it goes.
One type of request that is slow but doesn’t quite fit this pattern is sending large files through Rails. You might need to do this because the file needs to be secured (i.e. via the Rails session), or is generated on the fly. In these cases you can’t upload it to a CDN ahead of time, and making it asynchronous would not be a good user experience.
So how do you send a large file with Rails without bogging down the app for the duration of the download?
How send_file
and NGINX work together
If you are using NGINX, the answer is the X-Accel-Redirect
header. Here’s how it works:
As you can see, a simple send_file
call kicks off a chain of events that will only succeed if things are properly set up. Here are the steps needed to get it working.
1 Configure Rails
First, Rails needs to be told to use the X-Accel-Redirect
feature. Otherwise Rails will use a more basic (but slower) method of doing the streaming itself.
To enable NGINX acceleration, uncomment this line in config/environments/production.rb
:
config.action_dispatch.x_sendfile_header = "X-Accel-Redirect" # for NGINX
2 Set up an NGINX location to serve the files
NGINX can’t just serve any file on the filesystem; we have to specify a “document root” and a URI to access the files we want to send. I chose __send_file_accel
as the URI to avoid any conflicts with Rails routes.
The document root should be a well-known location that we can easily access within the Rails app. With Capistrano, my Rails app lives in /home/deployer/apps/my_app/releases
, so that is the natural place for the root.
# Allow NGINX to serve any file in /home/deployer/apps/my_app/releases
# via a special internal-only location.
location /__send_file_accel {
internal;
alias /home/deployer/apps/my_app/releases;
}
3 Send the X-Accel-Mapping header
Let’s say we want to send this file:
/home/deployer/apps/my_app/releases/20151003173639/tmp/file.zip
That means NGINX needs to receive this header:
X-Accel-Redirect: /__send_file_accel/20151003173639/tmp/file.zip
Rails (or more specifically, Rack) needs to know how to make this transformation. Here are the appropriate headers to send from NGINX to Rails (place these in the e.g. Unicorn reverse proxy configuration):
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /home/deployer/apps/my_app/releases/=/__send_file_accel/;
4 Test it!
Here’s a trivial example for sending a file in a Rails controller:
send_file Rails.root.join("tmp", "file.zip")
During a download you should see something like this in the Rails production.log
:
Sent file /home/deployer/apps/nginx-test/releases/20151011032644/tmp/file.zip (0.2ms)
Completed 200 OK in 2ms (ActiveRecord: 0.0ms)
In the response headers, you can see evidence of whether it was Rails or NGINX that served the file. If the headers contain X-Request-Id
, that means Rails served it.
X-Request-Id: a4d62cdb-569b-4120-b1ff-e2adbf77039a
X-Runtime: 0.005559
If everything is set up correctly, these headers should be absent, meaning NGINX was responsible for delivering the file, not Rails.
And that’s it! You’ve now offloaded sending large files from Rails onto NGINX.