A Recipe for Rails Continuous Integration

This guide is based on my experience testing Ruby projects using Jenkins CI, but the concepts should also be applicable to other CI environments.

tldr; just show me the script!

General best practices

Externalize the build script

CI servers like Jenkins and hosted CI services offer fancy control panels for setting things up. Even so, I suggest moving as much of the build/test process as possible into an external shell script that you check in as part of your project. Then configure Jenkins, etc. to simply call the script. The advantage is now your build process is versioned along with your source code, and you have more flexibility to switch CI products later.

Minimize CI-specific configuration

The config files your app needs to boot and run tests in CI should already be in your project repository. Ideally these are the same files that developers on your team use to bootstrap their own environments. The exception to the rule is, of course, secret tokens/passwords, which should never be in source control. Use the facilities provided by the CI product to inject these into the environment and access them in your app via ENV. For Jenkins, the EnvInject plugin is perfect for this.

Automate everything

The whole point of CI is that it is automatic. If you find yourself SSH-ing into the CI server to install gems, packages, Ruby versions, etc. whenever your project changes, that can interrupt your development process and reduce the value of a CI system. Make the build script bulletproof!

Mimic the developer on-boarding process

How does a new developer get started on the project? Is there a bin/setup script? Do they need to create a database? Install prerequisites? If your CI follows these same steps, then there will be consistency between your development and CI environments. Even better: your on-boarding process is now continuously tested, so there won’t be any unpleasant surprises when a new person joins the team.

Be careful with Bundler

Now that I’ve covered the general practices, let’s go into Rails specifics, starting with Bundler.

As Ruby developers, bundle install and bundle exec have become muscle memory, and we take Bundler’s behavior for granted. But there are some special considerations for CI.

Use the right version

Different versions of Bundler have subtly different behavior (and bugs) when resolving dependencies and installing gems, so it’s very important to consistently use the version of Bundler that has been “blessed” for a particular Rails project, especially in CI and in production.

How do you know what version of Bundler to use? Starting with Bundler 1.10, Bundler now adds a BUNDLED WITH metadata value to the Gemfile.lock. You can use that in a CI script to install the version needed by the project. For example:

bundler_version=`ruby -e 'puts $<.read[/BUNDLED WITH\n   (\S+)$/, 1] || "<1.10"' Gemfile.lock`
gem install bundler --conservative --no-document -v $bundler_version

Use the --deployment flag

The CI build should not be making any changes to your project’s dependencies under any circumstances. Instead, it should treat the Gemfile.lock as gospel and install exactly what is listed there. The bundle install --deployment command ensures this behavior.

Exercise the database migrations

It’s an unfortunate reality that Rails database migrations can break. For example:

• An old migration references code in the project, like a model class, that no longer exists.¹
• A migration was authored when the app was running an older version of Ruby or Rails, and no longer works in the current environment.

Now, normally developers don’t re-run old migrations, and that’s a problem: if nobody is running them, they aren’t being tested! This can come back to bite a team at exactly the wrong time, for example when trying to recover from an old database backup.

That’s why I recommend running migrations in CI. If an old migration starts to fail, CI will catch it. Here’s the script I use:

bundle exec rake db:drop || true
bundle exec rake db:create db:migrate
bundle exec rake db:seed

This ensures that each CI build uses a fresh database and all the migrations are run. The seeds also get exercised.

Check for security vulnerabilities

Next up on Rails-specific CI tools: security.

Brakeman.

This is an excellent tool for detecting security vulnerabilities in Rails apps. Brakeman specializes in finding seemingly innocuous coding mistakes that nonetheless open up big holes in Rails, like cross-site scripting, SQL injection, unsafe redirects, and remote code execution. The Brakeman website provides in-depth explanations of why each of its findings is a security concern and how the code can be rewritten to fix it.

bundler-audit.

Updating gems can be a tedious and risky process, which means that a lot of apps simply make do with old gem versions; it’s the “if it ain’t broke, don’t fix it” mentality. But what happens if a gem needs an urgent patch to fix a gaping security hole? That’s where bundler-audit comes in. It scans a Gemfile.lock and reports if there are any gems that need to be upgraded to fix known security issues (CVEs).

Brakeman and bundler-audit are great, but like all static analysis tools, they only work when you remember to run them. This makes security scanners great candidates for CI!²

Add both gems to the Gemfile (they don’t need to be required, because they run outside of Rails as command-line tools):

group :development do
  gem "brakeman", :require => false
  gem "bundler-audit", :require => false
end

Then invoke them as part of the CI script:

# Security audits
if bundle show brakeman &> /dev/null; then
  bundle exec brakeman --no-progress
fi
if bundle show bundler-audit &> /dev/null; then
  bundle exec bundle-audit update
  bundle exec bundle-audit -v
fi

Continuous deployment

Finally, after all the tests are green and the security checks pass, why not deploy the code to a staging environment? This allows project stakeholders to see the latest code in action within minutes of a developer pushing a commit.

With a tool like Capistrano, a CI script can kick off a deployment with a few lines of shell script:

# $GIT_BRANCH is automatically set by Jenkins
if bundle show capistrano &> /dev/null; then
  if [[ $GIT_BRANCH == origin/development ]]; then
    SSHKIT_COLOR=1 bundle exec cap staging deploy
  fi
fi

The full script

Here is the full script that I use in Jenkins to test each of my Rails projects. It comes standard with an Rails app generated by my Rails template.

If you aren’t too familiar with bash, note that set -e means the script will abort as soon as any command exits with a non-zero status (otherwise the script would continue all the way to the final deployment step even if e.g. rake test failed).

#!/bin/bash
#
# Script used to test this project in Jenkins and continuously deploy the
# development branch to the staging capistrano target. Assumes the Jenkins
# user is using bash and rbenv. YMMV.
#
set -e

# Ensure we are in the project directory
cd $WORKSPACE

# If ruby version is not installed, install it
if ! ruby -v &> /dev/null; then
  rbenv update
  rbenv install `cat .ruby-version`
fi

# Install necessary version of bundler
bundler_version=`ruby -e 'puts $<.read[/BUNDLED WITH\n   (\S+)$/, 1] || "<1.10"' Gemfile.lock`
gem install bundler --conservative --no-document -v $bundler_version

# Set up local config
cp config/database.example.yml config/database.yml
cp example.env .env

bundle install --deployment --retry=3
bundle exec rake db:drop || true
bundle exec rake db:create db:migrate
bundle exec rake db:seed

# Webkit needs an X server in order to render.
# See https://github.com/thoughtbot/capybara-webkit/issues/402
if type xvfb-run; then
  DISABLE_SPRING=1 DISPLAY=localhost:1.0 xvfb-run bundle exec rake test
else
  DISABLE_SPRING=1 bundle exec rake test
fi

# Security audits
if bundle show brakeman &> /dev/null; then
  bundle exec brakeman --no-progress
fi
if bundle show bundler-audit &> /dev/null; then
  bundle exec bundle-audit update
  bundle exec bundle-audit -v
fi

# Run a capistrano deploy if we just built the "development" branch.
if bundle show capistrano &> /dev/null; then
  if [[ $GIT_BRANCH == origin/development ]]; then
    SSHKIT_COLOR=1 bundle exec cap staging deploy:migrate_and_restart
  fi
fi

What actions do you include in your CI process? Is there anything I missed? Let me know!