This guide is based on my experience testing Ruby projects using Jenkins CI, but the concepts should also be applicable to other CI environments.
tldr; just show me the script!
General best practices
Externalize the build script. CI servers like Jenkins and hosted CI services offer fancy control panels for setting things up. Even so, I suggest moving as much of the build/test process as possible into an external shell script that you check in as part of your project. Then configure Jenkins, etc. to simply call the script. The advantage is now your build process is versioned along with your source code, and you have more flexibility to switch CI products later.
Minimize CI-specific configuration. The config files your app needs to boot and run tests in CI should already be in your project repository. Ideally these are the same files that developers on your team use to bootstrap their own environments. The exception to the rule is, of course, secret tokens/passwords, which should never be in source control. Use the facilities provided by the CI product to inject these into the environment and access them in your app via
ENV. For Jenkins, the EnvInject plugin is perfect for this.
Automate everything. The whole point of CI is that it is automatic. If you find yourself SSH-ing into the CI server to install gems, packages, Ruby versions, etc. whenever your project changes, that can interrupt your development process and reduce the value of a CI system. Make the build script bulletproof!
Mimic the developer on-boarding process. How does a new developer get started on the project? Is there a
bin/setup script? Do they need to create a database? Install prerequisites? If your CI follows these same steps, then there will be consistency between your development and CI environments. Even better: your on-boarding process is now continuously tested, so there won’t be any unpleasant surprises when a new person joins the team.
Be careful with Bundler
Now that I’ve covered the general practices, let’s go into Rails specifics, starting with Bundler.
As Ruby developers,
bundle install and
bundle exec have become muscle memory, and we take Bundler’s behavior for granted. But there are some special considerations for CI.
Use the right version. Different versions of Bundler have subtly different behavior (and bugs) when resolving dependencies and installing gems, so it’s very important to consistently use the version of Bundler that has been “blessed” for a particular Rails project, especially in CI and in production.
How do you know what version of Bundler to use? Starting with Bundler 1.10, Bundler now adds a
BUNDLED WITH metadata value to the
Gemfile.lock. You can use that in a CI script to install the version needed by the project. For example:
bundler_version=`ruby -e 'puts $<.read[/BUNDLED WITH\n (\S+)$/, 1] || "<1.10"' Gemfile.lock` gem install bundler --conservative --no-document -v $bundler_version
--deployment flag. The CI build should not be making any changes to your project’s dependencies under any circumstances. Instead, it should treat the
Gemfile.lock as gospel and install exactly what is listed there. The
bundle install --deployment command ensures this behavior.
Exercise the database migrations
It’s an unfortunate reality that Rails database migrations can break. For example:
- An old migration references code in the project, like a model class, that no longer exists.1
- A migration was authored when the app was running an older version of Ruby or Rails, and no longer works in the current environment.
Now, normally developers don’t re-run old migrations, and that’s a problem: if nobody is running them, they aren’t being tested! This can come back to bite a team at exactly the wrong time, for example when trying to recover from an old database backup.
That’s why I recommend running migrations in CI. If an old migration starts to fail, CI will catch it. Here’s the script I use:
bundle exec rake db:drop || true bundle exec rake db:create db:migrate bundle exec rake db:seed
This ensures that each CI build uses a fresh database and all the migrations are run. The seeds also get exercised.
Check for security vulnerabilities
Next up on Rails-specific CI tools: security.
Brakeman. This is an excellent tool for detecting security vulnerabilities in Rails apps. Brakeman specializes in finding seemingly innocuous coding mistakes that nonetheless open up big holes in Rails, like cross-site scripting, SQL injection, unsafe redirects, and remote code execution. The Brakeman website provides in-depth explanations of why each of its findings is a security concern and how the code can be rewritten to fix it.
bundler-audit. Updating gems can be a tedious and risky process, which means that a lot of apps simply make do with old gem versions; it’s the “if it ain’t broke, don’t fix it” mentality. But what happens if a gem needs an urgent patch to fix a gaping security hole? That’s where bundler-audit comes in. It scans a Gemfile.lock and reports if there are any gems that need to be upgraded to fix known security issues (CVEs).
Brakeman and bundler-audit are great, but like all static analysis tools, they only work when you remember to run them. This makes security scanners great candidates for CI!2
Add both gems to the
Gemfile (they don’t need to be required, because they run outside of Rails as command-line tools):
group :development do gem "brakeman", :require => false gem "bundler-audit", :require => false end
Then invoke them as part of the CI script:
# Security audits if bundle show brakeman &> /dev/null; then bundle exec brakeman --no-progress fi if bundle show bundler-audit &> /dev/null; then bundle exec bundle-audit update bundle exec bundle-audit -v fi
Finally, after all the tests are green and the security checks pass, why not deploy the code to a staging environment? This allows project stakeholders to see the latest code in action within minutes of a developer pushing a commit.
With a tool like Capistrano, a CI script can kick off a deployment with a few lines of shell script:
# $GIT_BRANCH is automatically set by Jenkins if bundle show capistrano &> /dev/null; then if [[ $GIT_BRANCH == origin/development ]]; then SSHKIT_COLOR=1 bundle exec cap staging deploy fi fi
The full script
Here is the full script that I use in Jenkins to test each of my Rails projects. It comes standard with an Rails app generated by my Rails template.
If you aren’t too familiar with bash, note that
set -e means the script will abort as soon as any command exits with a non-zero status (otherwise the script would continue all the way to the final deployment step even if e.g.
rake test failed).
#!/bin/bash # # Script used to test this project in Jenkins and continuously deploy the # development branch to the staging capistrano target. Assumes the Jenkins # user is using bash and rbenv. YMMV. # set -e # Ensure we are in the project directory cd $WORKSPACE # If ruby version is not installed, install it if ! ruby -v &> /dev/null; then rbenv update rbenv install `cat .ruby-version` fi # Install necessary version of bundler bundler_version=`ruby -e 'puts $<.read[/BUNDLED WITH\n (\S+)$/, 1] || "<1.10"' Gemfile.lock` gem install bundler --conservative --no-document -v $bundler_version # Set up local config cp config/database.example.yml config/database.yml cp example.env .env bundle install --deployment --retry=3 bundle exec rake db:drop || true bundle exec rake db:create db:migrate bundle exec rake db:seed # Webkit needs an X server in order to render. # See https://github.com/thoughtbot/capybara-webkit/issues/402 if type xvfb-run; then DISABLE_SPRING=1 DISPLAY=localhost:1.0 xvfb-run bundle exec rake test else DISABLE_SPRING=1 bundle exec rake test fi # Security audits if bundle show brakeman &> /dev/null; then bundle exec brakeman --no-progress fi if bundle show bundler-audit &> /dev/null; then bundle exec bundle-audit update bundle exec bundle-audit -v fi # Run a capistrano deploy if we just built the "development" branch. if bundle show capistrano &> /dev/null; then if [[ $GIT_BRANCH == origin/development ]]; then SSHKIT_COLOR=1 bundle exec cap staging deploy:migrate_and_restart fi fi
What actions do you include in your CI process? Is there anything I missed? Let me know!
It’s bad practice to reference models inside migrations, precisely because of this problem. Everyone makes mistakes, though, and sometimes these things slip through code review. ↩
Another option is to pay for services that will run these tools for you. For example, Code Climate uses brakeman behind the scenes, and Gemnasium can watch your repository and alert you about dangerously outdated gems. ↩