Bullet Proofing a Rails Deploy

Please note that all opinions are that of the author.

Last Updated On: 2025-11-26 06:51:47 -0500

I am now an almost 20 plus year veteran of the Rails development / deployment cycle. That means I’ve seen all the damn generations of this crap:

Mongrel (not really deploy but used to be a part of the Rails deployment environment)
Capistrano
Vlad
Custom Ansible Deployment
Kamal

And I can, with confidence, say the following:

RAILS DEPLOYMENT IS BETTER IN 2025 BUT IT STILL SUCKS

One of the rules of any software development platform (and Rails is a platform) is TANSTAAFL: There ain’t no such thing as a free lunch. In other words, for everything a platform gives you, you lose something else. Rails gives you what I would argue is the easiest way to develop a web application but the cost for that is, well, the worst deployment experience you can ever have. We can argue about why this is the case but here are some thoughts:

Any Rails Application is a Swiss Watch
Ruby itself is Fragile

Now I’d like to state that all of my Rails apps have 100% test coverage, a CI server and all the other bells and whistles – but they don’t – and I’d like to bet that your Rails apps aren’t perfect either.

So How You Bullet Proof a Rails Deployment

And, finally, we come to the thesis of this blog post, how you actually bullet proof a Rails deployment. There are four aspects:

1. Deploys Should Never Be Automated

This is a controversial point in a world where Heroku created the idea that deploys about to “git push” and magic happens. And, yes, I’ve worked on code bases where this works and it can be brilliant. But those environments were rarely scrappy, self funded startups. They were always places / code bases where you had these characteristics:

Well funded
Process oriented
Robust test coverage
Well documented
CI Server

These days – and for most of my career – I operate at the other end of the spectrum:

Poorly funded
Process poor / anti process
Tiny engineering teams of 1 or 2
Code review is something that rarely happens
Test coverage is an evolving, ahem, work in process
I never, ever use Heroku – I’m cheap and dyno’s are weirdly priced
Never once have I had a CI server on a project I bootstrapped from nothing

2. Test Coverage

Test coverage in a dynamic language like Ruby is absolutely essential and I’ll defend that viewpoint to my very grave but as important as test coverage is – it can never catch everything. When you are run an engineering team of 1 or 2 there is always the chance that things happen. Just as an example, there is some weirdness in OSX where copying a filename can inadvertently, in my editor, lead to that being pasted into the current code window. Now sometimes that will cause an error but Rails is a swiss watch of complexity and that one thing, in the wrong file, can cause a significant issue. Here’s an example of a single character error which isn’t caught easily but can break the world:

<%=# add_spacer %>

3. Syntax Checking

Even if you have 100% test coverage on your Rails app, I would wager that you don’t have coverage on your Rake tasks. Test coverage on Rake tasks has always been problematic in my experience and the reason that matters is the way that Rails loads your application in production:

All executable code in all directories is loaded when your Rails application is initialized.
A syntax error in your Rake task can prevent your application from starting.

Here is a Rake task which you can incorporate as a first step in a deploy pipeline which syntax checks all your .rb files:

namespace :code do
  desc "Check Ruby files for syntax errors"
  task :syntax do
    failed = false
    Dir.glob("**/*.rb").each do |file|
      next if File.directory?(file)
      # Skip anything in the docs directory because it isn't a code path subject to execution
      next if file =~ /^docs\//
      result = `ruby -c #{file} 2>&1`
      unless result.include?("Syntax OK")
        puts "[ERROR] #{file}:\n#{result}"
        failed = true
      end
    end
    abort("Syntax errors found!") if failed
    puts "All Ruby files passed syntax check."
    
    failed = false
    Dir.glob("**/*.rake").each do |file|
      next if File.directory?(file)
      # Skip anything in the docs directory because it isn't a code path subject to execution
      next if file =~ /^docs\//
      result = `ruby -c #{file} 2>&1`
      unless result.include?("Syntax OK")
        puts "[ERROR] #{file}:\n#{result}"
        failed = true
      end
    end
    abort("Syntax errors found!") if failed
    puts "All Rake tasks passed syntax check."
  end
end

4. Post Deploy Testing

As noted above, I don’t believe in automated deploys. I strongly, strongly, strongly believe in the traditional build master approach to software development when you don’t have 100% test coverage, code reviews and rigorous engineering practices. In these environments, I believe that deploy should be a deliberate action that a person on the team should own responsibility for. It might be multiple people but:

Someone has to push the deploy button.
That deploy process has to happen with a human in the loop monitoring things.
At the end of that deploy process, something should run which gives some level of sanity “ok at a bare minimum, this is good”.

Here’s a simple post-deploy testing rake task which can be baked into a deploy pipeline:

namespace :post_deploy do
  # bundle exec rake post_deploy:check --trace
  task :check => :environment do 
    agent = Mechanize.new
    agent.user_agent = "FuzzyBot/1.0 (+https://example.com/bot-info)"
    
    urls = []
    urls << "https://www.pollitify.com/pages/SoupForOurFamiliesGrants-press-release-soup-for-our-families"
    urls << "https://www.pollitify.com/grants"
    urls << "https://www.pollitify.com/"
    urls << "https://www.pollitify.com/events/"
    urls << "https://pollitify.com/events/u2dTFDo2-berkeley-rush-hour-resistance"
    urls << "https://www.pollitify.com/early"
    urls << "https://www.pollitify.com/news_feed_items"
    urls << "https://www.pollitify.com/posts"
    urls << "https://www.pollitify.com/users"
    urls << "https://www.pollitify.com/badges"
    urls << "https://www.pollitify.com/posts/new"
    urls << "https://www.pollitify.com/home/index?state_id=5"
    urls << "https://www.pollitify.com/polls/"
    urls << "https://www.pollitify.com/pages/"
    urls << "https://www.pollitify.com/user_logins/"
    urls << "https://www.pollitify.com/press/"
    urls << "https://www.pollitify.com/grants/"
    urls << "https://www.pollitify.com/mutualaid/"
    urls << "https://www.pollitify.com/mutual_aid/"
    urls << "https://www.pollitify.com/mutual-aid/"
    
    
    puts "========================================================="
    puts "HINT -- IF THE rake post_deploy:check"
    puts "fails then check the home page in logged out status"
    puts "========================================================="
    urls.each do |url|
      puts "About to get url: #{url}"
      page = agent.get(url)      
      if page.code != "200"
        raise "Error on: #{url}"
      else 
        puts "  Success!!!"
      end
    end
  end
end

This is not perfect by any means and it is exactly as simple as its brevity would suggest but it runs every single time on every one of our deploys and more than once it has caught something important.

Pulling it All Together

I use Kamal for deploy and I pull all this together with a simple shell script I run for every deploy:

#!/bin/bash
set -e
ECHO "STEP 1: Making Sure Rails Code Passes Syntax Check"
bin/rails code:syntax
# nvm use 22
# bin/rails test

ECHO "STEP 2: Running Deploy"
source .env.production

bundle exec kamal deploy

ECHO "STEP 3: Running Site Map Creation"
# Step 2: Find the running web container
WEB_CONTAINER=$(ssh root@1.2.3.4 "docker ps --filter 'name=SOMETHING-web' --format ''")

if [ -z "$WEB_CONTAINER" ]; then
  echo "Error: Could not find running web container"
  exit 1
fi

echo "  Running sitemap task inside container $WEB_CONTAINER..."

# Step 3: Run the sitemap rake task inside the container
ssh root@1.2.3.4 "docker exec -u rails $WEB_CONTAINER bundle exec rake sitemap:create"

echo "Sitemap generation complete."

unset DATABASE_URL

ECHO "STEP 4: Making Sure Site Still Works"
bundle exec rake post_deploy:check

A Closing Thought: Pay Attention

My final point on deploy is really, really simple: pay attention when you deploy. I’d argue that if we were in the middle ages, our map for a Rails deploy would include the words “here there be dragons”. Rails is still my favorite platform I’ve ever worked on but I’m just under no illusions when it comes to Rails deployment. Pay attention when you deploy because it can be very, very fragile.