Headless Continuous Integration With RSpec + Capybara + CruiseControl.rb

I have recently setup a headless continuous integration test server that runs in a headless environment. It runs both my rspec tests and all of my capybara request tests. If I receive a failure in a request test, it will store a screenshot and the HTML code of the page.

There is a few key components to getting this setup correctly, I thought I would share insight on what I needed to do in order to get it working.

My setup consists of:

  • A rails app with capybara and rspec tests.
  • A headless server which has Xvfb, CruiseControl.rb and Firefox 3.6.x installed.

First off, you will need the following entries in your Gemfile:

Gemfile
1
2
3
4
gem "rspec-rails"
gem "capybara"
gem 'headless', :git => "https://github.com/masatomo/headless.git", :branch => "fix_exit_code"
gem "selenium-webdriver"

Now, you will need to configure your spec_helper.rb file.

At the top of the file add:

spec_helper.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
def example_filename(metadata)
  [metadata[:file_path].split(Rails.root.to_s)[1].gsub(/[\/\.]/, "_"), metadata[:line_number]]
end

def save_screenshot(example, headless)
  if example.exception && example.metadata[:type] == :request && example.metadata[:js]
    require 'capybara/util/save_and_open_page'
    save_base_path = "#{Rails.root}/tmp"
    metadata = example_filename(example.metadata)
    save_filename = "#{metadata[0]}_#{metadata[1]}"
    FileUtils.mkdir_p("#{save_base_path}/screenshots")
    Capybara.save_page(body, "#{save_filename}.html")
    screenshot_path = "#{save_base_path}/screenshots/#{save_filename}.png"
    if headless
      headless.take_screenshot(screenshot_path)
    elsif Capybara.current_driver == :selenium
      Capybara.page.driver.browser.save_screenshot(screenshot_path)
    else
      #For other drivers that support a plain .render call and only expect the path as a parameter
      #This includes e.g. capybara-webkit
      if capybara.page.driver.respond_to?(:render)
        capybara.page.driver.render(screenshot_path)
      end
    end
  end
rescue => e
  puts "Exception saving screenshot"
  puts e
  puts e.backtrace.join("\n")
end

if ENV['HEADLESS'] == 'true'
  require 'headless'
  headless = Headless.new
  headless.start
  at_exit do
    exit_status = $!.status if $!.is_a?(SystemExit)
    headless.destroy
    exit exit_status if exit_status
  end
end

Selenium::WebDriver::Firefox::Binary.path = ENV["FIREFOX_BIN"] if ENV["FIREFOX_BIN"]

The example_filename method is used to create a custom filename which we use to store screenshots and html dumps from failed request tests.

The save_screenshot method handles saving the screenshots and html dumps.

The next block allows us to run our tests in headless mode. Remember, you will need to have Xvfb installed to run the tests in headless mode.

The last line allows you to specify a custom firefox binary to use. Because we have found latest versions of firefox do not play nice with selenium (which capybara uses), we install firefox 3.6 into ~/firefox-3.6 and use FIREFOX_BIN=~/firefox-3.6/firefox.

Now, inside of your RSpec.configure do |config| block, put the following:

spec_helper.rb
1
2
3
4
5
6
7
8
9
10
  config.before(:each) do
    if example.metadata[:type] == :request and example.metadata[:js]
      Capybara.current_driver = :selenium
    end
  end

  config.after(:each) do
    save_screenshot(example, headless)
    Capybara.use_default_driver
  end

The first block sets the selenium driver for request tests. The second block resets the driver back to a non-selenium driver and also calls the save_screenshot method that we defined earlier.

Now, we will need a custom script that runs our tests in CruiseControl.rb

run-tests.sh
1
2
3
4
5
6
7
8
9
10
#!/bin/bash -l

rvm rvmrc load .rvmrc || exit 1
cp ../database.yml config/
mkdir -p log
RAILS_ENV=test ./bin/rake db:drop db:create db:schema:load && HEADLESS=true ./bin/rspec spec
EXIT_CODE=$?
mv tmp/capybara $CC_BUILD_ARTIFACTS/capybara
mv tmp/screenshots $CC_BUILD_ARTIFACTS/screenshots
exit $EXIT_CODE

We use rvm, so the 3rd line handles loading the correct rvm environment. Then we reset the DB and start our test run.

We also ensure that the test script exits with the right exit code.

Now, we setup out cruise_config.rb file as well.

cruise_config.rb
1
2
3
4
5
6
7
8
9
10
11
Project.configure do |project|
  project.email_notifier.emails = ['brian@tecnobrat.com']
  project.email_notifier.from = 'noreply@tecnobrat.com'
  project.build_command = 'bash -l run-tests.sh'

  project.scheduler.polling_interval = 1.minutes
  project.scheduler.always_build = false

  project.triggered_by ScheduledBuildTrigger.new(project, :build_interval => 1.day, :start_time => 15.minutes.from_now)
  project.bundler_args = "--path=#{project.gem_install_path} --gemfile=#{project.gemfile} --no-color --binstubs"
end

This is a pretty standard cruise_config.rb file only extra thing is that we run the custom bash script we created.

Now cruise control should be able to run your tests, and if any request tests fail, you will get a screenshot and a html dump added to your build artifacts in CC.rb

Some gotchas

  • Ensure Xvfb is installed before running with HEADLESS=true
  • Use an older version of firefox, it works best with selenium (use FIREFOX_BIN=path/to/bin)
  • You need the fix_exit_code branch of headless, so ensure you have specified that in your Gemfile

So when you combine all of that together, you should be able to get a similar setup. If you have any questions please ask them in the comments!

Comments