GitHub Actions allow you to create custom continuous integration (CI) workflows directly in your GitHub repository. When you commit code to your repository, you can continuously build and test the code to make sure that the commit doesn't introduce errors.
You will of course be testing your code locally but using CI provides a fail-safe to ensure that the tests have been run on desired target operating systems and gives other developers & consumers of your code the certainty that the code has been fully exercised.
A test suite providing full code coverage is crucial but so is testing your code with different versions of Ruby and different gem dependencies. This is especially important when testing a Ruby gem that can be used in conjunction with combinations of Ruby and dependencies that differ from the development setup. Such testing will highlight issues within the gem code resulting from changes in these components like methods that are only available in later versions, methods that have been removed, deprecation warnings and breaking changes
This article is going to explain how to use GitHub Actions to run your tests against both different versions of Ruby and different gem dependencies. This is achieved by using a combination of a matrix strategy and the Appraisal gem, a Ruby library for testing your code against different versions of dependencies.
The GitHub documentation for building and testing Ruby includes an explanation of how to use a matrix strategy to run tests against different versions of Ruby along with a sample workflow file. However, it doesn’t include information on how to extend the strategy to include different versions of your gem dependencies.
The first step is to add the Appraisal gem from thoughtbot to your project and set up your “appraisals” to test your code against different versions of dependencies. The gem documentation explains how to set up your “appraisals” so it’s not repeated here but do note the recommendation in the Version Control section not to check in your lockfiles.
Once you have “appraisals” in place you’ll have a number of gemfiles specifying different versions for your dependencies eg.
# This file was generated by Appraisal
source "https://rubygems.org"
gem "activerecord", "~> 6.0.1"
gem "activesupport", "~> 6.0.1"
gemspec path: "../"
The next step is to create a workflow file in the .github/workflows directory. You can start with one of the samples listed on the Building and testing Ruby page. The GitHub Actions documentation includes lots of information on workflow files and the various options like setting the type of machine & operating system to use. Concentrating on the task in hand, let’s look at how to define matrix strategies for different versions of Ruby and gem dependencies.
Before you start, you need consider whether you need:
- a matrix that will test every version of Ruby with every variation of gem dependencies, or
- a matrix to test specific combinations of Ruby and gem dependencies
If you are creating a matrix for a gem with a dependency on Rails in order to run tests with different versions of Rails and Ruby then you’re likely to need option 2. That option will account for the fact that different versions of Rails are only compatible with certain versions of Ruby.
For a matrix that will test every version of Ruby with every variation of gem dependencies, you create a matrix like this:
strategy:
fail-fast: false
matrix:
ruby-version:
- '2.6'
- '2.7'
- '3.0'
gemfile:
- 'activesupport_5_0'
- 'activesupport_5_1'
- 'activesupport_5_2'
- 'activesupport_6_0'
- 'activesupport_6_1'
The gemfile names should match your Appraisal gemfiles and with fail-fast set to false GitHub will cancel all in-progress jobs if any matrix job fails.
For a matrix to test specific combinations of Ruby and gem dependencies, you create a matrix like this:
strategy:
matrix:
ruby-version: ['2.5']
gemfile: ['rails_6_0']
include:
- ruby-version: '2.5'
gemfile: 'rails_6_1'
- ruby-version: '2.6'
gemfile: 'rails_6_0'
- ruby-version: '2.6'
gemfile: 'rails_6_1'
- ruby-version: '2.7'
gemfile: 'rails_6_0'
- ruby-version: '2.7'
gemfile: 'rails_6_1'
- ruby-version: '2.7'
gemfile: 'rails_7_0'
- ruby-version: '3.0'
gemfile: 'rails_6_1'
- ruby-version: '3.0'
gemfile: 'rails_7_0'
This matrix defines the Ruby version and gemfile to use for the first job then in the include section lists further combinations for additional jobs. With the matrix defined in the workflow file, the next requirement is to set the BUNDLE_GEMFILE environment variable to tell Bundler which gemfile to use for the matrix job, instructing Bundler to use the Appraisal gemfiles. Also note, this env key must be located in line with the strategy key and outside of the steps key.
env:
BUNDLE_GEMFILE: $/gemfiles/$.gemfile
Now within the steps key add an entry to use the matrix Ruby version and include bundler-cache to run bundle install and cache the installed gems automatically.
- name: Set up Ruby $ and bundle $
uses: ruby/setup-ruby@v1
with:
ruby-version: $
bundler-cache: true
With those elements in place you will have a workflow file along these lines.
name: CI
on: [push,pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
ruby-version:
- '2.6'
- '2.7'
- '3.0'
gemfile:
- 'activesupport_5_0'
- 'activesupport_5_1'
- 'activesupport_5_2'
- 'activesupport_6_0'
- 'activesupport_6_1'
env:
BUNDLE_GEMFILE: $/gemfiles/$.gemfile
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Ruby $ and bundle $
uses: ruby/setup-ruby@v1
with:
ruby-version: $
bundler-cache: true
- name: Run the tests
run: bundle exec rspec
Now you’re all set and once you’ve pushed the changes to GitHub, navigate to the Actions for your repository. That will show you the overall status of the workflow run and opening the run will show the status of each of the matrix jobs.
Get in touch
Looking for help with setting up your GitHub Actions and creating CI workflows? With over 25 years software development experience we’re confident we’ve travelled down your road before and can offer the help and support you need at any stage of your development cycle. Contact us today by emailing hello@circle-sd.com or filling in our contact form.