Speeding up Rails and RSpec

A couple of days ago I was asked by a friend for some help with speeding up a Rails app in general, and in particular with reducing the total time the RSpec suite was taking to run, which was over twenty minutes on my computer. Twenty minutes is not huge in the Rails world, but this app is growing quite quickly and both tests and the app in general are getting slower and slower as they add features etc.

For the testing, they were evaluating a switch to Minitest and fixtures from RSpec and factory_girl (since that combination tends to be faster) but this would take ages and anyway they (like me) prefer RSpec and factory_girl.

After two days looking into it I was pleased with the results: the app now boots in just ~3 seconds without Spring, assets compilation is ridiculously fast - which also helps during deployments - and the whole RSpec suite runs in just around 1’40” on my computer. Not bad!

There were many reasons why the app and the tests were that slow. The main reasons though were:

• lots of gems. Really, lots
• lots of assets, most of them through gems
• several missing databases indexes slowing things down in production
• almost all RSpec/Capybara features require JavaScript
• lots of data created in the database during tests, whereas in many placed build_stubbed would work just as fine
• lazy loading of associations here and there
• etc.

Most of the optimisations I’ve made are specific to this app, so I won’t get into the details about these. However I wanted to share a few small tips that have helped massively and that can be applied to any Rails/RSpec app.

First thing I did, was to find out whether all of the gems specified in the Gemfile were actually required. I found a few which basically were no longer in use, so I removed these.

Next, Bootstrap and other CSS/JavaScript libraries were loaded through gems; these assets don’t change often, so I usually prefer to just download them and place them in the vendor directory, rather than using gems. So I did this and removed those gems too, and the app was already booting a little more quickly.

By removing these gems, the app took around 8 seconds to boot without Spring, down from ~10. I then found the gem bootsnap by Shopify which does indeed speed up the boot process as it claims; the boot time went down to ~3 seconds by just adding this gem! The README does say that this gem is still “beta quality” but I haven’t seen any issues so far so fingers crossed.

I then looked into speeding up the assets compilation since this was affecting the app in development and during tests, but it was also slowing down the deployments. For starters, the app was loading the whole Bootstrap, jQuery UI and other things although only parts of these libraries were in use. So I made sure the app only loads what’s actually required, which did help. But the biggest surprise was discovering the sassc-rails gem! I had never heard of it before so it was a nice find. It’s basically a drop in replacement for the sass-rails gem, written in C (using LibSass), which is A LOT faster. You just replace the sass-rails gem with it, no other changes required and the results are amazing: assets compilation is almost instantaneous, so you can see a massive difference not only during deployments, but also running the app in development mode and in tests! Before, whenever some changes were made to JavaScript or CSS files, RSpec features requiring JavaScript would start up slowly due to the assets being compiled. Now you almost don’t notice any slow down.

Specifically for the tests, the biggest improvements were:

• upgrade to the latest version of Rails (5.1.4 at the moment) and use the master branch of the RSpec gems instead of the gems released on Rubygems.org; this is because of the JavaScript feature specs: when you run JavaScript features, Capybara starts an HTTP server to test against but in another thread, and the database connections are not shared between this server and the test threads, so data created in the test threads would not be visible to/accessible by the HTTP server when RSpec is configured to use transactional fixtures; without hacks (like patching ActiveRecord so to force different threads to share the same database connection), you had to disable the transactional fixtures for JavaScript features and use something like the database_cleaner gem to actually persist the data in the test database and do a cleanup after each test (using a deletion or truncation strategy), as opposed to transactions. Because the data was actually persisted in the database, the HTTP server started by Capybara would be able to “see” and use that data. Problem is, creating and then deleting data is typically slower than using database transactions, so the more tests you have, the bigger the difference. This was the case with the app I was talking about. With Rails 5.1+ however you no longer need to use database_cleaner, because database connections are now safely shared by the HTTP server and the test threads. RSpec is compatible with these changes in the master branch, so that’s why I upgraded Rails and the app is now using the master branch of the RSpec gems;
• switch to capybara-webkit by Thoughtbot. Before the app was using Poltergeist, and switching to capybara-webkit yielded a massive gain! I am not sure about the future of capybara-webkit, but for now it just is much, much faster than Poltergeist or Firefox/Chrome (even headless) with Selenium.

These changes reduced dramatically the time taken for the feature specs to run down to ~1’40”.

All in all I am very pleased with the results, especially because it didn’t take too much time. In this quick post I haven’t gotten into the details of how to implement the suggested changes because it’s easy to follow the READMEs on Github, but please let me know in the comments if these work for you.