10 seconds is a very long time for any single test to run. My gut feeling is that your spec target is running both unit and integration tests at the same time. This is a typical thing that projects fall into and at some stage, you will need to overcome this technical debt if you want to produce more, faster. There are a number of strategies which can help you to do this… and I’ll recommend a few that I have used in the past.
1. Separate Unit From Integration Tests
The first thing I would do is to separate unit from integration tests. You can do this either by:
- Moving them (into separate folders under the spec directory) – and modifying the rake targets
- Tagging them (rspec allows you to tag your tests)
The philosophy goes, that you want your regular builds to be quick – otherwise people won’t be too happy to run them often. So get back to that territory. Get your regular tests to run quick, and use a continuous integration server to run the more complete build.
An integration test is a test that involves external dependencies (e.g. Database, WebService, Queue, and some would argue FileSystem). A unit test just tests the specific item of code that you want checked. It should run fast (9000 in 45 secs is possible), i.e. most of it should run in memory.
2. Convert Integration Tests To Unit Tests
If the bulk of your unit tests is smaller than your integration test suite, you have a problem. What this means is that inconsistencies will begin to appear more easily. So from here, start creating more unit tests to replace integration tests. Things you can do to help in this process are:
- Use a mocking framework instead of the real resource. Rspec has an inbuilt mocking framework.
- Run rcov on your unit test suite. Use that to gauge how thorough your unit test suite is.
Once you have a proper unit test(s) to replace an integration test – remove the integration test. Duplicate testing only makes maintenance worse.
3. Don’t Use Fixtures
Fixtures are evil. Use a factory instead (machinist or factorybot). These systems can build more adaptable graphs of data, and more importantly, they can build in-memory objects which you can use, rather than load things from an external data source.
4. Add Checks To Stop Unit Tests Becoming Integration Tests
Now that you have faster testing in place, time to put in checks to STOP this from occurring again.
There are libraries which monkey patch active record to throw an error when trying to access the database (UnitRecord).
You could also try pairing and TDD which can help force your team to write faster tests because:
- Somebody’s checking – so nobody gets lazy
- Proper TDD requires fast feedback. Slow tests just make the whole thing painful.
5. Use Other Libraries To Overcome The Problem
Somebody mentioned spork (speeds up load times for the test suite under rails3), hydra/parallel_tests – to run unit tests in parallel (across multiple cores).
This should probably be used LAST. Your real problem is all the way in step 1, 2, 3. Solve that and you will be in a better position to role out additional infrastructure.