03 May 2010 So how much slower are instantiated fixtures?
On an early Rails project I elected to use instantiated fixtures. My rationale was that it would make my test code cleaner and easier to understand. Sure, test_helper.rb warned me that they would be slow, but how bad could it be?
Well, pretty bad as it turns out. The average execution time of my test suite (with 1052 tests) using instantiated fixtures was around 590 seconds. That’s almost 10 minutes.
Suspecting that it might be the instantiated fixtures that were slowing things down, I modified a couple of tests to see if they could be sped up by switching to standard fixtures. The results were promising, so I switched everything.
The average execution time fell to 123 seconds. That’s a factor of 4.77 difference. In other words, my test time dropped from almost 10 minutes to just over two minutes.
The funny part is that instantiated fixtures didn’t make the code that much cleaner anyway, the reason being that model names would often end up embedded in fixture names. For example, if I had a User model, I might end up referencing a fixture as follows:
@user_in_more_than_one_group
The fixture was named this way because it wouldn’t make much sense if the name wasn’t prefixed with ‘user’. But when you think about it, there’s not much difference between this and the following:
users(:in_more_than_one_group)
So when I shifted away from instantiated fixtures, my code got only slightly more verbose, but the test execution time dropped by almost 80%.
Now it could be argued that these aren’t particular good fixture names, and that if they are for testing specific scenarios, I’d be better off using something like Factory Girl or Machinist. That’s a good point, but unfortunately I’m not in a position to make that change at the moment.
The bottom line is that I got caught out by just how slow instantiated fixtures are. Your results may differ, but it’s definitely worth investigating if your tests are slow and you’re using instantiated fixtures.
No Comments