During the last week I spend some quality time fixing several hundred JUnit tests so that they can finally run again in CruiseControl. Before that they were broken for a few months, because noone had the time to fix them. Looking back, I can only recommend to never abandon your test harness. It’s a major pain to know that your testbed is gone. Especially when you’re doing a bigger refactoring. And over the last few months, our project saw a lot of these. So chances are that fixing the test harness will take more time the longer you wait to fix it. When I fixed the test suite, many of the existing and broken tests where not up-to-date anymore and I threw them out without looking back.
So what happened? I switched the project’s persistence layer to Hibernate. Now that’s not a major refactoring per se, because most of the action happened under the hood. The persistence layer has been well hidden via DAOs, so that shouldn’t be a problem. But switching to Hibernate was like jumping into cold water, because I had to build up experience with Hibernate first. So it took some time to fix all persistent classes and write mappings for them. So I deactivated the tests in CruiseControl and from that moment on, we oftentimes fell back into an old and well known mode known as feature-xyz-is-broken-but-it-did-work-for-me-yesterday-so-whats-the-deal? We’ve been running CruiseControl for about a year and we’ve stacked up more and more test cases since then. Over the last months I got very frustrated because of the missing test harness. So it was about time to fix them. And now? Well, it’s not the perfect world, but it feels a lot better to know that if someone introduces a bug the chances are much higher that it will be found by CruiseControl.
So what can you do, when someone abandoned your test harness? For god’s sake, fix it! If you can’t fix them all, fix them in iterations and include all running test cases into a new test suite. For the ones still failing, keep a list, or throw them out, if they’re not up-to-date anymore. If you’re not sure about a certain test case, comment it out as well, and talk to your colleagues about it. If it’s too big a test case to fix, then try to break it down into smaller ones, or throw it out, if it’s not reasonable to do so, and put it on the list as well.