During the last week I spend some quality time fixing several hundred JUnit tests so that they can finally run again in CruiseControl. Before that they were broken for a few months, because noone had the time to fix them. Looking back, I can only recommend to never abandon your test harness. It’s a major pain to know that your testbed is gone. Especially when you’re doing a bigger refactoring. And over the last few months, our project saw a lot of these. So chances are that fixing the test harness will take more time the longer you wait to fix it. When I fixed the test suite, many of the existing and broken tests where not up-to-date anymore and I threw them out without looking back.
So what happened? I switched the project’s persistence layer to Hibernate. Now that’s not a major refactoring per se, because most of the action happened under the hood. The persistence layer has been well hidden via DAOs, so that shouldn’t be a problem. But switching to Hibernate was like jumping into cold water, because I had to build up experience with Hibernate first. So it took some time to fix all persistent classes and write mappings for them. So I deactivated the tests in CruiseControl and from that moment on, we oftentimes fell back into an old and well known mode known as feature-xyz-is-broken-but-it-did-work-for-me-yesterday-so-whats-the-deal? We’ve been running CruiseControl for about a year and we’ve stacked up more and more test cases since then. Over the last months I got very frustrated because of the missing test harness. So it was about time to fix them. And now? Well, it’s not the perfect world, but it feels a lot better to know that if someone introduces a bug the chances are much higher that it will be found by CruiseControl.
So what can you do, when someone abandoned your test harness? For god’s sake, fix it! If you can’t fix them all, fix them in iterations and include all running test cases into a new test suite. For the ones still failing, keep a list, or throw them out, if they’re not up-to-date anymore. If you’re not sure about a certain test case, comment it out as well, and talk to your colleagues about it. If it’s too big a test case to fix, then try to break it down into smaller ones, or throw it out, if it’s not reasonable to do so, and put it on the list as well.
You might as well switch to TestNG if you’re going through all this trouble upgrading your code to more recent technologies…
That task would’ve been even bigger than getting those tests back up and running again. But the idea is appealing ;)
Cheers, Mathias
Fair enough, but for the record, there are several tools to convert JUnit tests included in TestNG, ranging from individual classes (a Quick Fix in Eclipse) to converting entire hierarchies with one command…
I’m sure there are, but consider this: I’m glad my team is up to using JUnit. Neither I nor them have used TestNG until now. As good as it sounds, don’t forget about the learning curve. And right now I just can’t afford that learning curve, as small as it may seem. Right now tests need to be written and they need to be run. Everything is tuned for JUnit, so it would take some time to get everything and everyone going for TestNG. But I’ll definitely have a look at it anyway ;)
Cheers, Mathias
One approach that would have been worth trying in this is to have developed a parallel implementation of your persistence code for Hibernate. You could then have had parallel unit tests for it (by copying and pasting); these could have been disabled until you got them passing.
The advantage to this approach is that you don’t “break” your production code while you do this. In addition, you will probably end up with a nice abstraction layer over the persistence that will help isolate you from changes to Hibernate and protect you if you need to do something similar (for example: upgrading from Hibernate 2 to 3 is a huge exercise - Hibernate 4 will probably be the same).
I’m not entirely sure how TestNG would have helped here… the only thing that springs to mind is the “ignore” option….
Robert: That would’ve been a good option for sure. I could slap myself that I didn’t make the switch in a different branch. But it was a time, when production was quite low in the project, because many people spent time writing specs. Doesn’t really justify it, but oh well, lesson learned.
And I made the switch from Hibernate 2 to 3 only a few weeks later ;)
Fortunately it wasn’t that big of a deal because of the layered persistence.
Cheers, Mathias
jkghk…
jkghk…
Nice article !