For quite some time now I’ve been having using, or at least had an eye on, tools to measure different kind of code metrics. And I’ve been torn by their usefulness. I’ve been amazed by Clover and used it with pleasure, but there’s always the risk of being dogmatic about code metrics. The question is, who do you work for, yourself, and therefore the project you’re working on, or the tool?
I’ve seen colleagues sitting hours to make their code look good in Checkstyle’s reports, and I’ve caught myself working down that list of “errors” in an almost endless manner. Talking to friend lately about a project he’s joined, he told me that he checked in his first piece of code and not long after he got contacted by the developers telling him that he caused a lot of Checkstyle errors. This got me to thinking about code metrics and code style. Well, a recent discussion on the Pragmatic Programmers Mailing List may have added some thoughts as well.
The first thing are those code metrics. Robert C. Martin did a good job at introducing one of those metrics which I consider as useful. JDepend is a simple, yet useful implementation of Martin’s metrics, and I like to use it, though not obey it, as much as I can. Static code analysis can be a tricky thing, as you can see in FindBugs. It’s one of those tools that can find real bugs (or, if that sounds better to you, common programming mistakes), but can find a lot of false-positives as well. So there’s a thin line here. You can spend a lot of time chasing those “bugs” until you conclude that there is no bug (you’ve expected the word spoon here, didn’t you?).
But then how do you decide on what needs fixing or not? If you have thousands of those “nasty” Checkstyle errors in front of you, do you ignore them (and therefore tend to make this kind of code metric useless) or do you work from the list and fix them all. Or, even worse, do you change the Checkstyle configuration to include fewer checkers to reduce the overall number of errors and let your code formatter do the rest? I’ve seen and done all three of those and lately I asked myself what’s the need for tools like Checkstyle then? Is anybody really giving in to those numbers? If you have one of those huge documents describing the code style of your company, do you really enforce these?
When it comes to getting around enforcement like that, programmers tend to get very creative and do all sorts of stuff just to program the way they’re used to. Getting back to the discussion on the Pragmatic Programmers Mailing List (which is a terrific read, by the way), here are some examples. There was the possibility of having a code formatter run over your code on check-out and then again on check-in. On check-out the code is formatted to the programmer’s style and on check-in it’s being converted back to conform to the company’s coding conventions. Now that’s creativity! But hey, that’s what we’re being paid for, right?
Another way of “enforcing” code conventions is having a formatter hook into your SCM and format checked-in code as it arrives. I don’t want to see that merge, when you check in a file (being reformatted on the SCM server), decide to make some big changes to the code and then check in again. If programmers tend
But anyway, code conventions are another item that could be discussed in an endless manner. Those were just examples of programmer creativity to avoid being forced to do something with or about code metrics.
All that make me think about how useful those metrics really are. JDepend can give you really useful information about how loosely/closely coupled your code is. It has a steep learning curve and you’ve gotta learn to interpret the results correctly and draw your conclusions appropriately. But if you think again, that’s what you’ll have to do with every single one of these tools. With JDepend it just becomes more apparent that the results are not that easy to interpret. Having a list of formatting errors or bugs in front of you is tempting for a novice to work down that list until everyone is happy and cheers because Checkstyle doesn’t give any errors at all.
The same goes for Clover. While it’s a great tool for developers and managers alike to see the quality of the code ensured in some way (unit tests in this way), it’s hard to judge just how much code coverage is acceptable. Using test-driven development will ensure in its very own ways that your code will have a good coverage. But again, it’s a judgement call, if it’s necessary for every single method being tested or if you’re better off testing as much state as necessary to ensure a correctly working class or code base. I definitely prefer the latter, but I’ve seen programmers bump up their tests to see the coverage go as high as possible. It might sound stupid, but I’m sure there are even some out there that bump up the logging level just to have these kinds of statements included in the coverage results:
if (logger.isDebugEnabled()) logger.debug("logging for the fun of it");
So the question is, what are those tools good for, if not for the programmer feel good about himself? Do they bring any value to the progress of a project? You gotta be very careful at choosing the right tools. You gotta be careful at configuring them. And you gotta teach your fellow programmers to interpret those results correctly. Being tempted to fix the errors in your code is always good karma, but spending too much time with fixing doesn’t help a project to succeed.
If you really take the time to learn, how those tools can help you, they’ll add some value to the project, your code, the way you develop. But you gotta keep it small. Having more and more of these tools run over your code to give you their opinion on your code will just make you panic and be an even bigger waste of time. I prefer having Clover and JDepend on my side, both in appropriate and not-time-wasting doses. And a little bit of Checkstyle as well, but not with every available checker turned on. Tools like FindBugs can be a nice way to check your code every now and then, but using them on a daily basis is overkill in my opinion.
Different stories or opinions, anyone?
And now, for the finishing touch: a Technorati link to claim this weblog ;)
