Cenqua has released Clover 1.3.2, a $250 payware unit test coverage tool. This is a bug fix release.
Clover modifies the source code to enable it to follow which statements are executed when, and keeps a running count of how many times each statement is executed during the test suite. Any statement that executes zero times is not being tested. Unlike Jester, Clover
only tests whether the tests execute each statement and follow each branch. (It occasionally misses branches on the edges of >= or <=.)
It does not test whether the tests correctly detect bugs. On the other hand, it runs orders of magnitude faster than a tool like Jester does. It's easy to use Clover several times a day. Indeed you can use it after each change to the test suite. By contrast, a Jester run can take several days to complete.
Ideally you'd want to use both a tool like Jester and a tool like Clover since they do different things.
Clover has been a major help in developing XOM. It has located numerous bugs in XOM over the last year, and is largely responsible for the completeness of XOM's test suite. Clover has also helped to optimize XOM for both speed and size by finding dead, unreachable code I could cut out. As usual, I tested the
new release on the current XOM code base.
This run didn't find anything I didn't already know about, but it did inspire me to write tests for a few uncovered areas, mostly code that
worked around bugs or used special features of particular XML parsers. Wouldn't you know it? There was a major bug hiding in one place in the untested code (now fixed).
At this point XOM achieves over 90% test coverage, and most of the things that aren't tested are unreachable code like catch blocks for exceptions that won't be thrown or extra case statements included just to allow the switch statement to be compiled with the tableswitch instruction. Still, they're a few untested lines of code I think I could cover, though it's getting harder. This latest batch of tests required implementing my own pseudo-parser that could call methods the normal SAX parser wouldn't call. Expanding the test coverage further might require implementing my own ClassLoader to force other bugs to surface. These cases mostly reflect workarounds for known bugs in particular libraries, platforms, and virtual machines that don't normally appear with other libraries, platforms, and virtual machines.
Clover is one of the few tools I can recommend unconditionally, and would certainly pay for if Cenqua hadn't given me a free copy. (That's not personal, by the way. They'll happily provide free copies to all open source projects.) Clover integrates with Ant, NetBeans, Eclipse, and IntelliJ IDEA. new in this release is support for Oracle JDeveloper 10g. Clover can generate test coverage reports in XML, HTML, PDF, or via a Swing Viewer. Java 1.2 or later is required.