Ivan Moore has asked me to take over maintenance and adminstration of his open source Jester unit testing tool. I have a love-hate relationship with Jester. On the one hand, it's based on a brilliant, radical idea. The underlying approach (changing the code and seeing what doesn't break) almost always reveals interesting and useful information about your code base. At the same time, Jester is so incredibly difficult to configure and takes so long to run that I don't use it nearly as often as I should. My initial goal for the project is improving ease of use. I've already made some improvements to the Ant build file and the XML reports that you can now see in CVS. Next I hope to resolve some longstanding classpath issues so that not all jars need to be on the static classpath. I also hope to make it possible to run Jester on an existing source base rather than requiring a separate copy.
If you're interested in Jester, either as a user or a developer, I've set up jester-users and jester-devs mailing lists for ongoing discussion. Bug reports and feature requests should be entered in the appropriate sections on Sourceforge. Right now I'm focusing on the core of Jester. They're a few open RFEs that could easily be implemented independently if anyone's looking for a project. In particular, I think removing the dependence on mmmockobjects, and writing a Maven plug-in and an Ant task would be interesting initial tasks that don't require intimate knowledge of the Jester code base and algorithms, and could be executed fairly independently.
Cenqua has released Clover 1.3.6, a $250 payware unit test coverage tool. 1.3.6 is a bug fix release.
Clover modifies the source code to enable it to follow which statements are executed when, and keeps a running count of how many times each statement is executed during the test suite. Any statement that executes zero times is not being tested. Unlike Jester, Clover
only tests whether the tests execute each statement and follow each branch. (It occasionally misses branches on the edges of >= or <=.)
It does not test whether the tests correctly detect bugs. On the other hand, it runs orders of magnitude faster than a tool like Jester does. It's easy to use Clover several times a day. Indeed you can use it after each change to the test suite. By contrast, a full Jester run can take several days to complete.
Ideally you'd want to use both a tool like Jester and a tool like Clover since they do different things.
Clover has been a major help in developing XOM. It has located numerous bugs in XOM over the last year, and is largely responsible for the completeness of XOM's test suite. Clover has also helped to optimize XOM for both speed and size by finding dead, unreachable code I could cut out.
As usual, I tested the
new release on the current XOM code base. Unfortunately this time it failed. Since the last time I ran it, I'd added a UnicodeUtil class with some very large switch statements, and Clover wasn't able to handle this. Possibly the addition of the instrumentation code pushed the methods over Java's maximum method size. I'll have to check with Cenqua and see if they can fix that. In the meantime, I excluded that class from the coverage tests, and reran Clover. This time it finished. There were a couple of surprises. First of all my code coverage numbers were way down, from well over 90% to just over 60%.
The big problem was that the very large UnicodeUtil class
was showing 0% coverage. Actually, it is tested. It's just that I had to exclude it from Clover coverage.
The second surprise was that Jaxen got pulled in and counted in my statistics. Previously it had been referenced as a third party JAR file. However, now I'm compiling the latest version from source and bundling a stripped down version straight into XOM. The results here were actually quite useful because they showed me a number of Jaxen classes and even entire packages that XOM wasn't actually using. I can exclude these from the finished JAR file, further reducing its size. By the time I was done I'd cut out three complete packages and six assorted classes in other packages. Clover also identified a number of unused methods in Jaxen I'd also like to drop from the internal version XOM uses, but I don't know any easy way to do remove these automatically. If anyone does, please holler.
I use Clover with Ant, but there are also versions for NetBeans, Eclipse, IntelliJ IDEA, and Oracle JDeveloper 10g. Clover can generate test coverage reports in XML, HTML, PDF, or via a Swing Viewer. Java 1.2 or later is required.
jutils.com has released lint4j 0.8, a free-beer "static Java source code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis." Version 0.8 adds class file introspection, multiple formatters in the Ant task, the ability to read source files from archives, and file-based checking. I tested this out on the Jester code base. lint4j found a couple of places where exceptions are being ignored, and a lot of places where System.exit is called. It's not immediately clear if any of these are real bugs or not. I'll have to investigate further.