Further Modifications to JUnit

In the last installment, I demonstrated how an annotation type @Ignore could be used to mark methods so that JUnit would ignore them. While simply ignoring test methods is useful, it defeats my original purpose of ensuring that developers were cognizant of which methods have been ignored.

In order to make the ignored methods more visible, I’ve further modified JUnit to add a tab to the Swing UI that shows a list of the methods being ignored. Also, I’ve modified the @Ignore annotation type so that a developer can supply a reason for ignoring a test method. Here’s what the modified JUnit screen looks like:

(OK, so the icon needs work.) The modifications to the JUnit Swing user interface involved creating a new tab, Ignored Tests, that lists out the methods that have been marked as ignored.

Here is the test class code that generated the list of ignored methods in the above JUnit run:

    public class NodeTest extends TestCase {
       @Ignore(reason="nothing done yet")
       public void testSomething() {
       }
    
       public void testError() {
          throw new RuntimeException("error!");
       }
    
       public void testFailure() {
          fail("failure!");
       }
    
       @Ignore(reason="just because")
       public void testDeadlock() throws Exception {
          // ...
       }
    }

Member Value Pairs

Before, @Ignore took no parameters and thus was known as a marker annotation. Annotations can take parameters, as shown in the NodeTest code above. Note that the use of @Ignore as an annotation is a shortcut for the parameterless annotation @Ignore().

Now, @Ignore takes a member value pair as a parameter. A member value pair is a simple name, followed by an equals sign (=), followed by a value. The simple name in a member value pair must correspond to the name of a method in the annotation type declaration. In the annotation for testSomething above, the simple name is reason. The value is "nothing done yet".

In order to make @Ignore to support a reason annotation, I modified it to include a member named reason:

    package junit.framework;
    
    import java.lang.annotation.*;
    
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface Ignore {
       public String reason();
    }

Modifications to JUnit

In the last installment, I modified JUnit to simply ignore test methods marked with @Ignore. The required modifications to JUnit were trivial.

Now that I need to track the ignored methods so they can be displayed in the Swing UI, the modifications are a bit more involved. I’m not going to detail all the changes here, only some of the core ones. If you’re truly interested in seeing all the changes, send me an email.

Here is the chief modification to the class junit.framework.TestCase. I didn’t like throwing an exception (TestIgnoredException, a new RuntimeException subclass) to trigger an ignored test, but it kept me from having to modify the Test interface.

       protected void runTest() throws Throwable {
          assertNotNull(fName);
          Method runMethod= null;
          try {
             // use getMethod to get all public inherited
             // methods. getDeclaredMethods returns all
             // methods of this class but excludes the
             // inherited ones.
             runMethod= getClass().getMethod(fName, null);
          } catch (NoSuchMethodException e) {
             fail("Method \""+fName+"\" not found");
          }
          if (!Modifier.isPublic(runMethod.getModifiers())) {
             fail("Method \""+fName+"\" should be public");
          }
    
          if (runMethod.isAnnotationPresent(Ignore.class)) {
             Ignore ignore = runMethod.getAnnotation(Ignore.class);
             throw new TestIgnoredException(ignore.reason());
          }
    
          try {
             runMethod.invoke(this, new Class[0]);
          }
          catch (InvocationTargetException e) {
             e.fillInStackTrace();
             throw e.getTargetException();
          }
          catch (IllegalAccessException e) {
             e.fillInStackTrace();
             throw e;
          }
       }

If an Ignore annotation is present, I retrieve the Ignore object using the method getAnnotation. I can then use the methods defined in the @Ignore annotation type declaration.

Other Key JUnit Changes

I modified the method runProtected in junit.framework.TestResult:

    public void runProtected(final Test test, Protectable p) {
       try {
          p.protect();
       }
       catch (TestIgnoredException e) {
          addIgnoredTest(test, e.getMessage());
       }
       catch (AssertionFailedError e) {
          addFailure(test, e);
       }
       catch (ThreadDeath e) { // don't catch ThreadDeath by accident
          throw e;
       }
       catch (Throwable e) {
          addError(test, e);
       }
    }

The TestListener class, used by the UI classes, had to be enhanced:

    package junit.framework;
    
    public interface TestListener {
       // ...
       public void addIgnoredTest(Test test, String reason);
    }

In all honesty, my changes to JUnit seemed like a bit of a hack. I was lax to make dramatic, sweeping changes to the JUnit code (which are sorely needed). Not doing so made for an ugly set of changes.

Additional Member Value Pairs

You can define an annotation type so that it supports any number of member-value pairs in an annotation. In an annotation, you separate the member-value pairs with commas:

    @Ignore(reason="bogus!", initials="jjl")
    public void test1() {
    }

Each simple name of a member value pair must correspond to a method on the annotation type declaration. In this example, the new simple name initials corresponds to the @Ignore method initials(), defined here:

    package junit.framework;
    
    import java.lang.annotation.*;
    
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface Ignore {
       public String reason();
       public String initials();
    }

Annotation Type Member Return Types and Defaults

An annotation type member cannot take any parameters. Its return type must be one of String, any primitive type, an enum type, Class, an annotation type itself, or an array of any of the preceding types.

An annotation type member can have a default value. The existence of a default for an annotation member means that the user is not required to supply a member value pair for the corresponding simple type.

I’ve modified the @Ignore annotation type here to demonstrate the default capability as well as some of the various return types.

    package junit.framework;
    
    import java.lang.annotation.*;
    
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface Ignore {
       public String reason();
       public String initials();
       public String[] relatedTestMethods();
       public boolean isForPerformance() default false;
       public Class[] relatedTestClasses();
    }

Here’s a sample @Ignore annotation using the above annotation type declaration.

    @Ignore(reason="bogus!",
       initials="jjl",
       relatedTestMethods={"test2", "test3"},
       relatedTestClasses=Test.class)
    public void test1() {
    }

As demonstrated, member value pairs can be in any order since they are keyworded.

Also, note that the member relatedTestClasses is defined as returning a Class array. Yet the annotation supplies only the single value Test.class, without any array braces. This is a shortcut: braces are not required for single-element array values.

Single Member Annotation Types

If you supply a single member named value for an annotation type, users of the annotation type need not provide a simple name:

    public @interface Ignore {
       String value();
    }

This example allows for annotations such as @Ignore("bogus!"). You can also have the value member return a String array. The annotation type declaration:

    public @interface Ignore {
       String[] value();
    }

allows:

    @Ignore({"bogus!", "baby"})

Complex Annotation Types

You can create complex annotation types by having an annotation type member return an annotation type itself:

    import java.lang.annotation.*;
    
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface Ignore {
       String reason();
       Date date();
    }

Date is defined as:

    import java.lang.annotation.*;
    
    public @interface Date {
       int month();
       int day();
       int year();
    }

Note the absence of @Retention and @Target meta-annotations on Date; they are not necessary.

Here is an example use of the now-complex @Ignore annotation:

    @Ignore(reason="bogus!", date=@Date(month=11, day=5, year=2004))
    public void test1() {
    }

Final Notes on the Annotations Facility

Here are some odds and ends not covered in these two articles:

• You can only define one target for an annotation; you cannot, for example, declare that @Ignore modifies both methods and first-level types.

• The @Documented meta-annotation type lets you declare an annotation type to be included in the published API generated by tools such as javadoc.

• The @Inherited meta-annotation type means that an annotation type is inherited by all subclasses. It will be returned if you send getAnnotation to a method or class object, but not if you send getDeclaredAnnotations.

• The Method and Constructor classes have been supplemented with a getParameterAnnotations method.

• In order to internally support annotation types, Sun modified the Arrays class to include implementations of toString and hashCode for arrays.

• When adding new members to an annotation type, you should provide a default to avoid breaking existing code. Refer to the specification document for information on compatibility issues surrounding annotations.

• You cannot use null as an annotation value.

Summary

Annotations are a powerful facility that can help structure the notes you put into your code. One of the examples that is touted the most is the ability to annotate interface declarations so that tools can generate code from the pertinent methods.

The downside is that you make your code dependent upon an annotation type when your code uses it. Changes to the annotation type declaration could negatively impact your code, although Sun has built in some facilities to help maintain binary compatibility (see the specification document). You also must have the annotation type class file present in order to compile. This should not be surprising; an annotation type is effectively an interface type. There is an interesting discussion in the specification document about this dependency and some of the discarded ideas for minimizing it.

An exercise left to the reader is to modify the JUnit implementation to use annotation types such as @TestClass and @TestMethod for recognizing test classes and methods. This is how NUnit currently works. There are two significant benefits: JUnit test classes would no longer have to follow the reflections-based conventions for naming test methods, and your test classes would no longer have to inherit from a concrete superclass (junit.framework.TestCase).

It’s about time for a JUnit rewrite, anyway. I hope to see a version 4.0 of JUnit with the introduction of J2SE 5.0. [[ JUnit 4.0 is recently available. JjL, 8/25/2005 ]]

Reference: Java Community Process public review document, “JSR-000175: A Metadata Facility for the JavaTM Programming Language,” located at [ http://jcp.org/aboutJava/communityprocess/review/jsr175/index.html][1]