What's new in JUnit 5?

For the purposes of investigating JUnit 5 I have created a small sandpit project which will contain all the code examples that appear in this post. This is by no means an exhaustive analysis of JUnit 5, I will look at getting JUnit 5 up and running first of all. Then I will look at some of the interesting features I expect I am most likely to use.

Migrating from JUnit 4

Setting up the project to run JUnit 5 tests

My first starting point was to create a Maven project with a few classes and a test class written with JUnit 4. Then I had a look to see what was necessary to get it running as a JUnit 5 test.

One of the fundamental changes made in JUnit 5 is to break up the code used to write tests from the code used to execute the tests. Because of this separation it is relatively easy to execute JUnit 4 tests using JUnit Platform part of JUnit 5. All that is necessary is to do is update the Surefire plugin dependencies.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.19.1</version>
  <dependencies>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-surefire-provider</artifactId>
      <version>${junit.platform.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.vintage</groupId>
      <artifactId>junit-vintage-engine</artifactId>
      <version>${junit.vintage.version}</version>
    </dependency>
  </dependencies>
</plugin>

The version is set to 2.19.1 as at the moment the most recent version (2.20.1) does not work with JUnit 5. junit-platform-surefire-provider adds support for the new JUnit Platform to Surefire and junit-vintage-engine provides a test engine that can run JUnit 3 and 4 test. That is it you can now run your existing tests using the new JUnit Platform.

Having taken this first step, it is only another small step to start adding JUnit 5 tests.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.19.1</version>
  <dependencies>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-surefire-provider</artifactId>
      <version>${junit.platform.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit.jupiter.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.vintage</groupId>
      <artifactId>junit-vintage-engine</artifactId>
      <version>${junit.vintage.version}</version>
    </dependency>
  </dependencies>
</plugin>

JUnit Jupiter is the part of JUnit 5 that defines the new API for writing tests and here junit-jupiter-engine is the test engine to run these tests on the JUnit Platform. And as you would expect we also need to add the Jupiter API to the project dependencies.

<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter-api</artifactId>
  <version>${junit.jupiter.version}</version>
</dependency>

Your build will now run all your JUnit 4 and 5 tests. So there is no need to migrate all your tests immediately you can just start adding in new JUnit 5 tests.

Converting an existing JUnit 4 test

The easiest way to see what a Jupiter test looks like is to take a JUnit 4 tests and convert it over. For the examples in post I have turned to Star Trek and created some simple classes that mirror some of the functionality of the Enterprise’s main computer. This first example looks at the system to control who has access to what systems as the ship is put into different states of alert.

package com.scottlogic.tng;

import org.junit.Before;
import org.junit.Test;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

public class AccessControlJUnit4Test {

    private AccessControl testee;
    private User picard = new User("Jean-luc Picard", UserType.BRIDGE_CREW);
    private User barclay = new User("Reg Barclay", UserType.CREW);
    private User lwaxana = new User("Lwaxana Troi", UserType.NON_CREW);
    private User q = new User("Q", null);

    @Before
    public void setUp() {
        testee = new AccessControl();
    }

    @Test
    public void bridgeCrewCanAccessReplicatorsWhenYellowAlert() {
        testee.setAlertStatus(Alert.YELLOW);
        assertTrue(testee.canAccessReplicator(picard));
    }

    @Test
    public void crewCanAccessReplicatorsWhenYellowAlert() {
        testee.setAlertStatus(Alert.YELLOW);
        assertTrue(testee.canAccessReplicator(barclay));
    }

    @Test
    public void nonCrewCannotAccessReplicatorsWhenYellowAlert() {
        testee.setAlertStatus(Alert.YELLOW);
        assertFalse(testee.canAccessReplicator(lwaxana));
    }

    @Test
    public void userWithNullUserTypeCannotAccessReplicatorsWhenYellowAlert() {
        testee.setAlertStatus(Alert.YELLOW);
        assertFalse("User with null user type treated as non crew",
                    testee.canAccessReplicator(q));
    }
  
  	// Ommitted numerous similar tests

    @Test (expected = IllegalArgumentException.class)
    public void exceptionThrownIfTryToSetAlertStatusToNull() {
        try {
            testee.setAlertStatus(null);
            fail("Exception should have been thrown");
        } catch (IllegalArgumentException e) {
            assertEquals("Alert status cannot be set to null.", e.getMessage());
            throw e;
        }

    }
}

The first step to convert this test is to update the imports to the new classes.

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;

In this simple example most of the names are the same except @Before has become @BeforeEach. Another change is it is no longer necessary to make the class and test methods public so you may well be getting an IDE warning that they can be made package-private. The class declaration becomes simply:

class AccessControlJUnit5Test {

Now at this point you will be getting a few compilation errors. Firstly the tests which have a custom failure messages will not be compiling, this is because JUnit 5 has changed the order of the parameters. The message is now the last parameter, this feels a more natural arrangement to me. However particular care needs to be taken with this when using assertEquals on String objects as the method signature is unchanged so there will be no compilation failure but it will be asserting that the message and the expected value are equal and using the actual value as the failure message. Our updated test looks like this:

@Test
void userWithNullUserTypeCannotAccessReplicatorsWhenYellowAlert() {
  testee.setAlertStatus(Alert.YELLOW);
  assertFalse(testee.canAccessReplicator(q),
              "User with null user type treated as non crew");
}

The message argument can also now also be a lambda expression so if there is any complex logic required to produce the message it will be carried out only when the test fails.

The other compilation failure is on that ugly exception test. The @Test annotation no longer takes an expected argument. The Jupiter API provides a much tidier way of writing this type of test.

@Test
void exceptionThrownIfTryToSetAlertStatusToNull() {
  IllegalArgumentException actual = assertThrows(IllegalArgumentException.class,
                                                 () -> testee.setAlertStatus(null));
  assertEquals("Alert status cannot be set to null.", actual.getMessage());
}

This new assertThrows method takes advantage of the lambda expressions available in Java 8. You provide the type of exception you are expecting and then a lambda expression that calls the method you are expecting to throw the exception. The method also returns the exception if it is thrown and you can carry out any further assertions you wish.

This is the last change needed, all of the tests should now run and pass.

Most JUnit 4 tests can be migrated easily as described above. However a few of the methods have been renamed:

  • @Before and @After have become @BeforeEach and @AfterEach
  • @BeforeClass and @AfterClass have become @BeforeAll and @AfterAll
  • @Ignore has become @Disabled
  • @Category has become @Tag.

@RunWith , @Rule and @ClassRule have no direct replacement, they have been superseded by @ExtendWith and there is limited rule support in junit-jupiter-migrationsupport.

Setting up a new JUnit 5 project

If you are setting up a new project then the Maven set up is pretty similar to my previous example.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.19.1</version>
  <dependencies>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-surefire-provider</artifactId>
      <version>${junit.platform.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit.jupiter.version}</version>
    </dependency>
  </dependencies>
</plugin>

This is the same Surefire set up as before except the vintage engine is not required. And as you would expect you also just need the Jupiter API as a dependency and not JUnit 4.

<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter-api</artifactId>
  <version>${junit.jupiter.version}</version>
</dependency>

Test writing features

Now we have seen how to add JUnit 5 to a project and write a simple test we can delve more deeply into the features that are going to be relevant to the way you write your tests.

Display names and nesting

When a test fails it needs to be immediately clear what the scenario being tested was, what the expected behaviour was and what actually happened. Tying to name a test succinctly to express its purpose has often been challenging but display names and nesting adds greater clarity and flexibility. If we continue to look at the example of the the Enterprise’s access control system we ended up with a whole series of tests named along the lines of userOfTypeXCanAccessSystemYWhenAlertLevelIsZ but by using nesting and display names you get something like this:

@DisplayName("The USS Enterprise access control")
class EnterpriseAccessControlTest {

    private EnterpriseAccessControl testee;
    private User picard = new User("Jean-luc Picard", UserType.BRIDGE_CREW);
    private User barclay = new User("Reg Barclay", UserType.CREW);
    private User lwaxana = new User("Lwaxana Troi", UserType.NON_CREW);
    private User q = new User("Q", null);

    @BeforeEach
    void setUp() {
        testee = new EnterpriseAccessControl();
    }

    @Test
    @DisplayName("starts with no alert status")
    void startsWithNoAlert() {
        assertEquals(Alert.NONE, testee.getAlertStatus());
    }

    @Nested
    @DisplayName("when set to yellow alert")
    class whenYellowAlert {

        @BeforeEach
        void setUp() {
            testee.setAlertStatus(Alert.YELLOW);
        }

        @Test
        @DisplayName("only allows crew to access the replicators")
        void canAccessReplicators() {
            assertTrue(testee.canAccessReplicator(picard));
            assertTrue(testee.canAccessReplicator(barclay));
            assertFalse(testee.canAccessReplicator(lwaxana));
            assertFalse(testee.canAccessReplicator(q),
                    "User with null user type treated as non crew");
        }

        @Test
        @DisplayName("only allows crew to access the the transporter")
        void canAccessTransporters() {
            assertTrue(testee.canAccessTransporter(picard));
            assertTrue(testee.canAccessTransporter(barclay));
            assertFalse(testee.canAccessTransporter(lwaxana));
            assertFalse(testee.canAccessReplicator(q),
                    "User with null user type treated as non crew");
        }

        @Test
        @DisplayName("only allows bridge crew to access the the phasers")
        void canAccessPhasers() {
            assertTrue(testee.canAccessTransporter(picard));
            assertFalse(testee.canAccessTransporter(barclay));
            assertFalse(testee.canAccessTransporter(lwaxana));
            assertFalse(testee.canAccessReplicator(q),
                    "User with null user type treated as non crew");
        }
    }
  
    // Further nested classes and tests ommitted
}

The nested classes allow you to logically break up the tests add set up for specific subsets of the tests and the display name allows you to add a more human readable description in a pattern that is reminiscent of more behaviour driven testing frameworks such as Jasmine for JavaScript. This produces very clear output in IntelliJ

IntelliJ nested test results

However the test output on the command line is less helpful. The Surefire plugin does not yet support the display names (see this feature request). So when a test fails you will get it reported as the method name e.g.

canAccessTransporters()  Time elapsed: 0 sec  <<< FAILURE!
org.opentest4j.AssertionFailedError:
        at com.scottlogic.tng.EnterpriseAccessControlTest$whenNoAlert.canAccessTransporters(EnterpriseAccessControlTest.java:51)

Hopefully this feature will be available in the future as I have found it extremely easy when refactoring to end up with method names and display names out of sync.

The display name annotation also allows for special characters and emoji to be included but I’m not going to be rushing to include these in my test names as Git Bash on Windows certainly can’t cope with them and I would be surprised if build servers can cope with them.

Grouped Assertions

A problem with JUnit 4 is it just exits the tests when the first assertion fails. This does not always give you enough information to diagnose the problem so you can end up debugging through the code or commenting out the assertion and running the test again. JUnit 5 now provides a way round this issue. If we continue to look at the Star Trek example we can update one of the tests:

@Test
@DisplayName("only allows bridge crew to access the the phasers")
void canAccessPhasers() {
  assertAll(
    () -> assertTrue(testee.canAccessPhasers(picard)),
    () -> assertFalse(testee.canAccessPhasers(barclay)),
    () -> assertFalse(testee.canAccessPhasers(lwaxana)),
    () -> assertFalse(testee.canAccessPhasers(q))
  );
}

This method will always run all the assertions and then report on all the failures together. So how does this behave when there some of these assertions fail? When running through Surefire we get the following output.

canAccessPhasers()  Time elapsed: 0 sec  <<< FAILURE!
org.opentest4j.MultipleFailuresError:
Multiple Failures (3 failures)
        <no message> in org.opentest4j.AssertionFailedError
        <no message> in org.opentest4j.AssertionFailedError
        <no message> in org.opentest4j.AssertionFailedError
        at com.scottlogic.tng.AccessControlGroupedAssertionTest$whenYellowAlert.canAccessPhasers(AccessControlGroupedAssertionTest.java:113)

Yeah that is not great is it? It gives us the line number of the grouped assertion and we know 3 assertions failed but no clue as to which ones. Running through IntelliJ is a bit more helpful. It gives 4 stack traces, one for each of the failed assertions and one for the grouped assertion, so you can find the actual failures. We can make life easier by including messages with the assertions.

@Test
@DisplayName("only allows bridge crew to access the the phasers")
void canAccessPhasers() {
  assertAll(
    () -> assertTrue(testee.canAccessPhasers(picard),
                     "Bridge crew should be able to access phasers"),
    () -> assertFalse(testee.canAccessPhasers(barclay),
                      "Crew should not be able to access phasers"),
    () -> assertFalse(testee.canAccessPhasers(lwaxana),
                      "Non-crew should not be able to access phasers"),
    () -> assertFalse(testee.canAccessTransporter(q),
                      "User with null type should not be able to access phasers")
  );
}

Which gives much more helpful output.

canAccessPhasers()  Time elapsed: 0 sec  <<< FAILURE!
org.opentest4j.MultipleFailuresError:
Multiple Failures (3 failures)
        Crew should not be able to access phasers
        Non-crew should not be able to access phasers
        User with null type should not be able to access phasers
        at com.scottlogic.tng.AccessControlGroupedAssertionTest$whenYellowAlert.canAccessPhasers(AccessControlGroupedAssertionTest.java:113)

There are occasions where when an assertion fails some subsequent assertions make sense to be carried out but others do not so you can mix and match and nest grouped assertions. Continuing our Star Trek theme If we imagine a test for the system which allocates quarters to people on the ship it might look something like this:

@Test
@DisplayName("allocates quarters if the user has none.")
void allocatesQuarters() {
  testee.allocateQuarters(lwaxana);
  assertAll(
    () -> {
      Quarters assignedQuarters = lwaxana.getQuarters();
      assertNotNull(assignedQuarters, "Quarters should have been assigned to the user.");
      assertAll(
        () -> assertEquals(ROOM1_DECK, assignedQuarters.getDeck(),
                           "Assigned quarters has incorrect deck"),
        () -> assertEquals(ROOM1_NUMBER, assignedQuarters.getRoomNumber(),
                           "Assigned quarters has incorrect number")
      );
    },
    () -> {
      List<Quarters> unallocatedQuarters = testee.getUnallocatedQuarters();
      assertEquals(1, unallocatedQuarters.size(),"Incorrect unallocated quarters remaining");
      Quarters unassignedQuarters = unallocatedQuarters.get(0);
      assertAll(
        () -> assertEquals(ROOM2_DECK, unassignedQuarters.getDeck(),
                           "Unassigned quarters has incorrect deck"),
        () -> assertEquals(ROOM2_NUMBER, unassignedQuarters.getRoomNumber(),
                           "Unassigned quarters has incorrect number")
      );
    }
  );
}

I know what you are thinking, that is a horrible, unreadable test, I’ll come back to that in a moment but for now I will just run through what this test does. First it calls out to the system under test then it has a grouped assertion with 2 blocks in it, one that tests the user has been updated and one that tests the remaining rooms list has been updated. Within those blocks there are assertions that there is a Quarters object and then grouped assertions on the details. So let’s look at some failure scenarios.

allocatesQuarters()  Time elapsed: 0.016 sec  <<< FAILURE!
org.opentest4j.MultipleFailuresError:
Multiple Failures (2 failures)
        Quarters should have been assigned to the user. ==> expected: not <null>
        Incorrect unallocated quarters remaining ==> expected: <1> but was: <2>
        at com.scottlogic.tng.RoomManagerGroupedAssertionTest.allocatesQuarters(RoomManagerGroupedAssertionTest.java:33)

Here it is pretty clear that the method has not done anything. The user still has no quarters assigned and the unallocated quarters list has not been modified.

allocatesQuarters()  Time elapsed: 0 sec  <<< FAILURE!
org.opentest4j.MultipleFailuresError:
Multiple Failures (2 failures)
        Multiple Failures (2 failures)
        Assigned quarters has incorrect deck ==> expected: <15> but was: <12>
        Assigned quarters has incorrect number ==> expected: <234> but was: <4>
        Multiple Failures (2 failures)
        Unassigned quarters has incorrect deck ==> expected: <12> but was: <15>
        Unassigned quarters has incorrect number ==> expected: <4> but was: <234>
        at com.scottlogic.tng.RoomManagerGroupedAssertionTest.allocatesQuarters(RoomManagerGroupedAssertionTest.java:33)

And in this case it is again clear what has happened, the wrong room has been assigned. The problem here is that, as I mentioned earlier, the test code itself is pretty obtuse. I am in an odd situation here in that I am writing code to try and generate specific test code. However having written this it is clear that it could be improved. The code is in fact much clearer without the grouped assertions.

@Nested
@DisplayName("and the user has no quarters")
class UserHasNoQuarters {

  @BeforeEach
  void setUp() {
    testee.allocateQuarters(lwaxana);
  }

  @Test
  @DisplayName("updates the user to have quarters.")
  void allocatesQuartersToUser() {
    assertEquals(room1, lwaxana.getQuarters());
  }

  @Test
  @DisplayName("updates the unallocated quarters list.")
  void removesAllocatedQuarters() {
    List<Quarters> unallocatedQuarters = testee.getUnallocatedQuarters();
    assertEquals(1, unallocatedQuarters.size());
    assertEquals(room2, unallocatedQuarters.get(0));
  }
}

I would be wary of nesting grouped assertions. This is not to say you should not do it but it looks like something that can very easily become difficult to read.

Default test method on interfaces

With the addition of default method implementations to interfaces in Java 8 this has raised the possibility of writing tests once in an interface and then making multiple other tests implement that interface to run those tests. The most obvious scenario where this would make sense is where you have classes implementing an interface you are likely to have similar tests.

So far we have looked at a couple of systems on board the Enterprise now lets say these implement a common interface to allow level 1 diagnostics to be run, which returns the system name and the last user to access the system. Let’s see if we can pull those tests out into an interface.

public interface DiagnosticInfoTest<T extends DiagnosticInfo> {

    T getInstance();
    String getExpectedSystemName();
    void accessWithUser(User user);

    @Test
    @DisplayName("returns the system name.")
    default void returnsSystemName() {
        assertEquals(getExpectedSystemName(), getInstance().getSystemName());
    }

    @Test
    @DisplayName("returns the last user who accessed the system.")
    default void returnsLastUserToAccessSystem() {
        User riker = new User("William Riker", UserType.BRIDGE_CREW);
        accessWithUser(riker);
        assertEquals(riker, getInstance().getLastUserToAccessSystem());
    }
}

This is our interface with the tests for the methods on the interface. So our test class now needs to implement this interface and the required methods.

class RoomManagerInterfaceTest implements DiagnosticInfoTest<RoomManager> {

    @Override
    public RoomManager getInstance() {
        return testee;
    }

    @Override
    public String getExpectedSystemName() {
        return "Room Manager";
    }

    @Override
    public void accessWithUser(User user) {
        testee.allocateQuarters(user);
    }
  
  // Existing code ommited
}

Then similar changes are applied to the other test classes. When this test class is run the two tests in the interface are also run. For this scenario this is very effective. It avoids duplication and it makes it very easy to add comprehensive tests for future implementations of the interface.

Repeated and parameterised tests

JUnit 5 offers the ability to run a test repeatedly with the following annotation @RepeatedTest(10), the required argument specifies how many times the test will be executed. I has so far failed to come up with a scenario where running the exact same test repeatedly would yield any more useful information than running it once.

However part of the experimental API extends this further and allows a test to be run repeatedly with different arguments. To use this junit-jupiter-params needs to be added as a dependency. If we look back at the access control example we can again refactor this to use parameterised tests.

@ParameterizedTest(name = "User {1}, when Alert level is {2} should have access to transporters of {0}")
@MethodSource("transporterTestProvider")
void returnsCorrectAccessForTransporter(boolean expected, User user, Alert alertStatus) {
  testee.setAlertStatus(alertStatus);
  assertEquals(expected, testee.canAccessTransporter(user),
               () -> generateFailureMessage("transporter", expected, user, alertStatus));
}

private static Stream<Arguments> transporterTestProvider() {
  return Stream.of(
    Arguments.of(true,  picard,  Alert.NONE),
    Arguments.of(true,  barclay, Alert.NONE),
    Arguments.of(false, lwaxana, Alert.NONE),
    Arguments.of(false, q,       Alert.NONE),
    Arguments.of(true,  picard,  Alert.YELLOW),
    Arguments.of(true,  barclay, Alert.YELLOW),
    Arguments.of(false, lwaxana, Alert.YELLOW),
    Arguments.of(false, q,       Alert.YELLOW),
    Arguments.of(true,  picard,  Alert.RED),
    Arguments.of(false, barclay, Alert.RED),
    Arguments.of(false, lwaxana, Alert.RED),
    Arguments.of(false, q,       Alert.RED)
  );
}

private String generateFailureMessage(String system, boolean expected, User user, Alert alertStatus) {
  String message = user.getName() + " should";
  if (!expected) {
    message += " not";
  }
  message += " be able to access the " + system + " when alert status is " + alertStatus;
  return message;
}

A parameterised test needs the @ParameterizedTest annotation together with another annotation that provides the source for the parameters. In this case I have used @MehtodSource which takes the name of a static method which supplies the parameters. There are other source annotations which can provide parameters from an array of primitives, an enum class, an arguments provider class or a CSV file. The name of the test can be specified in the @ParameterizedTest annotation and can contain the current arguments by referencing their index. However this suffers from the same problem as @DisplayName, it is not yet supported by Surefire. You are also reliant on any object parameters having useful toString methods. As we have seen before you can include messages on the assertions to add clarity to your Surefire output.

returnsCorrectAccessForTransporter(boolean, User, Alert).returnsCorrectAccessForTransporter(boolean, User, Alert)
  Run 1: PASS
  Run 2: PASS
  Run 3: PASS
  Run 4: PASS
  Run 5: PASS
  Run 6: PASS
  Run 7: PASS
  Run 8: PASS
  Run 9: PASS
  Run 10: AccessControlParameterisedTest.returnsCorrectAccessForTransporter:31 Reg Barclay should not be able to access the transporter when alert status is RED ==> expected: <false> but was: <true>
  Run 11: PASS
  Run 12: PASS


The limitations of the name variable mean the output of the tests is not as clear as the version using the nested classes and display names. However there is a certain appeal to this format of test, it is relatively condensed and creating the provider method effectively creates a matrix of all the expectations which makes it really easy to see exactly what is covered by the tests. However something similar can be created by using helper methods instead of a parameterised test.

@Test
@DisplayName("grants correct access to transporters")
void correctAccessToTransporterIsGranted() {
  assertAll(
    () -> assertTransporterAccess(true,  picard,  Alert.NONE),
    () -> assertTransporterAccess(true,  barclay, Alert.NONE),
    () -> assertTransporterAccess(false, lwaxana, Alert.NONE),
    () -> assertTransporterAccess(false, q,       Alert.NONE),
    () -> assertTransporterAccess(true,  picard,  Alert.YELLOW),
    () -> assertTransporterAccess(true,  barclay, Alert.YELLOW),
    () -> assertTransporterAccess(false, lwaxana, Alert.YELLOW),
    () -> assertTransporterAccess(false, q,       Alert.YELLOW),
    () -> assertTransporterAccess(true,  picard,  Alert.RED),
    () -> assertTransporterAccess(false, barclay, Alert.RED),
    () -> assertTransporterAccess(false, lwaxana, Alert.RED),
    () -> assertTransporterAccess(false, q,       Alert.RED)
  );
}

private void assertTransporterAccess(boolean expected, User user, Alert alertStatus) {
  testee.setAlertStatus(alertStatus);
  assertEquals(expected, testee.canAccessTransporter(user),
               () -> generateFailureMessage("transporter", expected, user, alertStatus));
}

These two different versions are pretty similar in their readability and information provided when they fail. My personal preference would be to go for the latter one as I think it is slightly easier to follow and it does not involve using the experimental API.

Other new testing features

The features I have covered a just a select hand full of the new features. I have tried to pick out the ones that I think are going to be most useful in the widest number of scenarios. Only time will tell which features will actually turn out to be the most used in real world situations so here is a quick summary of some of the other features that may prove interesting.

  • assertions with timeouts.
  • tagging tests to enable running a filtered set of tests.
  • changing the test lifecycle to only create one test instance per class (rather than per test).
  • parameter resolvers which allow objects to be supplied to test methods.
  • dynamic tests that allow tests to be generated at run time rather than at compile time.
  • registering extensions add additional functionality to tests.

IDE and build support

So far I have been using an up-to-date version of IntelliJ Idea with a Maven build. The JUnit 5 support in IntelliJ is excellent. Maven is easy to set up and whilst the Surefire plugin does not perfect, with the inclusion of some meaningful messages on assertions it is fine.

My experience with Eclipse was much less pleasant. The eclipse wiki makes adding JUnit 5 support sound simple but after numerous tries and spending the best part of an afternoon on it I failed to get JUnit 5 working with either Eclipse Oxygen or Photon. I am optimistic that when Photon becomes the stable release the JUnit 5 support will be functional. Netbeans does not currently support JUnit 5 either see this feature request.

It is possible to write Jupiter tests that can be run with a JUnit 4 test runner but I personally don’t feel it is worth the effort of a two stage migration like this. I’d wait till I was able to use an IDE that supports JUnit 5 and then migrate to the JUnit 5 platform before adding tests that use the Jupiter API.

As regards build tools, as well as Maven there is a Gradle plugin. I converted my Maven build to a Gradle build using gradle init, this did not convert over the JUnit 5 configuration but it was pretty straight forward to add back in. The build.gradle file for the build that supports JUnit 4 and 5 looks like this:

description = 'junit4-and-5'
dependencies {
    compile group: 'org.slf4j', name: 'slf4j-api', version:'1.7.5'
    compile group: 'org.slf4j', name: 'slf4j-simple', version:'1.7.5'
    testCompile group: 'junit', name: 'junit', version:'4.12'
    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-api', version:'5.0.0'
    testRuntime group: 'org.junit.jupiter', name: 'junit-jupiter-engine', version:'5.0.0'
    testRuntime group: 'org.junit.vintage', name: 'junit-vintage-engine', version:'4.12.0'
}

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.0'
    }
}

apply plugin: 'org.junit.platform.gradle.plugin'

junitPlatform {
    platformVersion '1.0.0'
}

And the JUnit 5 only project was pretty similar except the junit and junit-vintage-engine dependencies are not required. Much to my delight once my project was converted over I found the display names work with the Gradle build producing nicer output when test fail compared to Maven and Surefire. e.g.

JUnit Jupiter:The USS Enterprise access control:when set to red alert:only allows bridge crew to access the the phasers
    MethodSource [className = 'com.scottlogic.tng.AccessControlGroupedAssertionTest$whenRedAlert', methodName = 'canAccessPhasers', methodParameterTypes = '']
    => org.opentest4j.MultipleFailuresError: Multiple Failures (1 failure)
        Crew should not be able to access phasers

At the moment there is not any Ant support, see this feature request.

Conclusion

So do I think it is worth moving to JUnit 5? Yes, I think the new features provide great opportunity to write some clear and easy to maintain tests. Also how easy it is to maintain JUnit 4 tests along side JUnit 5 tests means the migration can be very easy and gradual. You can leave your existing tests alone and just convert tests where you need the new features.

Do I think it is worth moving to JUnit 5 now? Maybe. Much as I like the new features of JUnit 5 you need to be able to use them with your build and IDE. If you have a Maven or Gradle build and your whole team is happy to use an up to date version of IntelliJ I’d say yes now is the time to start using JUnit 5. If you are stuck using an Ant build or a different IDE I think it would be prudent to wait for a bit.

MORE BY JAMES

blog comments powered by Disqus