DerbyJunitTestConfiguration

Test Configurations

This is about test configurations in a pure JUnit setup (the eventual goal).

1. Test Configurations
   1. Goals
   2. Primary Configurations
      1. All test fixtures running as embedded and client
      2. All fixtures running as embedded and client but some based upon JDBC level
      3. All test fixtures running only as embedded
      4. All test fixtures running only as client
   3. Suites
   4. System Properties
   5. Folders
   6. Default Configuration (embedded)
   7. Derby Client Configuration

Goals

• Ideally running the Derby tests should allow multiple configurations to exist concurrently, for example to parallelize test runs or stress test by kicking off multiple runs within the same JVM.

• The default way for running tests must not require any system property or other setup that would make it harder for tests to run elsewhere as standard JUnit tests.

Primary Configurations

Derby's JUnit tests will be self-contained for the primary configurations which are:

• embedded

• derby client

This means that each test's suite() method will return a suite that runs test fixtures in all the primary configurations as required. This is option B in this derby-dev discussion: http://mail-archives.apache.org/mod_mbox/db-derby-dev/200609.mbox/%3c451C1CDF.7050706@apache.org%3e

Tests can fall into any of the following categories:

• embedded & client - runs fixtures in both configurations, some individual fixtures with a test class may only run in a single configuration due to items not supported in a configuration, bugs in a configuration or no value in running the fixture in that configuration. Typically tests that are aimed at testing JDBC methods or objects (e.g. functional test packages jdbcapi and jdbc4) will run in both configurations so that both drivers are tested.

• embedded only - Typically SQL language tests (e.g. function test package lang) can run only in embedded since they are testing execution of SQL statements within the embedded engine. For example testing of server side routines, DDL, triggers or GRANT/REVOKE would not gain additional test coverage by running with the client driver. SQL language tests that relate to data types most likely will benefit from running with the client as well to ensure testing of transmission of the datatype across the DRDA protocol. Test of the storage sub-system (store package) most likely will not need to run with the client configuration.

• client only - E.g. specific testing of a client data source.

Whenever test fixtures are added to a suite using the reference to the class (e.g. suite.addSuite(MyTest.class)) then the order of execution of the fixtures is not defined and may vary across different platforms. Thus this requires that the fixtures be independent of each other, which is a good practice to follow. Remember that the setUp() and tearDown() methods will be run for each fixture. If a test needs ordering among fixtures then adding the fixtures explicitly will preserve the order.

    // adds all fixtures with no defined order of execution
    suite.addSuite(MyTest.class);

    // again, no order defined as MyTest.class is used
    return TestConfiguration.defaultSuite(MyTest.class);

    // adds fixtures that will executed in order testA first, then testB, and last testC
    suite.add(new MyOrderedTest("testA"));
    suite.add(new MyOrderedTest("testB"));
    suite.add(new MyOrderedTest("testC"));

Here are examples of how to write the suite() method for the test class MyTest for various combinations. Any tests that are added into a suite without any client server decorator will run the fixtures only as embedded.

It's also useful to look at actual test classes to see how various other situations are handled.

All test fixtures running as embedded and client

Note this default suite setup includes a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.defaultSuite(MyTest.class);
   3 }

All fixtures running as embedded and client but some based upon JDBC level

Note manually added CleanDatabaseTestSetup.

   1 public static Test suite() {
   2     TestSuite suite = new TestSuite();
   3 
   4     // run as embedded
   5     suite.addTest(baseSuite());
   6 
   7     // run as client server
   8     suite.addTest(TestConfiguration.clientServerDecorator(baseSuite()));
   9 
  10     return new CleanDatabaseTestSetup(suite);
  11 }
  12 private static Test baseSuite() {
  13     TestSuite suite = new TestSuite(MyTest.class)
  14     if (JDBC.vmSupportsJDBC3()) {
  15           suite.addTest(new MyTestCase("jdbc3SomeTestCase"));
  16     }
  17     return suite;
  18 }

All test fixtures running only as embedded

Note this setup does not include a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.embeddedSuite(MyTest.class);
   3 }

All test fixtures running only as client

Note this setup does not include a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.clientServerSuite(MyTest.class);
   3 }