Differences between revisions 14 and 15
Revision 14 as of 2007-12-20 18:00:25
Size: 9087
Editor: DanDebrunner
Comment:
Revision 15 as of 2009-09-20 22:11:57
Size: 9087
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
[[TableOfContents]] <<TableOfContents>>

Test Configurations

This is about test configurations in a pure JUnit setup (the eventual goal).

Goals

  • Ideally running the Derby tests should allow multiple configurations to exist concurrently, for example to parallelize test runs or stress test by kicking off multiple runs within the same JVM.
  • The default way for running tests must not require any system property or other setup that would make it harder for tests to run elsewhere as standard JUnit tests.

Primary Configurations

Derby's JUnit tests will be self-contained for the primary configurations which are:

  • embedded
  • derby client

This means that each test's suite() method will return a suite that runs test fixtures in all the primary configurations as required. This is option B in this derby-dev discussion: http://mail-archives.apache.org/mod_mbox/db-derby-dev/200609.mbox/%3c451C1CDF.7050706@apache.org%3e

Tests can fall into any of the following categories:

  • embedded & client - runs fixtures in both configurations, some individual fixtures with a test class may only run in a single configuration due to items not supported in a configuration, bugs in a configuration or no value in running the fixture in that configuration. Typically tests that are aimed at testing JDBC methods or objects (e.g. functional test packages jdbcapi and jdbc4) will run in both configurations so that both drivers are tested.

  • embedded only - Typically SQL language tests (e.g. function test package lang) can run only in embedded since they are testing execution of SQL statements within the embedded engine. For example testing of server side routines, DDL, triggers or GRANT/REVOKE would not gain additional test coverage by running with the client driver. SQL language tests that relate to data types most likely will benefit from running with the client as well to ensure testing of transmission of the datatype across the DRDA protocol. Test of the storage sub-system (store package) most likely will not need to run with the client configuration.

  • client only - E.g. specific testing of a client data source.

<!> Whenever test fixtures are added to a suite using the reference to the class (e.g. suite.addSuite(MyTest.class)) then the order of execution of the fixtures is not defined and may vary across different platforms. Thus this requires that the fixtures be independent of each other, which is a good practice to follow. Remember that the setUp() and tearDown() methods will be run for each fixture. If a test needs ordering among fixtures then adding the fixtures explicitly will preserve the order.

    // adds all fixtures with no defined order of execution
    suite.addSuite(MyTest.class);

    // again, no order defined as MyTest.class is used
    return TestConfiguration.defaultSuite(MyTest.class);

    // adds fixtures that will executed in order testA first, then testB, and last testC
    suite.add(new MyOrderedTest("testA"));
    suite.add(new MyOrderedTest("testB"));
    suite.add(new MyOrderedTest("testC"));

Here are examples of how to write the suite() method for the test class MyTest for various combinations. Any tests that are added into a suite without any client server decorator will run the fixtures only as embedded.

It's also useful to look at actual test classes to see how various other situations are handled.

All test fixtures running as embedded and client

<!> Note this default suite setup includes a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.defaultSuite(MyTest.class);
   3 }

All fixtures running as embedded and client but some based upon JDBC level

<!> Note manually added CleanDatabaseTestSetup.

   1 public static Test suite() {
   2     TestSuite suite = new TestSuite();
   3 
   4     // run as embedded
   5     suite.addTest(baseSuite());
   6 
   7     // run as client server
   8     suite.addTest(TestConfiguration.clientServerDecorator(baseSuite()));
   9 
  10     return new CleanDatabaseTestSetup(suite);
  11 }
  12 private static Test baseSuite() {
  13     TestSuite suite = new TestSuite(MyTest.class)
  14     if (JDBC.vmSupportsJDBC3()) {
  15           suite.addTest(new MyTestCase("jdbc3SomeTestCase"));
  16     }
  17     return suite;
  18 }

All test fixtures running only as embedded

<!> Note this setup does not include a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.embeddedSuite(MyTest.class);
   3 }

All test fixtures running only as client

<!> Note this setup does not include a CleanDatabaseTestSetup.

   1 public static Test suite() {
   2    return TestConfiguration.clientServerSuite(MyTest.class);
   3 }

Suites

Top level suites have two roles:

  • To allow a set of tests to be run, e.g. all JDBC api tests (tests.jdbc._Suite), all SQL language tests (tests.lang._Suite), all tests (suites.All)

  • To run a number of tests (or suites) in a secondary configuration, e.g. all language tests with encryption.

Top level suites should not contain magic that requires the use of a top-level suite to run an individual test.

System Properties

Derby's JUnit infrastructure classes to run tests may setup various system properties. Tests that set specific system properties are not listed here.

System Property

Class

Value

Description

Related Method(s)

java.security.policy

SecurityManagerSetup

<NONE>

No security manager installed

BaseTestCase.assertSecurityManager

URL to policy file (default)

Security manager installed

derbyTesting.codeclasses

SecurityManagerSetup

Path to classes folder

Used by testing policy files for permissions granted to all classes, only set when loading from classes and a security manager is installed

derbyTesting.testjar

SecurityManagerSetup

Path to derbyTesting.jar

Used by testing policy files for permissions granted to tsting classes, only set when loading from jar files and a security manager is installed. Not used by policy files yet

derbyTesting.codejar

SecurityManagerSetup

Path to derby.jar & derbynet.jar

Used by testing policy files for permissions granted to network server and engine classes, only set when loading from jar files, jars are present and a security manager is installed. Currently used for all jar files to match existing harness

derbyTesting.clientjar

SecurityManagerSetup

Path to derbyclient.jar

Used by testing policy files for permissions granted to client classes, only set when loading from jar files, client jar is present and a security manager is installed. Not used by policy files yet

derby.system.home

TestConfiguration

${user.dir}/system (default)

not set

Databases created in ${user.dir}/database

Folders

The tests only use folders below ${user.dir}, thus all paths in the table are relative to ${user.dir}

Folder

Description

system

Location of derby.system.home when set.

system/singleUse

Location for single use databases when derby.system.home is set. Single use databases are for tests that leave the database in such a state that it cannot be re-used. Database will have a unqiue name within the folder.

databases

Location of databases when derby.system.home not set. This will allow clear policy files so that user code does not have permission to write into the database folders.

databases/singleUse

Location for single use databases when derby.system.home is not set.

logs

Location of any log files generated by the tests

extin, extout, extinout

Location of input files, output files or files used for both input and output for tests.

fail

Top level folder to store failure information

fail/embedded

Failures for tests with embedded configuration

fail/client

Failures for tests with Derby client configuration

fail/db2client

Failures for tests with DB2 client configuration

Default Configuration (embedded)

The default configuration is:

  • Embedded driver
  • derby.system.home=${user.dir}/system

  • database name wombat

  • user APP

In general tests should not rely on the user name or the database name, this will allow tests to run in multiple configurations without changes.

Derby Client Configuration

Setup by decorator created by TestConfiguration.clientServerDecorator(Test) or TestConfiguration.clientServerSuite(Class) static methods. Inherits configuration setup from current configuration but:

  • Uses client driver
  • Starts network server in its setUp method, server output in ${user.dir}/logs/serverConsoleOutput.log in append mode.

  • Stops network server in its tearDown method.

DerbyJunitTestConfiguration (last edited 2009-09-20 22:11:57 by localhost)