Replacing functionality of the Derby test harness

Goals

Conversion or writing JUnit tests FAQ

SQL tests

Harness based script tests (.sql scripts) can be converted in three ways:

Decorators

A number of useful decorators and convenience methods that return decorators exist, this section shows which to use for certain situations. Remember that a decorator can wrap a set of test fixtures:

  return new CleanDatabaseTestSetup(suite);

or a single test fixture

 suite.addTest(new CleanDatabaseTestSetup(new MyJDBCTest("testJDBCStuff")));

Decorators can also be nested, some thought may be required to ensure the nesting results in the correct execution order:

  return new CleanDatabaseTestSetup(
             DatabasePropertyTestSetup.setLockTimeouts(suite, 2, 4));

Question

Answer

How to set system properties for a test?

Setting properties on a database level is preferred as setting properties at a system level may interfere with other tests that could be run in parallel. However the SystemPropertyTestSetup decorator does exist to allow setting system properties, passing in the set of properties to set.

How to set database properties for a test?

Use the DatabasePropertyTestSetup decorator passing in the set of properties to set.

How to reduce the lock timeout values for a test?

Use the conveniencedecorator returned by DatabasePropertyTestSetup.setLockTimeouts or DatabasePropertyTestSetup directly.

How to set BUILTIN authentication with a set of users

Use the convenience decorator returned by DatabasePropertyTestSetup.builtinAuthentication, use a unique password token

How to run as a different user?

Use the decorator returned by TestConfiguration.changeUserDecorator

How to run tests with Derby's client driver?

Any test class's suite() method should return a suite that runs fixtures in embedded and client/server mode as required. TestConfiguration.defaultSuite() takes a Class object and returns a suite that runs all the test methods starting with test in both embedded and client server. When more control is needed over wich tests are run in emebdded and client/server TestConfiguration provides these methods: clientServerDecorator, clientServerSuite & embeddedSuite. In addition the NetworkServerTestSetup decorator can be used directly.

Since the database is re-used when running tests how do I ensure the test starts out with a clean database?

Use the CleanDatabaseTestSetup decorator.

How to execute DDL at setUp that is dropped automatically at tearDown?

Provide a class that extends CleanDatabaseTestSetup and implement the decorateSQL method. This can be achieved as an inner class, search for references to CleanDatabaseTestSetup for examples.

How to run a test in SQL authorization mode?

Use the utility method TestConfiguration.sqlAuthorizationDecorator. The simple DatabasePropertyTestSetup cannot be used directly since setting the SQL authorization mode cannot be undone. The returned decorator changes the default database to a different one that has SQL authorization mode set. This database, like the default database, is re-used by each call to TestConfiguration.sqlAuthorizationDecorator.

Harness features

The test decorator pattern, a class extending TestSetup or the Derby specific BaseJDBCTestSetup seems the natural way to replace most of the functionality of the test harness.

Folder Policy

It would be good to have some up-front policy about how folders are used, to provide consistency and help with good security manager testing. See:

Old Harness Suite status

Progress on removing tests from the old suites, with the assumption they are converted to JUnit tests.

Suite

Initial Test Count

Remaining Test Count (2007/02/15)

2013/09/26

Comments

demo

2

2

0

see DERBY-2903 and DERBY-3294

derbylang

148

91 (~70% .sql tests)

37 (34 .sql)

LangJunitTestConversion

derbynetclientmats

8

6

4 ( 1 junit running in old harness)

ClientJunitTestConversion (derbynet/* only)

derbynetmats

45

42

0

ClientJunitTestConversion (derbynet/* only)

derbytools

15

15

2

encodingTests

4

4

1

encryption

3

3

2

encryptionAES

2

2

0

encryptionAll

8

8

5

DERBY-1001, DERBY-3711. For an encryption Decorator (among other things), see DERBY-1952.

encryptionBlowfish

3

3

1

encryptionCFB

3

3

1

encryptionDES

3

3

1

encryptionECB

3

3

1

encryptionOFB

3

3

1

i18nTest

7

7

0 (removed)

j9derbynetmats

30

0(ignore)

old suite for running on a specific jvm

jdbc20

10

5

0 (removed)

JdbcapiJunitTestConversion

jdbc40

8

4

0(removed)

jdbcapi

22

15

0 (removed)

JdbcapiJunitTestConversion

jdbcxa40

2

0 (removed)

jdk14

9

7

0 (removed)

JdbcapiJunitTestConversion

largeDataTests

1

1

0 (removed)

Not in derbyall

multi

1

1

0 (removed)

DERBY-1764

nist

125

0 (removed)

DERBY-2007

propertyinfo

1

1

1

simpledemo

1

1

1

storemats

6

6

1

storemore

38

38

16

storerecovery

13

13

9

storetests

8

8

5

DERBY-6709

storeunit

34

34

34

unit

4

4

4

upgrade

1

1

0 (removed)

DERBY-2217

xa

11

11

5

JdbcapiJunitTestConversion

xmlTests

2

0 (removed)

DERBY-1758

Total

344 test runs with 306 unique tests

134 test runs

Test Conversions

ConvertLobTestsToJunit

ConvertSqlScriptTestsToJunit

KillDerbyTestHarness (last edited 2014-08-21 17:14:05 by MyrnavanLunteren)