CDC Testing

This page summarizes the status of Derby testing on J2ME/CDC/Foundation Profile using JSR169 (referred as CDC in rest of the document).

Goal

The goal is to have all tests that can be run in CDC to be enabled. There will be a subset of tests which cannot be run in CDC because of platform/JSR169 limitations. A part of the goal is to identify and list such tests with valid reason for their exclusion.

Plan

To achieve this goal:

* Exclude tests which fail in CDC. This is to ensure a clean run of derbyall in CDC. Specify the reason for exclude, if known. Otherwise, indicate that this needs further investigation.

* For tests that can be run (at least partially) in CDC, modify the test/master files or split the test into two parts - for CDC and non-CDC environments. Some test failures could be possible bugs. Open JIRA issues for those.

* For tests that cannot be run, list a valid reason.

Status

Current CDC Test Coverage

The numbers are based on tests run with IBM's WCTME5.7 j9 foundation jvm. Numbers may be approximate.

Table of tests run in CDC by suite

* Total - Number of tests run/Total number of tests in the suite
* Skipped - Number of tests which are skipped using excludes
* To be enabled - Number of tests which have to be enabled to run in CDC
* Cannot be run - Number of tests which need to be skipped because of platform/JSR169 limitations.

Test Suite

Total

Skipped

To be enabled

Cannot be run

Comments

derbylang

135/150

15

0

15

Listed in table below

propertyinfo

0/1

1

0

1

lang/GetPropertyInfoTest.java test uses Driver.getPropertyInfo method

storeall

90/97

7

0

7

Listed in table below

storeunit

33/34

1

0

1

unit/T_RawStoreFactory.unit

unit

3/4

1

1

unit/cacheService.unit hangs

jdbcapi

19/23

4

1

3

jdbcapi/checkDataSource30.java could be enabled

jdbc20

9/10

1

0

1

jdbcapi/getCurConnJdbc20.sql

jdk14

6/10

4

0

4

Listed in table below

nist

126/126

0

0

derbytools

7/14

7

3

4

3 tests can be possibly re-written

i18nTest

5/7

2

0

2

Listed in table below

Total

90.9% (433/476)

9.1%

1.1%

8%

Goal: To be enabled=0%. Do one of the following for each test which is skipped.
* Enable the tests to run in CDC.
* Add to list that cannot be run specifying reason.

Test suites which cannot be currently run in CDC

The following table lists the suite that cannot be currently run in CDC.

Test Suite

Comments

derbynetclientmats

Network Server not supported in CDC

derbynetmats

Network Server not supported in CDC

derbynetautostart

Network Server not supported in CDC

xa

XA not supported in CDC

encryptionAll

encryption not supported in CDC

encryption

encryption not supported in CDC

jdbc40

JSR169 is based on JDBC 3.0

encodingTests

Tests run only with jdk1.5

upgrade

Tests require jdk1.4 or higher

demo

demo uses Driver/DriverManager classes

simpledemo

uses Driver/DriverManager classes

Tests which cannot be currently run in CDC

The following table lists tests which cannot be run in CDC. These must be tests that exclusively test features that are not supported by CDC/JSR169.

Test Suite

Test Name

Reason

jdbc20

jdbcapi/getCurConnJdbc20.sql

Tests jdbc20 statement using server-side JDBC. JSR169 is based on JDBC3.0

jdbcapi

jdbcapi/bestrowidentifier.sql

Tests call to getBestRowIdentifier method using server-side JDBC

jdbcapi

jdbcapi/secureUsers1.sql

Test uses ldap which is not supported in j9_foundation

jdbcapi

jdbcapi/checkDriver.java

Test java.sql.Driver class

jdk14

jdbcapi/checkDataSource30.java

Can try to include this test

jdk14

jdbcapi/parameterMetaDataJdbc30.java

Tests ParameterMetaData - not available in JSR169

jdk14

lang/procedureJdbc30.java

Tests procedures with server-side JDBC

jdk14

jdbcapi/savepointJdbc30_XA.java

savepointJdbc30 was split into two. Tests which can be run with JSR169 are in savepointJdbc30_JSR169

derbylang

lang/authorize.sql

test uses server-side JDBC - exclude for now

derbylang

lang/connect.sql

test uses ij 'driver' command - exclude for now

derbylang

lang/errorCode.sql

test uses ij 'driver' command - exclude for now

derbylang

lang/GetPropertyInfoTest.java

test uses Driver.getPropertyInfo method

derbylang

lang/nestedCommit.sql

test uses procedure with server-side JDBC

derbylang

lang/unaryArithmeticDynamicParameter.java

uses jdk14 jdbc metadata calls

derbylang

lang/dcl.sql

uses server-side JDBC - exclude for now

derbylang

lang/declareGlobalTempTableJavaJDBC30

test requires javax.naming.Referenceable

derbylang

lang/functions.sql

test uses server-side JDBC - exclude for now - can be written as Java test

derbylang

lang/holdCursorIJ.sql

Holdability can not be changed using jdbc3.0 apis in JDKs lower than JDK1.4.

derbylang

lang/releaseCompileLocks.sql

uses server-side JDBC

derbylang

lang/procedure.java

uses server-side JDBC

derbylang

lang/cast.sql

test requires java.math.BigDecimal

derbylang

lang/triggerRefClause.sql

test requires java.math.BigDecimal

derbylang

lang/triggerGeneral.sql

test requires java.sql.DriverManager

derbytools

tools/ij4.sql

ij test with no default driver or database

derbytools

tools/ij.sql

test is specific to ij and tests bad/good urls

derbytools

tools/ij5.sql

test uses ij.connection.<connection name>

derbytools

tools/ijConnName.sql

test uses ij.connection.<connection name>

storemore

store/Beetle6038.java

encryption not supported in Derby J2ME implementation

storemore

store/OnlineBackupTest3.java

test uses procedure with server-side JDBC

storemore

store/BackupPathTests.java

test uses procedure with server-side JDBC

storerecovery

store/LogChecksumSetup.java

uses derby.subSubProtocol in URL

store

store/LogChecksumRecovery.java

uses derby.subSubProtocol in URL

store

store/LogChecksumRecovery1.java

uses derby.subSubProtocol in URL

storeunit

unit/T_RawStoreFactory.unit

DERBY-654

i18nTest

i18n/messageLocale.sql

test requires java.sql.Driver

i18nTest

i18n/caseI_tr_TR.sql

test requires java.sql.Driver

The following table lists tests which can possibly be enabled to run with CDC.

Test Suite

Test Name

Reason

derbylang

lang/authorize.sql

test uses server-side JDBC - exclude for now

derbylang

lang/connect.sql

test uses ij 'driver' command - exclude for now

derbylang

lang/errorCode.sql

test uses ij 'driver' command - exclude for now

jdk14

jdbcapi/checkDataSource30.java

This is based on checkDataSource test which is enabled

derbylang

lang/dcl.sql

uses server-side JDBC - exclude for now

derbylang

lang/functions.sql

test uses server-side JDBC - exclude for now - can be written as Java test

derbytools

tools/dblook_test.sql

Possible rewrite using DataSource class

derbytools

tools/URLCheck.sql

reason for exclusion not specfied

derbytools

tools/ij6.sql

reason for exclusion not specfied

jdbcapi

jdbcapi/UpdateXXXTest.junit

reason for exclusion not specfied

JIRA issues

The following JIRA issues have been opened for this effort.

Jira Entry

Comments

Status

DERBY-398

Exclude tests which cannot be run in CDC

Fixed

DERBY-447

Found by this effort

Fixed

DERBY-452

Found by this effort

Fixed

DERBY-453

Create a test wrapper to run tests using BigDecimal

Fixed

DERBY-470

Found by this effort

Fixed

DERBY-542

Doc task for DERBY-470

Fixed

DERBY-518

Found by this effort

Fixed

DERBY-597

Opened to enable more tests which use ij

Fixed

DERBY-648

Make show connections statement work in CDC

Fixed

DERBY-680

Found by this effort

Not started

DERBY-993

test junitTest/LangSuite cannot be run with J2ME

Fixed

Progress Table

The following table lists the svn revisions for fixes checked in for this effort:

Trunk Revision

10.1 Revision

Pass %

Comment

77.2%

2005/10/25
328246

-

80.75%

Enhance ij connect statement to work in J2ME/CDC/FP

2005/11/03
330547

-

81.22%

Make show connections statement work in CDC

2005/11/08
331885

-

81.22%

Make show connections statement work in CDC

2005/11/11
332436

-

82.43%

Enable tests in jdbcapi and derbylang suite

2005/11/23
348593

-

87.35%

Bring store tests into J2ME/CDC Foundation testing

CdcTesting (last edited 2009-09-20 22:12:01 by localhost)