Derby Public Interfaces & Forward Compatibility

WORK IN PROGESS

/!\ Initial set of public interfaces from an e-mail discussion on the derby-dev list. The list is subject to change. /!\

Goal

The goal is to allow any application written against the public interfaces of an older version of Derby to run, without any changes, against a newer version of Derby.

Detailed information on Derby's versioning scheme.

Exceptions

There will be occasions where the Derby community does introduce regressions or what might be considered regressions by some. Examples of such exceptions include fixing incorrect implementations of a standard or a correctness bug (for example a query returning the wrong results). Such changes may break existing applications. We will strive to avoid such changes but they may occur. Any such change should be a deliberate choice by the community, not by accident. If at all possible such changes should be deferred to minor release boundaries.

Specific examples:

We will mark any fixes or product changes that intentionally change product behaviour and might affect existing applications by checking the Existing Application Impact checkbox. Also the "Release Note Needed" box will be checked as well and user impact described in ReleaseNoteFormat.

Users of Derby have several options under the hopefully rare occasion that a new release breaks their application:

What is an Interface?

An interface is any "interaction point" of the software, something that customers may touch and may therefore depend upon. Each interface is assigned a specific stability level which communicates the guarantee, or the contract, to how and when these interfaces will change in a non-compatible way. Here are some examples of interfaces:

Stability Levels

To help describe the types of stability guarantees Derby provides for a given interface, we identify the following stability levels. These definitions are inspired by the OpenSolaris interface taxonomy

Stable

This is an interface which we expect others to depend upon to remain stable enough for them to build their own products or systems on top of. Most of our public interfaces should be given the stability level of Stable. Stable interfaces should never change between minor releases, and even between major releases changes should be extremely rare.

Note that an interface must be documented before it can be classified Stable or Unstable. Any undocumented interface can only be given a Private or Private Stable stability level. New features can not be considered Stable or Unstable until they are documented.

It is important to note that having a major release does not mean that the release will have incompatible changes to a Stable interface. It is just that a major release is the only time where an incompatibility can be introduced on a Stable interface

Any changes to Stable interfaces should be documented in the release notes.

Standard

These are interfaces that we make available which are produced by a standards body, or which is an industry standard that we feel is unlikely to change in incompatible ways.

Any changes to Standard interfaces should be documented in the release notes.

Unstable

This is usually for new functionality that we feel is experimental or unstable. Documentation should make it very clear that this interface may change between minor releases. Changes to unstable interfaces should be documented in the release notes.

Note that an interface must be documented before it can be classified Stable or Unstable. Any undocumented interface can only be given a Private or Private Stable stability level. New features can not be considered Stable or Unstable until they are documented.

Private

Private interfaces are interfaces which are not documented for external users, and generally users should not be depending upon, and which can change at any time.

Private Stable

Some private interfaces require stability even though they are not documented in the user documentation and not supported for direct use by the users of Derby, in order for us to guarantee compatibility across releases. For instance, the on-disk format for Derby databases must remain stable across minor releases and in general changes to this format should be rare, because of the impact on upgrade.

Deprecated

An interface can be downgraded from some other classification (such as Stable) to Deprecated to encourage users to move to a newer interface that replaces this interface. An interface marked as deprecated must continue to be supported until the next major release. An interface can be reclassified as deprecated at any minor release boundary, but requires consensus approval within the Derby community. The interface must be clearly documented as deprecated in the documentation and/or release notes, and if possible warnings should be issued when an interface is used that it is deprecated (with the ability to turn these warnings off).

Interfaces that have been reclassified to Deprecated should be documented in the release notes.

A deprecated interface may be removed from the project after four minor and/or major releases.

What is an Incompatible Change?

What constitutes an incompatible change depends upon the type of interface

Changes to Java Classes

We are going to follow the definitions for what types of changes are considered acceptable for what types of releases as defined by the Apache Jakarta Commons project. See

http://jakarta.apache.org/commons/releases/versioning.html

Changes to Stored Procedures

System procedures and metadata procedures are both Stable interfaces, so it is important to understand what entails a compatible change for stored procedures.

In general, if you need to make a change for new functionality that is incompatible, it is recommended that you create a new stored procedure and have the newer version of the code use that new stored procedure.

Changes to Database Tables

Documented system tables are a Stable interface, so care must be taken when considering changing these

Changes to the system tables cannot occur for soft upgrade. Associated upgrade code and tests will be added for compatible changes.

Incompatible Changes for Data File Formats

I need help with this one

Changes to Configuration Properties

Changes to Command Line Interfaces (CLI)

These are both scripts and Java classes that implement a command-line interface

A CLI is subdvided into multiple interfaces: its name, its arguments, its return code, stdin, stdout, environment variables it uses, its error messages. Each of these may have a different stability level. I describe an incompatible change for each of these

Changes to a Network Protocol

DRDA is the current protocol we use, and is the underpinning of client/server compatibility

Generally the network protocol needs to be versioned and the client and server need to negotiate to a given version level. The types of changes described below are changes to an existing version of the protocol.

Compatible Change

Incompatible Change

Other Miscellaneous Interfaces

Here we describe what an incompatible change means for other miscellaneous interfaces

message. Can add new messages.

Interface Table

This table describes each of the interfaces Derby provides, and the stability guarantee we provide for that interface. This is the interface table for the 10.1 release. This is subject to change (in a compatible way, meaning new interfaces may be added or old interfaces may be upgraded to a more stable level) at each minor release boundary.

NOTE: This table is still a draft form and is subject to change

NOTE: Any interface that is in the public documentation is considered a Stable interface unless explicitly called out as otherwise in the table below

Interface Name

Stability Level

Specification Reference (if available)

Comments

SQL language support

Standard

SQL92 and SQL99

See (SQL table) for complete listing

JDBC support

Standard

[need reference here]

Embedded published/public api classes (defined by the published api javadoc target)

Stable

http://db.apache.org/derby/docs/10.1/publishedapi/

System tables (catalogs) as documented

Stable

Note that adding columns to existing system tables, or add new allowable values for type columns etc., is considered a compatible change. JDBC DatabaseMetaData is the preferred way to obtain information about database.

DRDA

Standard

[need ref]

Jar file names

Stable

Class path requirements

Stable

Jar file manifest entires that define external behavior

Stable

e.g. OSGi bundling now and auto-loading of JDBC driver in the future

derby.log file format

Unstable

On-disk database file format

Private Stable

http://db.apache.org/derby/papers/versionupgrade.html

On-disk transaction log file format

Private?

Not sure if this is correct -- can this change incompatibly across point or minor releases? http://db.apache.org/derby/papers/versionupgrade.html

Metadata stored procedures

Private Stable

These may be used by earlier clients to implement JDBC DatabaseMetadata calls

SQL States defined by SQL specification

Standard

There may be reasons to change a SQL State at a minor release boundary; for example,if the wrong SQL State is being used. These will be addressed on a case-by-case by the community.

Non-standard Derby SQL States

Unstable

Error message text

Private

Note that this means canon-based tests can not make claims that a regression has occurred because the output from error messages has changed

Install directory hierarchy

Unstable

Tools (ij, dblook, etc.) command-line name and arguments

Stable

Tools class names

Stable

Tools stdout

Private

Tools stderr

Private

Tools error codes

Unstable

ij properties

Stable

Environment variables used by Derby

Unstable

Default port number used by Derby network server

Stable

JDBC URL format and driver properties for Derby JDBC drivers

Stable

Documented derby properties

Stable

Undocumented derby properties

Private

Network server properties

Stable

Defaults returned by DatabaseMetadata methods

Stable

Documented configuration defaults

Stable

Console output format for tools and network server

Private

Debug trace outputs

Private

System stored procedures

Stable

Query plan output as displayed by RUNTIMESTATISTICS

Private

There are example outputs in the documentation but the full format of RUNTIMESTATISTICS is not document and therefore this should be considered a private interface. Do we need a Private Unstable classification for this?

JVM Support and Version Interoperability

FAQ

Would this set of rules drive the decision of when to declare a major release versus a minor release?

They would be one of the deciding factors, likely a major deciding factor. Other factors will likely be involved. It is more than likely that we may want to declare a new major release even though we have not introduced any incompatibilities.

We currently don't have any rules about when we would change to 11.0 from the current 10.x pattern.

Making an incompatible change to a Stable or Standard interface would require us to change to 11.0.

Would it be because someone changed some "Standard" interface in an incompatible way that we move to 11.0?

If a standards body makes an incompatible change to a standard interface that we rely on, and we make the choice to upgrade to the new, incompatible revision of that standard, then, yes, we should move to a new major release. A standards body making an incompatible change to a standard should be fairly rare, however.

If no-one made such a change, would be be having 10.14 in a number of years?

Yes, possibly, unless the community has other reasons to upgrade the version.

ForwardCompatibility (last edited 2009-09-20 22:11:44 by localhost)