Handling Test Regressions

It is crucial that our regression test suite, derbyall, run with no failures. This page describes mechanisms and guidelines to help ensure that this is the case.

Awareness

It's important for Derby developers to be aware of the state of the regression suite. For this reason, any time an automated regression test fails, an email will be sent to derby-dev indicating that there were test failures and pointing to which tests failed.

We are also creating a separate mailing list called derby-test-results that posts the results of each regression test run, regardless of whether it passed or failed. A developer can subscribe to this if they want to keep a constant pulse of test results. Of one particular use is knowing whether your recent checking caused tests to hang or fail to build so that test results don't even get sent out.

Automated regression test results with lots of detail about the failure and showing the changes that were applied for each revision can be obtained at http://www.multinet.no/~solberg/public/Apache/index.html.

Managing Test Regressions

What follows are guidelines for how test regressions should be handled

TestRegressions (last edited 2009-09-20 22:11:38 by localhost)