DerbySnapshotOrRelease

Table of Contents

  1. Releases
    1. High level time line
    2. Release process steps details
      1. Before a candidate can be generated
        1. Prepare the community for a new release
        2. Work the toward a release candidate
        3. Environment requirements for the release manager
        4. Steps to prepare your machine/code check-out for the release process
      2. For the first release candidate on a new branch
      3. For each release candidate
        1. Check-ins just before generating release artifacts
        2. Build the release artifacts
        3. During the vote
        4. After an unsuccessful vote
      4. After a successful vote
  2. Snapshots
  3. Making a (private) Quick Build of Release Artifacts

Releases

High level time line

Once before release:

For each release candidate:

Once after candidate passes vote:

Release process steps details

Before a candidate can be generated

Prepare the community for a new release

  1. Announce your intention/desire to have a release on the list

    • As new features are added and bugs fixed, it is likely that someone will announce their intention to produce a release (if they are a committer) and volunteer to be the release manager. It may also be the case that a non-committer will ask when the next official release will occur. This should be a sign to committers that there is demand for a release. :)

  2. Volunteer as release manager (or try to enlist one)

    • Since only committers have the necessary access to Apache resources to shepherd a release to its completion, a committer must volunteer to be the release manager. Usually (hopefully) someone will volunteer if it is clear there is demand for a release. A release manager is under no obligation to finish once they volunteer, and another committer can pick up and complete their work, or even produce a competing release if so desired.

  3. Create wiki pages for the release. (Unless you have already done so. It may be convenient to have a wiki page prior to feature freeze and branch cutting). Use pages for a previous release as a template. You're likely to create a general page and a page to hold the testing information.

  4. For major releases, create a new branch for the release, in both the code and docs trees. 

    • svn copy -r {rev} https://svn.apache.org/repos/asf/db/derby/code/trunk/ https://svn.apache.org/repos/asf/db/derby/code/branches/{branchname}/ -m "Created the {branchname} source branch" 
svn copy -r {rev} https://svn.apache.org/repos/asf/db/derby/docs/trunk/ https://svn.apache.org/repos/asf/db/derby/docs/branches/{branchname}/ -m "Created the {branchname} doc branch"

  5. After creating the branch, bump the version number on trunk. There is not currently an ant target for bumping the version number on the trunk so you need to edit tools/ant/properties/release.properties on trunk by hand. Bump the major/minor properties as appropriate, zero out the maint property (if it isn't zero already), and ensure the beta flag is set to true (if it isn't already). E.g. after bumping from 10.4 to 10.5 release.properties should look similar to: 

    • #Wed Jul 19 08:21:42 PDT 2006 
drdamaint=0
maint=0000000
major=10
minor=5
eversion=10.5
beta=true
copyright.comment=Copyright 1997, 2008 The Apache Software Foundation or its licensors, as applicable.
vendor=The Apache Software Foundation

      (!) It is probably a good idea to update to the head of trunk, and do a clean build (ant clobber; ant all) and run some tests before committing the change.

  6. On the new branch, edit tools/ant/properties/release.properties and update the value of the maint property. The exact value of depends on the type of release (feature, update or snapshot). For a new feature release the value will typically be 

    • maint=1000000

      This will set the version string for the branch to 

      <major>.<minor>.1.0 (beta)

      The relationship between the maint property and the version number reported by sysinfo is given by the following equation: maint=100000*3rd+4th, (or maint=1000000*fixpack+point as described in the [WWW] Derby Versioning Scheme. Note that a "fixpack" of 0 overrides the beta property in tools/ant/properties/release.properties and tags the version string with "alpha"). See also the description of how to spin a release candidate.

      (!) It is probably a good idea do a clean build (ant clobber; ant all) and run some or all tests before committing this change to the branch.

      <!> For releases off branches older than 10.4, you need to first run ant regex (in tools/release) to automatically update version references in a number of master files. These changed master files also need to be checked in.

  7. Add the new branch number to the list of branches on the source page of the website.

    • {i} For instructions on how to build the website using Forrest, please see: [WWW] http://db.apache.org/derby/papers/derby_web.html

      The actual page to modify is src/documentation/content/xdocs/dev/derby_source.xml. For a minor (bug fix) release, consider bumping the third version of the source tree it will come off (likely a branch).

Work the toward a release candidate

  1. Arrange for the new version(s) in JIRA.

    • Unless you are a Jira-admin you need to post to derby-dev requesting that new versions be added to JIRA. For any release you need a new version for the (beta) release candidate. When creating a new branch you also need a new development version for trunk, (unless this has been done earlier for some reason).

  2. Target the bugs you feel should be fixed in JIRA

    • All features and bug fixes should be tracked in JIRA: [WWW] http://issues.apache.org/jira/browse/DERBY

      Mark the Fix In field for the JIRA entries for the items you want to be in the release with the proper version.

      (!) Also, it's a good idea to post to derby-dev to get an idea of what features or fixes other contributors would like in the release.

  3. Fix bugs and update STATUS (and CHANGES.html/RELEASE_NOTES.html as needed)

    • Get to work! Add features, fix bugs, and update STATUS as you go. The wiki is nice, as are personal webpages, but STATUS is the designated place for Apache projects to keep their current status. Apache members and committers expect to be able to grab the STATUS file from the code tree to determine the current status of the project.

      (!) It's a nice thing to keep the STATUS file on branches up to date with the current status of the branch.

      {i} Derby branches up to 10.2 included a file CHANGES for that purpose; 10.3 branch and trunk have RELEASE_NOTES.html checked in and CHANGES.html which is generated for the release. The process of generating these files is described at the ReleaseNoteProcess wiki page.

  4. Drive the bug list to zero.

    • As the list of remaining bugs in JIRA approaches zero, be sure to mark their status properly in JIRA. Mark blocker and critical bugs as such so that others can see the status at a glance. Move non-showstopper bugs out to future releases if it appears they will not be fixed for this release.

  5. Drive the creation of release notes.

    • The release note generator expects files called ReleaseNote.html for each item marked that is:

      • resolved to 'Fixed'

      • fixed in the release under study but not in the previous release

      • marked with 'Existing Application Impact' or 'Release Note Needed'.

      {i} More info can be found at ReleaseNoteProcess.

  6. Check that all creative works have ASF license headers.

  7. Check that the year and other information is correct.

    • Ensure that the year in NOTICE is correct.

      <!> Ensure that all versions and copyright details in the docs tree are correct, this includes the top level conrefs.dita, as well as lower level dita files.

Environment requirements for the release manager

To be able to produce the release artifacts, you need to

and after the release has been voted on:

This means you have to have to be able to do the following / use the following tools ( {i} see: BUILDING.txt):

And you have to have the following files available ( {i} see: BUILDING.txt):

You need to at least have the doc tree and source tree available, and your ant.properties file needs to include: 

j14lib=<location of jdk14(2)/jre/lib> 
jdk16=<location of jdk16>
jsr169compile.classpath=<location of jsr169 implementation>
junit=<location of junit.jar>
build.compiler=<modern, or see BUILDING.txt for other options>
proceed=true
relnotes.src.reports=<location where you want to save/access the xml scripts for generating release notes>
#sane=<sane should *not* be set>

{i} Special consideration for non-linux users:

Steps to prepare your machine/code check-out for the release process

  1. Get a clean copy of the source tree for your version

    • A clean check-out is best.

  2. Obtain files for building all optional pieces:

    • Consult BUILDING.txt and make sure you have all relevant pieces not in the source tree:

    • jdk14

    • jdk15 (after 10.3.*)

    • jdk16

    • junit.jar

  1. Copy tools/ant/properties/packaging.tmpl to tools/ant/properties/packaging.properties and modify as necessary.

    • The build will attempt to use a file called tools/ant/properties/packaging.properties to carry out checksum (md5)and signing (pgp) tasks.

      {X} Because this differs per operating system, you have to create this file based on the template (tools/ant/properties/packaging.tmpl) which is set for a likely linux environment.

      {i} Most Unix distributions come with either md5 or md5sum. An md5sum utility for Windows can be downloaded as a part of the GnuWin32 port of the core Gnu utilities, from: [WWW] http://gnuwin32.sourceforge.net/packages/coreutils.htm. A standalone md5 utility can be found at [WWW] http://www.fourmilab.ch/md5/. Note, for the executable available from this last location, the correct option for output is md5 -n.

  2. Make sure your public PGP/GPG key is in the KEYS file, and preferably after it has been signed.

    • {i} For information about PGP and why it is used to sign release binaries at Apache, please read [WWW] http://people.apache.org/~henkp/trust/.

      You should create a PGP key for yourself if you do not have one, and upload it to at least one, preferably two, of the main public keyservers, e.g. pgp.mit.edu. You will need this key to sign the release binaries. Your key should be tied into the Apache web of trust, which means you should have at least one person sign your key, and you should have done gpg --refresh-keys to get that person's signature before you follow the steps at the top of the KEYS file.

      {i} GPG is available for a variety of platforms from [WWW] http://gnupg.org. PGP is a commercial product which is available from [WWW] http://pgp.com.

      Your KEY needs to be in the KEYS file in trunk, KEYS file on the branch if you've created one, and the KEYS file in /www/www.apache.org/dist/db/derby at people.apache.org.

      {X} You need to make sure the tools/ant/properties/packaging.properties file has correct information for your pgp tool.

  3. Ensure you have appropriate jdks available

    • {i} See: BUILDING.txt for information on JSR169 and OSGI support.

      For 10.2 forward, make sure that you have JDK 1.6 available and that you've set the jdk16 property to the JAVA_HOME for your JDK 1.6 installation in your ant.properties so that the JDBC 4.0 classes are built and the JDBC 4.0 javadoc is created.

      For versions after 10.3, you need to also have jdk1.5 available

  4. Check out a clean copy of the doc tree

    • Once you have a copy of the documentation on your local machine, you will need to update the property ${docs.out} in tools/ant/properties/packaging.properties to point to your local copy of the documentation.

      To build the documentation, you need to obtain DITA-OT1.1.2.1_bin-ASL.zip and place it in the docs' tree lib.

      {i} See: [WWW] http://db.apache.org/derby/manuals/dita.html for more info about building the documentation.

  5. (Optional:) Have eclipse installed or make arrangements with someone willing to build the eclipse ui plugin. As release manager you need to have access to the plugin zips if you don't build them yourself (e.g. the volunteer can attach the plugin zips to a Jira issue). <!> If you are not building the plugin yourself, make sure that the beta status of the plugin matches that of the release candidate you are building.

For the first release candidate on a new branch

  1. Update tools/ant/properties/release.properties by hand, and set the beta property to false (unless this is a beta release candidate).

  2. Check in the modification.

  3. You will need to clobber and build again before you can see the changed beta property reflected in the source. Note that the first release off a new branch is automatically beta, even if you set the beta property to false.

For each release candidate

Verify that:

The third and fourth parts of the version number are combined into a single property, maint, where maint = (third digit * 1000000) + fourth digit. Note that removing the beta flag will not have an effect unless the 3rd digit (fixpack) is greater than 0, since version numbers with fixpack=0 always are considered alpha. Fixpack (3rd digit) will normally be set to 1 when the branch is cut, but if it isn't, it must be incremented before the release candidate can be created.

Check-ins just before generating release artifacts

  1. [WWW] Adjust version numbers in documentation. Make sure the copyright notices has the correct years.

  2. Generate RELEASE_NOTES.html in the branch and check it into the svn repository. Please consult the instructions for [WWW] generating release notes.

  3. For versions < 10.3; finalize CHANGES. For later releases the CHANGES file has been superseded by CHANGES.html This can include items in addition to the release notes, which are purely for user purposes.

    • {i} The tool java/tools/ChangesFileGenerator.java can help you generate this file. It can be invoked by running ant genchanges in tools/release.

  4. Check in the latest SQLState documentation. (This step can be done in advance, but make sure that no SQLState has been modified before creating the release).

    • Build the source tree. The SQLStates are documented in the Reference Guide on the following page: rrefexcept71493.dita. This file is generated by the Derby build and placed in classes/doc. Take the version of this file generated by building the code branch and check it into the doc branch at src/ref/rrefexcept71493.dita.

  5. Sync the source and doc repository.

    • 'svn up' in your subversion view to bring all files in your view up to the latest revision. Otherwise, the version output by svn which is captured for the build number will be a range (e.g. 290275:320938).

Build the release artifacts

  1. Check you have all required pieces:

    • - doc tree

      • - with DITA library - with latest SQLState

      - KEYS checked in

      - RELEASE_NOTES.html and CHANGES (only for version < 10.3) checked in. <!> CHANGES.html (version 10.3 and later) should not be checked in.

      - md5 & pgp and docs info set correctly in tools/ant/properties/packaging.properties and available (PATH)

      - ant.properties set correctly for: jdk15, jdk16, jsr169

      - sane not set in ant.properties

      - non-source files for building source available: junit.jar, felix.jar, (with 10.3 and earlier: osgi.jar), dita library (again).

  2. Build the documentation.

    • The documentation needs to be included in the -bin distribution and src, so you will need to access the doc branch when running the ant release target. The doc build is not controlled by the source tree build, and thus needs to be completed beforehand.

      {i} Information on building the docs is located at [WWW] http://db.apache.org/derby/manuals/dita.html.

  3. Create the distributions for release by running: 

    • svn up 
ant clobber
rm -rf jars javadoc snapshot  # really clean
rm tools/release/*.zip tools/release/*.tar.gz  # really,
rm tools/release/*.md5 tools/release/*.asc     # really clean
ant sane ; ant all ; ant buildjars   # for lib-debug
ant clobber
ant insane
ant -Dsane=false snapshot
ant publishedapi
cd tools/release
ant release
ant sign

      You will need to enter your PGP passphrase several times as the release distributions are signed.

      (!) You can save yourself some typing by using gpg's --passphrase-fd <fd> option. This instructs gpg to read the passphrase from the specified file descriptor.

      This will create the following files in your tools/release directory:

      • db-derby-[version]-bin.tar.gz

      • db-derby-[version]-bin.zip

      • db-derby-[version]-lib.tar.gz

      • db-derby-[version]-lib.zip

      • db-derby-[version]-lib-debug.tar.gz

      • db-derby-[version]-lib-debug.zip

      • db-derby-[version]-src.tar.gz

      • db-derby-[version]-src.zip

      The Eclipse core plugin is generated in the snapshot directory at the top level by the snapshot target. You should also create the Eclipse UI plugins (see plugins/eclipse/readme.txt, except use the core plugin created in the snapshot directory), but this requires Eclipse. If you don't want to do it yourself, those interested in the Eclipse plugins will likely volunteer to generate them for you.

      <!> Note that ui-plugin can be built as a beta. If you don't build the ui-plugin yourself, make sure that its beta status matches that of the release candidate you are building.

      You should also create checksums and signatures for these files with: 

      gpg --armor --detach-sign derby_ui_plugin_[version].zip 
gpg --armor --detach-sign derby_core_plugin_[version].zip
md5 -q derby_ui_plugin_[version].zip > derby_ui_plugin_[version].zip.md5
md5 -q derby_core_plugin_[version].zip > derby_core_plugin_[version].zip.md5

      {X} There is a problem with the ant sign target on Cygwin that may occur elsewhere. If for some reason the ant sign target hangs, it may be prompting and waiting for input that you cannot see.

      (!) In that case, you can also use this simple script to automate signing the release archives: 

      #!/bin/sh signone() { (-)
  gpg --detach-sign --armor $1
  md5 -q $1 > $1.md5
}

signone $1-bin.tar.gz
signone $1-bin.zip
signone $1-lib.tar.gz
signone $1-lib.zip
signone $1-lib-debug.tar.gz
signone $1-lib-debug.zip
signone $1-src.tar.gz
signone $1-src.zip

      Invoking this 'sign.sh db-derby-10.1.1.0' would sign all of the release archives for Derby 10.1.1.0, for example.

      {X} Be sure to replace the commands for gpg and md5 with the correct commands for your system. Note that on cygwin, the md5 switch is "-n" rather than "-q".

  4. Verify the signatures and checksums.

    • As an example, the Derby 10.1 archives would be verified with GPG as follows: 

      gpg --verify derby_ui_plugin_1.1.0.zip.asc derby_ui_plugin_1.1.0.zip 
gpg --verify derby_core_plugin_10.1.1.zip.asc derby_core_plugin_10.1.1.zip
gpg --verify db-derby-10.1.1.0-src.zip.asc db-derby-10.1.1.0-src.zip
gpg --verify db-derby-10.1.1.0-src.tar.gz.asc db-derby-10.1.1.0-src.tar.gz
gpg --verify db-derby-10.1.1.0-lib.zip.asc db-derby-10.1.1.0-lib.zip
gpg --verify db-derby-10.1.1.0-lib.tar.gz.asc db-derby-10.1.1.0-lib.tar.gz
gpg --verify db-derby-10.1.1.0-lib-debug.zip.asc db-derby-10.1.1.0-lib-debug.zip
gpg --verify db-derby-10.1.1.0-lib-debug.tar.gz.asc db-derby-10.1.1.0-lib-debug.tar.gz
gpg --verify db-derby-10.1.1.0-bin.zip.asc db-derby-10.1.1.0-bin.zip
gpg --verify db-derby-10.1.1.0-bin.tar.gz.asc db-derby-10.1.1.0-bin.tar.gz

      The md5 checksums can be verified by generating them via another method. For example, using openssl: 

      openssl md5 < db-derby-10.1.1.0-src.zip

      And comparing the output of openssl to the output from ant in db-derby-10.1.1.0-src.zip.md5

  5. Keep the jars/insane/*.jar files available. You will need them for maven deployment after the vote is complete.

    • (!) It is advisable to copy the jars to another directory and include the version number of the release candidate in the directory name. This way you avoid having to extract the jars from the release artifacts if you inadvertently overwrite the jars (through a rebuild) before deploying to the maven repository.

  6. Bump the fourth digit of the source in preparation for a possible next build: 

    • cd tools/release 
ant bumplastdigit

      Commit the changed version number.

  7. Post the distributions

    1. Make sure the umask of your Apache account grants both read and write access to group (db) and only read access to others. {i} Read access for others is necessary for web access, and the write permission for the db group will make life easier for future release managers.

    2. Create a new directory for the release candidate in your people.apache.org:public_html directory.

    3. Upload all files from the tools/release directory into this directory.

    4. Upload snapshot/derby_core_plugin_x.y.z.rev.* into this directory.

    5. (Optional:) Upload derby_ui_plugin_a.b.c.* into this directory.

      • {i} If someone has volunteered to build the Eclipse plugin for you, you will have to defer this step until the volunteer has had a chance to download the other release artifacts which she will need to build the plugin.

        <!> Remember to sign the derby_ui_plugin.x.y.z.zip before you upload it!

    6. Post to derby-dev so that others can begin testing. (!) It is convenient to create a link to the release candidate directory from the release wiki page.

  8. Vote on the distributions

    • Call for a vote on derby-dev to have the distributions posted on your public_html accepted for the release.

      {i} It is not always easy to decipher all the voting rules which govern the acceptance/rejection of a release candidate. First there are the [WWW] Apache rules for releases, the [WWW] DB decision guidelines, and finally the [WWW] DB PMC Bylaws.

      The existing practice is: A vote needs to be running for at least 7 days, so, give at least that much time before closing the vote to give adequate time for others to inspect and test the distributions. If no-one has responded after a week, prod gently until you get a response. A majority of votes, and at least one binding +1 vote are required for acceptance.

      Forward or bcc a copy of the call for vote to [MAILTO] private@db.apache.org so the DB PMC is aware that a vote is in progress. Also forward the results post to [MAILTO] private@db.apache.org. ( <!> Note: do not cc the PMC; bcc or forward a copy of the post.)

During the vote

  1. Address items on the ReleaseVettingChecklist

After an unsuccessful vote

  1. If vote does not pass and go back to targeting bugs in Jira.

    • If the vote does not pass and additional release candidates need to be made, then presumably it won't have passed because of a showstopper-type bug or similar issue, so you need to go back to the bug-fixing section.

After a successful vote

  1. If the vote passes, put distributions onto mirrors

    • {i} First, read Apache's information about mirroring: [WWW] http://www.apache.org/dev/mirrors.html and [WWW] http://www.apache.org/dev/mirror-step-by-step.html

      Copy the distribution archives and their signatures/checksums to a new directory underneath /www/www.apache.org/dist/db/derby on people.apache.org.

      If the new release is the most current release, link the -current links to the new files. Here's a quick-and-dirty shell script for doing so: 

      ln -s $1/$1-bin.tar.gz db-derby-current-bin.tar.gz 
ln -s $1/$1-bin.tar.gz.asc db-derby-current-bin.tar.gz.asc
ln -s $1/$1-bin.tar.gz.md5 db-derby-current-bin.tar.gz.md5
ln -s $1/$1-bin.zip db-derby-current-bin.zip
ln -s $1/$1-bin.zip.asc db-derby-current-bin.zip.asc
ln -s $1/$1-bin.zip.md5 db-derby-current-bin.zip.md5
ln -s $1/$1-src.tar.gz db-derby-current-src.tar.gz
ln -s $1/$1-src.tar.gz.asc db-derby-current-src.tar.gz.asc
ln -s $1/$1-src.tar.gz.md5 db-derby-current-src.tar.gz.md5
ln -s $1/$1-src.zip db-derby-current-src.zip
ln -s $1/$1-src.zip.asc db-derby-current-src.zip.asc
ln -s $1/$1-src.zip.md5 db-derby-current-src.zip.md5

      which, if you called the script linkcurrent.sh and put it in your home directory, could be run as follows: 

      cd /www/www.apache.org/dist/db/derby 
~/linkcurrent.sh db-derby-10.1.1.0

      using db-derby-10.1.1.0 as the release to link to -current as an example.

  2. Copy the latest version of the documentation to the doc directory on the website.

    • The documentation is included in the release you uploaded, but unfortunately its directory structure is not quite right for the website. To get around this you need to unzip the db-derby-x.y.z.w-bin.zip archive, rename some directories and copy the pdf files into the html hierarchy. Here is a shell script outlining the process: 

      mkdir ~/x.y
cd ~/x.y
unzip $1
mv db*bin/docs .
mv db*bin/javadoc ./publishedapi
rm -r db*bin

for i in adminguide devguide getstart ref tools tuning
do
  mv docs/pdf/$i/*.pdf docs/html/$i
  rmdir docs/pdf/$i
done

mv docs/html/* .
rmdir docs/html
rmdir docs/pdf/pt_BR
rmdir docs/pdf
rmdir docs

      Pass in the location of the db-derby-x.y.z.w-bin.zip. Move the x.y directory that gets created in your home directory over to /www/db.apache.org/derby/docs.

      (!) It is probably easier (especially if you have a high-latency connection to people.apache.org), to create the x.y directory on the same machine where you built the release, but then you need to upload the documention directory. A recursive copy using scp is going to take a long time, so it is better to create an archive and copy that: 

      zip -r x.y.zip x.y 
scp x.y.zip you@people.apache.org:/www/db.apache.org/derby/docs/
ssh you@people.apache.org
cd /www/db.apache.org/derby/docs/
unzip x.y.zip
rm x.y.zip

      <!> Make sure that all files and directories have permissions that makes them accessible on the web! Note that you will not be able to verify this in a web browser until the new directories and files have been rsynced to real webserver.

  3. Create a page for the release, build/update site

    1. Create an HTML file for the release.

      • (!) It is a good idea to use the previous release pages as templates, filling in the changed details as necessary. 

        cd src/documentation/content/xdocs/releases 
cp release-x'.y'.z'.w'.html release-x.y.z.w.html
vi release-x.y.z.w.html

        <!> Note the following things when creating the release page:

        • Be sure to specify that the src.tar.gz requires gnu tar to unravel (because of our usage of ant tar to create this, using longfile=gnu for handling long filenames).

        • Use search-and-replace to change all occurences of the old version number to the new one.

        • Replace the old release notes section with the appropriate fragment from RELEASE_NOTES.html in the new release.

        • Remove all square brackets [] from the release notes page. When the cgi machinery sees square brackets, it truncates the release page. (Should be handled by the ReleaseNote vetter?)

    2. In order for the release HTML file to be pulled into the build, it is necessary to add a line to the <uris> section of src/documentation/conf/cli.xconf. Near line 310 of that file, add: <uri type="append" src="releases/release-x.y.z.w.html"/> (with the correct version for your release).

    3. Add a mirroring CGI script for the release. The CGI file should have the same name as the HTML file, but obviously a .cgi extension.

      • (!) The easiest thing is probably just to copy the CGI script for an existing release.

      • Forrest will not copy the release CGI script over unless you make a link to it from another page. Add the link to derby_downloads.html, as described below, before building.

      • Make sure that the CGI script is made executable by setting svn:executable on it 

        cd src/documentation/content/xdocs/releases  
cp release-x'.y'.z'.w'.cgi release-x.y.z.w.cgi
svn add release-x.y.z.w.cgi      
svn propset svn:executable ON release-x.y.z.w.cgi

        <!> Be sure to set svn:executable also for the generated copy which forrest builds into build/site/releases.

    4. Remove mirroring for older releases which no longer require it.

      • Svn delete the CGI script. <!> Make sure you do this both in src/documentation/content/xdocs/releases and build/site/releases.

      • Edit the release page to remove the CGI boilerplate and convert the distribution links from mirror references to ordinary links. This means replacing the [preferred] tags with the URL of the archive, i.e. http://archive.apache.org/dist/.

        (!) To see what the release page should look like, look at the release page for an even older release. 

        cd build/site/releases 
svn delete release-x'.y'.z'.w'.cgi
cd -
cd src/documentation/content/xdocs/releases
svn delete release-x'.y'.z'.w'.cgi
vi release-x'.y'.z'.w'.html

    5. Update src/documentation/content/xdocs/derby_downloads.xml:

      • Add a link to the CGI script for the new release at the top of the releases section.

      • Move the previous release down to the other old releases.

      • Change the link from the CGI script to the HTML file for all releases for which you disabled mirroring (in the previous step).

    6. Run 'forrest site' at the top of the site tree.

      • {i} [WWW] Forrest instructions

        {X} Due to [WWW] FOR-480, the release page will be built into your $FORREST_HOME/main/site directory. You will need to copy it to the build directory.

        {X} The build process frequently will create "false modifications" in build/tmp or build/site/skin. These must be reverted before checking in the changes. Subversion may report some files as changed which should be static. Revert anything in build/site/skin or build/site/papers before committing your website changes ([WWW] see the explanation).

        <!> Remember to svn add the new files and set svn:executable on the new CGI script in build/site/releases after running forrest! 

        rm -rf $FORREST_HOME/main/site/*  # Remove any files from a previous build 
forrest site
cp -r $FORREST_HOME/main/site build/site
svn revert -R build/tmp build/site/skin
svn add build/site/releases/release-x.y.z.w.html
svn add build/site/releases/release-x.y.z.w.cgi
svn propset svn:executable ON build/site/releases/release-x.y.z.w.cgi

    7. Check the changes. If they look good, 'svn commit' them.

    8. Update the website on people.apache.org: 

      • ssh you@people.apache.org 
cd /www/db.apache.org/derby
svn up

        <!> Note that people.apache.org is rsync'd to the actual website every hour or so. Verify that the new release appears on the [WWW] download page. Also check you still can download any release for which you disabled mirroring. Finally check that you can access the new documentation at the [WWW] documentation page.

        <!> Also, when you run 'svn up' on people.apache.org, make sure that the new .cgi file is executable and otherwise has the correct permissions!

    9. Once you have committed your changes and updated the website on people, you can review your changes by following the instructions at [WWW] http://www.apache.org/dev/project-site.html

    10. If you removed mirroring for an existing release (by removing the CGI script and changing the links), you should now delete those releases from the mirror directory as it is important to keep the mirror directory tidy.

      • <!> Don't delete them until you have verified that the updated web page with new links to the archive directory actually works!

        {i} It is no longer necessary to manually copy releases to the archive. Anything placed in the mirror directory (/www/www.apache.org/dist/db/derby) will automatically be copied to the archive directory (/www/archive.apache.org/dist/db/derby).

  4. Deploy to Maven repository.

    1. If you do not already have the latest Maven 1 distribution, [WWW] download it, unpack it, and put the bin directory into your path so that you can run maven commands.

      • {i} As of this writing, the latest 1.x version of Maven was 1.1.

    2. cd into Derby's maven directory. Typically this will be in the same sandbox that you used to build the release, but this is not strictly required.

    3. Edit project.xml so that it contains the correct version number for this release between the <currentversion> tags.

    4. Edit project.properties:

      • Add username, and password for your account on people.apache.org so you can properly authenticate and copy the files to people. The scpexe protocol should work without problems, especially if you have an ssh public key already on people.

      • Make sure derby.jars points to the directory holding the release jars. jars/insane is the default, so if you backed up the jars somewhere else when you built the release you must adjust this property accordingly. You also need modify this property if you are using a sandbox other than the one you used to build the release.

      • Uncomment #maven.repo.list=apache. It commented out by default to prevent accidental deployments.

        Your local diff should look something like: 

        sanity=insane 
-derby.jars=../../jars/${sanity}
+derby.jars=/path/to/maven/jars
 
-#maven.repo.list=apache
+maven.repo.list=apache
 maven.repo.apache=scpexe://people.apache.org
 maven.repo.apache.directory=/www/people.apache.org/repo/m1-ibiblio-rsync-repository
-maven.repo.apache.username=
-maven.repo.apache.password=
+maven.repo.apache.username=you
+maven.repo.apache.password=secret
 maven.repo.apache.group=db

    5. Run the Maven commands.

      • Run maven to attain the multiproject:install goal to install the artifacts into your local maven repo.

        • {X} Note: For 10.3.2.1 I had to type maven multiproject:install to get this to work.

      • Run maven clean to attain the multiproject:clean goal to clean up the maven tree.

      • Run maven multiproject:deploy to copy all the artifacts into the apachecvs repository.

        <!> The multiproject:deploy command will not work unelss you uncomment the maven.repo.list property in project.properties.

        {i} This does not build using maven, it works by copying the jars pointed to by ${derby.jars} (jars/${sanity} by default).

    6. Sign the individual jars.

      • Unfortunately, Maven renames the jars during deployment, so the jars now found in the Maven repository have the version number embedded in the file name. E.g derby.jar has become derby-x.y.z.w.jar. The name of the signature file needs to same as the name of jar with the asc extension, but Maven apparently knows nothing about signatures, so we have to create correctly named signatures by hand. The following script signs and renames the jars appropriately, and should be run in the ${derby.jars} directory. 

        for i in *.jar 
do
  gpg --armor --detach-sign $i   // enter your PGP passphrase for each iteration.
done
for i in *.jar.asc
do
  PREFIX=`echo $i | sed 's/.jar.asc//g'`
  NEWNAME=$PREFIX-x.y.z.w.jar.asc   //use the correct version number for this release.
  mv $i $NEWNAME
done

    7. Upload the signatures to the jars directory: 

      • sftp {username}@svn.apache.org 
cd /www/people.apache.org/repo/m1-ibiblio-rsync-repository/org.apache.derby/jars/
put *.jar.asc

        {i} If you don't upload the signatures for the jar files, you will probably receive an email saying that you did not upload appropriate PGP signatures for the new files added to www.apache.org/dist/.

    8. Revert local modifications in the maven directory.

    • {i} The deployment of the jars and poms to the Maven 1 repository will be automatically converted to appropriate jars and poms for Maven 2 and deployed to that repository as well. See [WWW] DERBY-1378 for more information on the automatic conversion to Maven 2.

  5. {X} Maven may not work for you especially on Windows. If Maven does not copy the build artifacts to subdirectories under /www/people.apache.org/repo/m1-ibiblio-rsync-repository/org.apache.derby/, then you will have to do this yourself. In this scenario, you must use Maven to deploy the build artifacts to your local file system, sign them and then sftp the results to people.apache.org.

    1. To deploy the build artifacts to your local file system, set up project.properties something like this: 

      • maven.repo.list=apache 
maven.repo.apache=file://~/zdir
maven.repo.apache.directory=garbage
maven.repo.apache.username=garbage
maven.repo.apache.password=garbage
maven.repo.apache.group=garbage

    2. Then do

      • $ maven clean

        $ maven multiproject:deploy

        This will build the artifacts into a subtree rooted at garbage.

    3. Sign the jars using the following script: 

      •    for i in `ls *.jar`
   do
     gpg --armor --detach-sign $i
   done

    4. Use sftp to bulk put the artifacts into the corresponding subdirectories of www/people.apache.org/repo/m1-ibiblio-rsync-repository/org.apache.derby/.

  6. Update the release section in the Derby project DOAP file for an official release.

  7. Announce the release.

    • Twenty-four hours after putting the release files in the mirror directory, verify that you can reach them through a mirror (the mirroring will likely take effect long before this). Then, you should email derby-dev, derby-user, [MAILTO] general@db.apache.org, [MAILTO] announce@apache.org and anyone else you think might be interested an announcement concerning the release. See past release announcements for examples.

      <!> Note that you can only send emails to [MAILTO] announce@apache.org from your Apache account!

      Include a description of the project, and a description of any significant new features or important bug fixes.

      (!) Every tech news blog remotely related to Java or databases will pick up the announcement and post it verbatim, so it's a good idea to spell check it, proofread it a couple of times, and/or get input from derby-dev on its content.

  8. Tag the release in subversion. 

    • svn copy -r {rev} https://svn.apache.org/repos/asf/db/derby/code/branches/{version}/ https://svn.apache.org/repos/asf/db/derby/code/tags/{version}/ 
svn copy -r {rev} https://svn.apache.org/repos/asf/db/derby/docs/branches/{version}/ https://svn.apache.org/repos/asf/db/derby/docs/tags/{version}/

  9. Check in a copy of the jars to the jars directory of the repository.

    • Copies of the released jars are kept in the repository for use with the upgrade tests. You can check in the new jars without having to checkout all of the old jars. First, make a non-recursive checkout of the jars directory:

      svn co -N https://svn.apache.org/repos/asf/db/derby/jars/

      Then create a new directory in the checkout with the version number, copy all of the jar files into the directory, then svn add the new directory and check the directory in.

  10. Update the News section of the website.

  11. Update JIRA versions and merge development versions into released version.

    1. Mark the release id as a released version in JIRA.

      • Go to the Administration page, select Derby, then click the Manage versions link on the bottom right.

      • Click the Release link next to the version id of the release and add the release date. In addition, you may need to create a new version id for the next release vehicle on the branch.

    2. The older development versions used for tracking changes between releases are no longer needed, and old versions should be merged into the release version. E.g. for the 10.3.1.4 release, 10.3.0.0, 10.3.1.0, 10.3.1.1, 10.3.1.2, and 10.3.1.3 need to be merged into the released 10.3.1.4.

      • Click the Merge button next to the oldest release.

      • Select all the versions in the list on the right to merge and then the released version to merge to on the left (in the example above, this would be 10.3.1.4

      • Click the Merge button and confirm the merge.

  12. Update STATUS again.

    • Update STATUS to reflect that the release has occurred. Check in the new file.

You're done!

Snapshots

In essence a snapshot differs from a full release (feature, or bug fix release) in the following ways:

  1. A snapshot is an alpha or beta distribution cut from the trunk, while a release (candidate) is supposed to be a full-fledged, upgradeable distribution, and likely cut from a branch.

  2. You don't have to polish the release notes for snapshots.

  3. For snapshots, the vetting cycle is shorter and the community is willing to tolerate regressions and serious bugs in the interests of garnering early feedback.

  4. Snapshots don't necessarily include everything included in a full release, for example demos, plugins etc.

  5. Snapshots are placed in a different location:

    • It's a good idea, once the snapshot has appeared on the site, to announce to derby-dev and derby-user that the new snapshot has been posted so interested testers can grab it.

Making a (private) Quick Build of Release Artifacts

Sometimes it is useful for a Derby developer to build a private version of the release artifacts (archives), for example to verify modifications made to the structure of a specific release distribution (-bin, -lib, -lib-debug, -src, etc.).

For example, when adding a new demo or changing which files gets included in certain subdirectories of a -bin distribution, reviewing generated release artifacts is often the only way to verify that modifications made to the build scripts are correct (of course, having a developer with knowledge of the build scripts and experience with the release building process review the patch thoroughly would help as well).

Note that this section describes only a small subset of the steps needed to build a real Apache Derby release. The generated artifacts are suitable for private testing purposes only.

To build a set of release artifacts that is suitable for reviewing the file structure of a release, among other things, do the following steps (extracted from the full release procedure description above):

  1. Check out (or update) the source tree from which you want to build release artifacts, for example

  2. Make sure you are able to build the code jars the regular way, as described in [WWW] BUILDING.txt

  3. Check out (or update) the corresponding documentation tree, for example

  4. Build the documentation (all) as described on the [WWW] web site

  5. Copy tools/ant/properties/packaging.tmpl to tools/ant/properties/packaging.properties and modify as necessary (see details above). The important thing here is to include a pointer to you doc build, for example: 

    • docs.out=/export/home/tmp/user/docTrunk/trunk/out

  6. Include the following in your ant.properties file (for 10.3 or newer), modified to suit your environment: 

    • j14lib=/usr/local/java/jdk1.4/jre/lib
# For 10.4 or newer:
j15lib=/usr/local/java/jdk1.5/jre/lib
# For JDBC 4 support
jdk16=/usr/local/java/jdk1.6.0
# If you want Java ME-support:
jsr169compile.classpath=/home/user/lib/cdc/jsr169.jar:/home/user/lib/cdc/btclasses.zip
#  May not be necessary (?):
relnotes.src.reports=/home/user/derby/relnotes-reports # empty directory

  7. In the top-level directory of the source code tree, do: 

    • # If you have built release artifacts or snapshots before, clean up. Also
# clean up jars and javadoc:
rm -rf jars javadoc snapshot                   # clean.
rm -rf tools/release/crlf/ tools/release/lf/   # clean more.
rm tools/release/maintversion.properties       # really clean.
rm tools/release/*.zip tools/release/*.tar.gz  # really,
rm tools/release/*.md5 tools/release/*.asc     # really clean

# In any case do this:
ant clobber
ant sane ; ant all ; ant buildjars   # for lib-debug
ant clobber
ant insane
ant -Dsane=false snapshot
cd tools/release
ant release

The release artifacts should now be available as .zip and .tar.gz files in the tools/release/ directory.

To clean up, run svn stat to see which files don't naturally belong in the repository.

last edited 2008-05-06 17:54:27 by MyrnavanLunteren