This page is to help a Lucene/Solr committer create a new release (you need committer rights for some of the steps to create an official release). It does not reflect official release policy - many of the items may be optional, or may be modified as necessary.

See the ASF Releases Policy page.

Release Process

Prior to every major or minor release (i.e. all except bugfix-only point releases) a feature freeze phase takes place for about 1-2 weeks. At the beginning of the feature freeze the trunk is branched and no commits other than serious bug fixes, documentation or build updates are permitted. This period of time should be used for extensive testing, documentation improvements and for cleaning up old JIRA issues.

Prerequisites

  1. (Optional) Edit your gnupg/gpg.conf file and add a default-key value with your Code Signing Key (as a HEX value) otherwise you may need to specify -Dgpg.key=XXXXXX later for signing. Make sure it is your 4K-bit key and not a 1K-bit key (esp. for ASF old timers). See http://www.apache.org/dev/release-signing.html for more information.

  2. Make sure you put your GPGP key's fingerprint in the OpenPGP Public Key Primary Fingerprint field in your profile on https://id.apache.org

  3. You may need to setup people.apache.org as a known host for ssh if you haven't already. This happens when you log in manually and follow the prompts.

Planning

  1. On dev@lucene.apache.org, decide on

    1. which JIRA issues shall be committed before a release is made; set the appropriate "Fix Version" in JIRA for these issues
    2. the date for branching the trunk and the start of the feature freeze phase
    3. the length of the feature freeze phase
    4. a tentative release date
  2. Put up (and/or fix up existing) release notes, at http://wiki.apache.org/lucene-java/ReleaseNoteXY(Z) and http://wiki.apache.org/solr/ReleaseNoteXY(Z) and ask on dev@ for input

Branching & Feature Freeze

  1. Run ant precommit to run a bunch of sanity & quality checks, then fix any problems that are found.

  2. For the first release in a minor release series - i.e. X.Y.0 - create a minor release branch off the current major version branch, e.g. for minor release 4.2:
    svn copy https://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x \
    https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_1 -m "Starting Lucene/Solr 5.1 branch."
  3. Add a new version for the next release using the addVersion.py script. If it is a bugfix release, we will be adding the bugfix version, otherwise we will add the version to come after the release we are producing.

    • If a bugfix release, start on the release branch:
      •  python3 -u dev-tools/scripts/addVersion.py X.Y.Z 

      •  svn commit 

      • Using the commit id from the previous commit, run addVersion.py on the stable branch, adding  --changeid ID --downstream-repo /path/to/release/branch/checkout

      •  svn commit 

      • Run addVersion.py on trunk, modifying the stable branch command with the new change id and path to the stable branch checkout.

      •  svn commit 

      • Make sure that the backcompat index for the previous release has been added to the release branch. (Note that this will ordinarily *not* have been done if the current release is X.Y.1, i.e. the first bugfix release off the stable branch.) See the post-release section "Generate Backcompat Indexes" below - remember you'll be generating an index for the *previous* release.

    • If a minor release, start on the stable branch:
      •  python3 -u dev-tools/scripts/addVersion.py X.Y+1.0 

      •  svn commit 

      • Using the commit id from the previous commit, run on trunk, adding  --changeid ID --downstream-repo /path/to/stable/branch/checkout

      •  svn commit 

    • If a major release, only run on trunk:
      •  python3 -u dev-tools/scripts/addVersion.py X+1.0.0 

      • The script will print a list of items that need to be done manually for a major release bump.
      •  svn commit 

  4. Send a note to dev@ to inform the committers that the branch has been created and the feature freeze phase has started. Include Do's and Don'ts for the feature freeze phase:
    • No new features may be committed to the branch.
    • Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.

    • All patches that are intended for the branch should first be committed to trunk, merged into the minor release branch, and then into the current release branch.
    • Normal trunk and minor release branch development may continue as usual. However, if you plan to commit a big change to the trunk while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.
    • Only Jira issues with Fix version "X.Y" and priority "Blocker" will delay a release candidate build.

Release Workflow

  1. Go to the JIRA "Manage Versions" Administration pages (https://issues.apache.org/jira/plugins/servlet/project-config/LUCENE/versions and https://issues.apache.org/jira/plugins/servlet/project-config/SOLR/versions) and add a new (unreleased) version for the next release on the trunk (for a major release) or branch (for a minor release).

  2. Check out the branch with: svn co https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_X_Y

  3. Build the code and javadocs, and run the unit tests: ant clean test, ant javadocs (in lucene/). Make sure that you are actually using the minimum compiler version supported for the release. For example, 5.x releases are on Java7 so make sure that you use Java7 for the release workflow.
  4. Examine the results. Did it build without errors? Were there Javadoc warnings? Did the tests succeed? Does the demo application work correctly? Does Test2BTerms pass (this takes a lot of memory)?
  5. Remove lucene/benchmark/{work,temp}/ if present

  6. Build as defined below in the "Building the Release Artifacts" section.
  7. If during the feature freeze phase bug fixes are committed to the X.Y branch then build another release candidate and announce on dev@ and java-user@ that everyone should use the new RC for testing.
  8. Call a release vote on dev@ and cc general@lucene.apache.org .

Building the Release artifacts

If after the last day of the feature freeze phase no blocking issues are in JIRA with "Fix Version" X.Y then it's time to build the release artifacts.

  1. It is recommended to clean your Ivy cache by executing rm -rf ~/.ivy2/cache before building the artifacts. This ensures that all Ivy dependencies are freshly downloaded, so we emulate a user that never used the Lucene build system before (this step ensures downloadability of all artifacts). If you have a ton of stuff in your ivy cache, it's a good idea to move the cache temporarily to another location and restore it after the RC has been built so you wouldn't have to download everything again.

  2. Option 1 (recommended): use dev-tools/scripts/buildAndPushRelease.py to build a release candidate and stage the results locally. Run buildAndPushRelease.py with  --help  to learn the available options. Here's an example of what was done for the 5.1 release:

    1. Build the release candidate: python3 -u dev-tools/scripts/buildAndPushRelease.py --push-local /tmp/releases/5.1 --rc-num 2 /complete/path/to/svn/checkout/of/releasebranch

    2. Optionally, run the smoke test script against the local release candidate: python3 -u dev-tools/scripts/smokeTestRelease.py /tmp/releases/5.1/lucene-solr-5.1.0-RC2-rev1672403

    3. Import the artifacts into SVN: svn -m "Lucene/Solr 5.1.0 RC2" import /tmp/releases/5.1/lucene-solr-5.1.0-RC2-rev1672403 https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403

    4. Don't delete these artifacts from your local workstation as you'll need to publish the maven subdirectories once the RC passes (see below).
  3. Option 2: manually build a release candidate:
    1. Verify that svnversion reports a single revision with no modified changes

    2. Remove lucene/benchmark/{work,temp}/ if present

    3. Build the release artifacts: ant -Dversion=X.Y.Z [-Dgpg.exe=/path/to/gpg -Dgpg.key=YourKeyID -Dgpg.passphrase=YourPassPhrase] prepare-release (Items in [] may be optional)

    4. Sanity check the overall size of each artifact. EG, compare to the last release.
    5. Make sure that for each release file an md5 checksum file exists.
    6. ant -Dversion=X.Y.Z copy-to-stage to move the artifacts up to the staging area (see the build target for all the options for securly copying the artifacts up)

    Example for 3.1:
    •      PROMPT$ ANT_OPTS="-Xmx256m -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M" ant -Dversion=3.1.0 -Dgpg.key=FE045966 prepare-release
           # Inspect the artifacts
           PROMPT$ ant -Dversion=3.1.0 -Drc=rc3 -Dscp.user=gsingers copy-to-stage

Testing the Release artifacts

  1. There is a script in SVN to do automated checks on a release candidate, e.g.

python3 -u dev-tools/scripts/smokeTestRelease.py https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403
  1. If the smoke test passes against the staged artifacts, send an email to the dev mailing list announcing the release candidate. Here is a template you can use:

Please vote for the # release candidate for Lucene/Solr 5.1.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403/

You can run the smoke tester directly with this command:
python3 -u dev-tools/scripts/smokeTestRelease.py
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403/

Here's my +1
SUCCESS! [0:43:35.208102]
  1. If the key you used to sign the release candidate artifacts has not been signed by other Apache committers, then testers may see the following warning:

 verify trust
      GPG: gpg: WARNING: This key is not certified with a trusted signature!

Publishing

Once three PMC members have voted for a release, it may be published. You should wait at least 72 hours, not including weekend days. For instance, if you announce the RC on Friday, give until end of day Tuesday or Wednesday morning for the vote.

  1. Announce that the vote has passed on the dev mailing list
  2. Tag the release from the same revision from which the passing release candidate's was built:
    svn copy -r1492800 https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_X_Y \
    https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_X_Y_Z -m "Lucene/Solr release X.Y.Z"
  3. If this is a point release, copy the CHANGES.txt entry for this point release back to the trunk's CHANGES.txt. Remove any duplicate entries from trunk's CHANGES.txt (ie, each issue should appear only once, in the earliest point-release that contains the fix, on the assumption that all future releases also contain the fix).
  4. Delete the maven artifacts from the staging repo (assuming you still have them locally); if you don't have the maven directories locally, you'll need to download them from subversion before deleting. Also delete any failed RCs that didn't pass the voting process from the staging repo.
  5. Move the new release artifacts to the releases repo:

svn move -m "Move Lucene RC2 to release repo." https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403/lucene https://dist.apache.org/repos/dist/release/lucene/java/5.1.0
svn move -m "Move Solr RC2 to release repo." https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.1.0-RC2-rev1672403/solr https://dist.apache.org/repos/dist/release/lucene/solr/5.1.0
  1. Commit the release candidate. # this might take some time: ~ 2 beers
  1. Publish Maven Artifacts

  2. Maven central should show the release after a short while, but you need to wait 24 hours to give the Apache mirrors a chance to copy the new release. There is a script that will continually check the number and percentage of mirrors (and Maven Central) that have the release: dev-tools/scripts/poll-mirrors.pl.

Pushing website changes & Announcing

Website += javadocs

The problem: Lucene's and Solr's voluminous per-release javadocs break the standard CMS process for the website (i.e., committer commits to the source tree; the CMS buildbot generates the site, then commits to the staging tree; committer reviews and then publishes to the production tree), because dynamic website updates, currently scheduled at 21 minutes after the hour on the hour, interrupt the extremely long commit times for javadocs being staged by buildbot, resulting in failed commits, caused by conflicts with the dynamic updates: by the time the buildbot-triggered commit has finished, its svn tree has been rendered stale.

The solution: skip committing javadocs to the source tree, then staging, then publishing, and instead commit javadocs directly to the production tree. Ordinarily this would be problematic, because the CMS wants to keep the production tree in sync with the staging tree, so anything it finds in the production tree that's not in the staging tree gets nuked. However, the CMS has a built-in mechanism to allow exceptions to the keep-production-in-sync-with-staging rule: extpaths.txt.

extpaths.txt lists paths in the production tree, relative to the project website's root directory, that are allowed to be out of sync with the staging tree.

For more info, see the following sections in the Apache CMS Reference:

Update extpaths.txt

  1. svn co --depth=immediates https://svn.apache.org/repos/asf/lucene/cms/trunk/content website-source

  2. cd website-source

  3. Add Lucene javadocs dir: echo core/X_Y_Z >> extpaths.txt

  4. Add Solr javadocs dir: echo solr/X_Y_Z >> extpaths.txt

  5. svn commit -m "Update CMS production sync exceptions for X_Y_Z javadocs" extpaths.txt

  6. cd ..

  7. rm -rf website-source

Push javadocs to the CMS production tree

Don't be alarmed that you don't see your extpaths.txt changes yet in this production checkout: after you do this section you will then be able to publish the lucene website from staging and extpaths.txt will update in this repository automatically.

  1. Ensure your refridgerator has at least 4 beers.
  2. svn co --depth=immediates https://svn.apache.org/repos/infra/websites/production/lucene/content/ website-production

  3. untar the lucene and solr release .tgz to create lucene-X.Y.Z and solr-X.Y.Z
  4. cd website-production; svn update --depth=immediates core solr; cd ..

  5. cp -r lucene-X.Y.Z/docs website-production/core/X_Y_Z

  6. svn add website-production/core/X_Y_Z

  7. svn commit website-production/core/X_Y_Z # this might take some time: ~ 3 beers

  8. cp -r solr-X.Y.Z/docs website-production/solr/X_Y_Z

  9. svn add website-production/solr/X_Y_Z

  10. svn commit website-production/solr/X_Y_Z # this might take some time: ~ 1 beer

  11. Confirm you can browse to these directories manually, and especially that solr javadocs link back to lucene's correctly.

Update redirect to latest Javadoc

We make it possible to link to latest javadoc by providing redirect links for e.g. http://lucene.apache.org/solr/api/solr-core/ which will auto redirect to whatever is the latest released version, i.e. http://lucene.apache.org/solr/4_3_0/solr-core/ for 4.3.0. This is handled in .htaccess:

  1. Goto CMS root http://lucene.apache.org/

  2. Click on the CMS bookmarklet

  3. Scroll down to the file .htaccess and click [Edit]

  4. Locate the three lines starting with RedirectMatch temp /core|solr/api/... and change the version url component to match X_Y_Z above. Click Submit

  5. Click Commit, fill in a message and commit your change. No need to publish site yet - that will be done in next chapter.

NOTE As an alternative to CMS online editing, you can checkout the site from SVN and edit/commit .htaccess that way.

Update the rest of the website

NOTE: don't do this until you are ready to go to production. Every hour at :21 (e.g. 8:21, 9:21, ...) buildbot will commit all content from staging to production.

  1. Once mirrors are ready, use the bookmarklet or whatever to do the minor changes and news blurbs and stuff.
  2. Update the core & solr doap.rdf files to reflect the new versions using the Apache CMS

  3. Publish the site, e.g. by visiting http://lucene.staging.apache.org, invoking the CMS bookmarklet, then clicking the publish link.

  4. Wait for these changes to appear on both of Apache's main webservers (US: http://lucene.us.apache.org, EU: http://lucene.eu.apache.org, http://lucene.apache.org is dependent on your own geographic location, so the other mirror may still be outdated) before doing the next steps (see http://www.apache.org/dev/project-site.html for details on how the site is mirrored to Apache's main web servers). Once they appear, verify all links are correct in your changes!

Announce the release

Release announcements can be shared/edited on the wiki at http://wiki.apache.org/lucene-java/ReleaseNoteXY(Z) and http://wiki.apache.org/solr/ReleaseNoteXY(Z)

For feature releases, your announcement should describe the main features included in the release; typically this is pulled from the wiki.

Mails to the announce@apache.org list must be sent from an @apache.org email address and should contain a signature.

Because you're likely not subscribed to the general, dev, and -user lists with your @apache.org address, sending the announcement email to those list with your @apache.org address will need to be moderated through, which will likely result in delayed transmission. To avoid this, send the emails these lists using your subscribed email address, and then separately send the announcements to announce@apache.org.

  1. Announce the Lucene release on mailing lists general@lucene.apache.org , dev@lucene.apache.org , java-user@lucene.apache.org and announce@apache.org

  2. Announce the Solr release on mailing lists general@lucene.apache.org , dev@lucene.apache.org , solr-user@lucene.apache.org and announce@apache.org

  3. Add the new version to Wikipedia (english and maybe your own language)

Post-release

Generate Backcompat Indexes

After each version of Lucene is released, compressed CFS and non-CFS indexes created with the newly released version are added to lucene/backwards-codecs/src/test/org/apache/lucene/index/, for use in testing backward index compatibility via org.apache.lucene.index.TestBackwardsCompatibility, which is also located under the backwards-codecs/ module. These indexes are created via methods on TestBackwardsCompatibility itself - see comments in the source for more information.

There is a script (dev-tools/scripts/addBackcompatIndexes.py) that automates most of the process: it downloads the source for the specified release; generates indexes for the current release using TestBackwardsCompatibility; compresses the indexes and places them in the correct place in the source tree; modifies TestBackwardsCompatibility.java to include the generated indexes in the list of indexes to test; and then runs TestBackwardsCompatibility. Run this script on the stable branch and on trunk, and if the current release is a bugfix release, also on the release branch. On each branch, after running the script, svn add the generated indexes, then svn commit.

TestBackwardsCompatibility will not test indexes for Version.LATEST, and will fail if it finds indexes from that version, so before you run addBackcompatIndexes.py, make sure dev-tools/scripts/addVersion.py has been run with the next version to be released on each of the branches - see item #3, about adding a new version, in the Branching & Feature Freeze section, above. This will likely only be an issue on the release branch, if the current release is a bugfix release, since addVersion.py will have already been run on the stable branch and on trunk before the release process has gotten to this point.

To print the script's usage, run it with the --help option: python3 -u dev-tools/scripts/addBackcompatIndexes.py --help

The script uses a scratch directory - /tmp/lucenebwc/ by default if you don't specify --temp-dir DIR - to store the source for the current release that it downloads, along with the generated indexes. This directory will be removed by the script unless you specify the --no-cleanup option, which is useful when running the script on multiple branches; in the example below, the --no-cleanup option is included on the non-final script invocations, but not on the final invocation, so that the scratch directory will be cleaned up after it's no longer needed.

Here's an example for the 5.1.0 release:

# If this were a bugfix release, e.g. 5.1.1, on the release branch we would first run addVersion.py
# if version 5.1.2 has not yet been added, and then addBackcompatIndexes.py:
#
#    cd /path/to/lucene/dev/branches/lucene_solr_5_1
#    svn up && svn stat   # Make sure there are no uncommitted changes
#    java7    # aliased to: export JAVA_HOME="$JAVA7_HOME" ; export PATH="$JAVA7_HOME/bin:$PATH"
#    ant clean
#    python3 -u dev-tools/scripts/addVersion.py 5.1.2  # Add the next version after the current release
#    svn ci -m "Add version 5.1.2"
#    python3 -u dev-tools/scripts/addBackcompatIndexes.py --no-cleanup 5.1.1
#    svn add lucene/backward-codecs/src/test/org/apache/lucene/index/index.5.1.1-*.zip
#    svn stat # Look for pending mods to TestBackwardsCompatibility.java and adds of the generated indexes
#    svn ci -m "Add 5.1.1 back compat test indexes"
#
# Then also execute all the commands below, substituting 5.1.1 where it says 5.1.0.

cd /path/to/lucene/dev/branches/branch_5x
svn up && svn stat   # Make sure there are no uncommitted changes
java7     # aliased to: export JAVA_HOME="$JAVA7_HOME" ; export PATH="$JAVA7_HOME/bin:$PATH"
ant clean
python3 -u dev-tools/scripts/addBackcompatIndexes.py --no-cleanup 5.1.0
svn add lucene/backward-codecs/src/test/org/apache/lucene/index/index.5.1.0-*.zip
svn stat # Look for pending mods to TestBackwardsCompatibility.java and adds of the generated indexes
svn ci -m "Add 5.1.0 back compat test indexes"

cd /path/to/lucene/dev/trunk
svn up && svn stat   # Make sure there are no uncommitted changes
java8     # aliased to: export JAVA_HOME="$JAVA8_HOME" ; export PATH="$JAVA8_HOME/bin:$PATH"
ant clean
python3 -u dev-tools/scripts/addBackcompatIndexes.py 5.1.0
svn add lucene/backward-codecs/src/test/org/apache/lucene/index/index.5.1.0-*.zip
svn stat # Look for pending mods to TestBackwardsCompatibility.java and adds of the generated indexes
svn ci -m "Add 5.1.0 back compat test indexes"

Update JIRA

  1. Go to the JIRA "Manage Versions" Administration pages (https://issues.apache.org/jira/plugins/servlet/project-config/LUCENE/versions and https://issues.apache.org/jira/plugins/servlet/project-config/SOLR/versions). Next to the version you'll release, click the gear pop-up menu icon and choose "Release". It will ask you for the release date -- enter it. It will give the option of transitioning issues marked fix-for the released version to the next version, but do not do this as it will send an email for each issue -- we'll address that separately.

  2. Go to JIRA search in both Solr and Lucene and find all issues that were fixed in the release you just made, whose Status is Resolved. This URL may work (but edit the fixVersion part): (https://issues.apache.org/jira/issues/?jql=project+in+(LUCENE,SOLR)+AND+status=Resolved+AND+fixVersion=4.10.1). Do a bulk change (Under Tools... menu) to close all of these issues (this is a workflow transition task). Uncheck the box that says "send an email for these changes".

  3. Do another JIRA search in both Solr and Lucene to find all issues with Unresolved Resolution and fixVersion of the release you just made, and do a bulk change to the fixVersion to be both the trunk version and the next version on the branch you just released from. Uncheck the box that says "send an email for these changes".
  4. Add a new Version for the next possible release version on the "Manage Versions" Administration page (https://issues.apache.org/jira/plugins/servlet/project-config/LUCENE/versions). e.g. If the current release is 5.2.1, add 5.2.2 with a description so that contributors can commit to the release branch with the next release version. In case of a minor release e.g. 5.2, this step needs to be done when the new release branch is cut.

Don't mirror old releases

Shortly after new releases are first mirrored, they are copied to the archives. Only the latest point release from each active branch should be kept under the Lucene PMC svnpubsub area dist/releases/lucene/. Older releases can be safely deleted from dist/releases/lucene/, since these releases are already backed up in the archives.

svn remove old releases, including X.Y-1, from dist/releases/lucene/java/ and dist/releases/lucene/solr/, then svn commit.

Update WIKI

The Solr WIKI has a page for every version which is often linked to from WIKI pages to indicate differences between versions, example: http://wiki.apache.org/solr/Solr4.3. Do the following:

  1. Update the page for the released version with release date and link to release statement
  2. Create a new placeholder page for the "next" version, if it does not exist

ReleaseTodo (last edited 2015-06-22 19:06:32 by AnshumGupta)