The ManifoldCF Release Process

ManifoldCF is a little harder to release than most software projects because it has a fair number of connectors which are conditionally compiled. Thus, you cannot do a full release until the required dependencies are installed on your release machine. Luckily, for packages that are proprietary, stubs have been created which allow you to skip the installation of proprietary libraries in order to create a release.

Releases heretofore have been done on Windows (both US locale and Japan locale), and thus we feel that the release process is fairly robust for that platform. Batch scripts have been created that ease the process mightily. On Linux there are also scripts which have been used for one release.

Preparing release branch

Since this process has been error-prone and is no longer even remotely automatic (due to the unreliability of mvn versions:set), I've written scripts that do the work.  Specifically:

 

  • create_release_branch.py <release_branch_version> <new_trunk_version>
  • update_release_branch.py <release_branch_major_version> <new_branch_version>

 

The first script makes a copy of trunk and copies it to "branches/release-<version>-branch".  It modifies build.xml, CHANGES.txt, and all the pom.xml files on both branch and trunk.  The second script operates on an existing release branch to mark a bug fix release.  It updates build.xml, CHANGES.txt, and all the pom.xml files on "branches/release-<major-version>-branch".  Both scripts can be found in svn under https://svn.apache.org/repos/asf/manifoldcf/release-scripts .

Before releasing

Before you cut a release, it's always a good idea to run all the tests you can on the software. Also necessary is a "rat" run, which detects unknown or incorrect licenses on source files. See the complete checklist below.

  1. Download the dependencies for building and testing (ant make-core-deps make-deps). You will need svn 1.7+ installed for this step.
  2. Check licenses (ant rat-sources). You are interested in verifying that there are zero files with unapproved licenses. (This comes at the very start of a long report.) You will need Apache Rat installed for this step.
  3. Run standard integration tests (ant test-dr test-hs). If you have PostgreSQL installed and MySQL, run those too (ant test-pg test-my).
  4. Run the UI tests (ant uitest). You'll need Python 2.7+ for these.

If everything looks good, you're ready to go ahead and cut the release!

Building ManifoldCF for release

NOTE: In order to proceed, you MUST have svn 1.7 or higher installed! The ant build scripts require this in order to be able to apply patches on Windows platforms. You will also need GnuPG installed (from http://www.gnupg.org/download/), and have your signing key created and installed (read this: http://www.apache.org/dev/release-signing).

The first step of the build process is to create a release tag for the release candidate:

svn -m "Create RC0 tag for MCF 2.0" copy https://svn.apache.org/repos/asf/manifoldcf/branches/release-2.0-branch https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.0-RC0

The next step of the build process is to check out the tagged RC into a work area:

svn co https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.0-RC0 myworkarea

Then, check out the release scripts:

svn co https://svn.apache.org/repos/asf/manifoldcf/release-scripts

The release scripts perform all the required fetching of dependencies automatically, through ant. But they do not know how to deal with proxies, so if you have a proxy you will need to figure out how to deal with that.

Next, you will need to check out the ManifoldCF dist/dev svn directory:

svn co https://dist.apache.org/repos/dist/dev/manifoldcf devarea

The final step is to fire off the release script itself. You will need to supply your people.apache.org username also, as well as your SSH private key:

cd myworkarea
../release-scripts/release.bat 2.0 ../devarea

Your release candidate should be created, copied, and committed to an appropriate subdirectory of the dist/dev/manifoldcf area.

Calling the vote

Once the artifact has been uploaded to people.apache.org, you should call a vote to release the artifact. This involves sending an email to dev@manifoldcf.apache.org, with the title "[VOTE] Release Apache ManifoldCF xxxx RCnnn". In the vote email, mention where the artifacts can be downloaded from (https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-xxxx), as well as the full svn URL of the tag you created (https://svn.apache.org/repos/asf/manifoldcf/tags/release-xxxx-RCnnnn). The vote must have three +1's from committers, and must be open at least 72 hours.

If the vote does not pass

If the vote does not pass, this usually implies that fixes will be made and a new RC created. Fixes usually are committed to trunk first and then pulled up into the release branch using the "svn merge" command, e.g. "svn merge -c ZZZZ https://svn.apache.org/repos/asf/manifoldcf/trunk". Before the creating of the next RC, you should svn move the old artifacts from the apache-manifoldcf-XXX directory to apache-manifoldcf-XXX-RCnnnn. Then, upload the new RC as before.

If the vote passes

When the vote passes, you should "svn move" the artifacts from the dist/dev area directly into the dist/release area. DO NOT REGENERATE THE ARTIFACTS - the community has voted on them and they are the only legal copies you should distribute:

svn -m "Release MCF 2.0" move https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.0 https://dist.apache.org/repos/dist/release/manifoldcf/

Once that is done, you should also use "svn move" to rename the artifact RC tag you created to remove the -RCnnnn suffix from it.

svn -m "Release RC0" move https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.0-RC0 https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.0

The second step is to modify the MCF site to reflect the new release.  To do this, you must do the following:

  1. Check out the mcf site trunk: "svn co https://svn.apache.org/repos/asf/manifoldcf/site/trunk"
  2. Add the release to build.xml (and remove any older releases we no longer want to publish information for)
  3. Add the release download links to all of the various language's download.xml pages
  4. On the download pages, be sure to change the previous release so it is no longer downloaded from dyn/closer.cgi, and is instead downloaded from the Apache archive server
  5. Update the index.xml pages to announce the new release

Publishing the site is covered elsewhere.  When that has been completed, you need to remove the old release from the dist area:

  1. Check out the ManifoldCF dist release area: "svn co https://dist.apache.org/reps/dist/release/manifoldcf"
  2. svn remove the old release: "svn remove apache-manifoldcf-XXX"
  3. Commit the removal

 

  • No labels