This page is to document the release procedure for Pig. Pig is a fairly young Apache project. Its release process is work in progress and is modeled from Hadoop Release Procedure.

Note that only Pig commiters can create a release.

Preparation

TODO:

  1. Define issue management process like assigning/removing issues from release]

Creating Release Branch

We only branch for major (X.0.0) and minor(X.Y.0) releases but not for patches (X.Y.Z). Patch is and update to an existing branch created for X.Y.0.

  1. Send email to pig-dev@hadoop.apache.com to notify that you about to branch the tree and ask to hold off any commits till this is finished.

  2. Update CHANGES.txt to include the release version and date (use Unreleased for the date if it is unknown) and remove Trunk (unreleased changes). Below is the example of the top of the CHANGES.txt file after the update:

    Pig Change Log

Release 0.1.1 - Unreleased

INCOMPATIBLE CHANGES

NEW FEATURES

IMPROVEMENTS
PIG-253: integration with hadoop-18

BUG FIXES
PIG-342: Fix DistinctDataBag to recalculate size after it has spilled. (bdimcheff via gates)

  3. Commit these changes to trunk:

    svn commit -m "Preparing for release X.Y.Z"

  4. Create a branch for the release series:

    svn copy https://svn.apache.org/repos/asf/hadoop/pig/trunk \
https://svn.apache.org/repos/asf/hadoop/pig/branches/branch-X.Y -m "Branching for X.Y releases"

  5. Update CHANGES.txt to add back in Trunk (unreleased changes). Top of the CHANGES.txt should look like this now:

    Pig Change Log

Trunk (unreleased changes)

  INCOMPATIBLE CHANGES

  IMPROVEMENTS

  OPTIMIZATIONS

  BUG FIXES

Release 0.1.1 - Unreleased
....

  6. Update the default version in build.xml on trunk to X.Y+1.0-dev.

  7. Commit these changes to trunk:

    svn commit -m "Preparing for X.Y+1.0 development"

TODO:

  1. Add documentation update the process once we integrate the documentation into forrect. (Will need docs target in build.xml)

Updating Release Branch

The steps in this section are needed for all the releases (major, minor, and patches).

  1. Check out the branch with:

    svn co https://svn.apache.org/repos/asf/hadoop/pig/branches/branch-X.Y

  2. Run rat report and make sure that all files that can have apache license agreement. Note that currently the rat report is part of output and needs to be manually extracted. TODO: write the report into a file.

    ant -Djava5.home=<java5 jdk dir>  -Dforrest.home=<forrest installation dir> releaseaudit >out
  3. For patches, update CHANGES.txt to include the release version and date. See #2 from Create Release Branch section.
  4. Update RELEASE_NOTES.txt with release notes for this release.

  5. Update the version number in build.xml to be X.Y.N-dev, where N is one greater than the release being made.

  6. Commit these changes:

    svn commit -m "Preparing for release X.Y.Z"

  7. Tag the release candidate:

    svn copy https://svn.apache.org/repos/asf/hadoop/pig/branches/branch-X.Y \
https://svn.apache.org/repos/asf/hadoop/pig/tags/release-X.Y.Z-rcR -m "Pig X.Y.Z-rcR release."

Building

  1. Build the release and run unit tests: 

    ant clean
ant test
ant clean
ant -Dversion=X.Y.Z  -Djava5.home=<Java 5 JDK location> -Dforrest.home=<Forrest installation dir>  tar
  2. Test the tar file by unpacking the release and

    • building pig.jar: ant 

    • building and running tutorial 

      cd tutorial
ant

    • build piggybank

      cd contrib/piggybank/java
ant

    • running unit tests ant test

  3. Generate the MD5 checksum of the release file: 

    cd build
md5sum pig-X.Y.Z.tar.gz > pig-X.Y.Z.tar.gz.md5

  4. Sign the release (see Step-By-Step Guide to Mirroring Releases for more information). [TODO: add details on how to generate and store keys]

    gpg --armor --output pig-X.Y.Z.tar.gz.asc --detach-sig pig-X.Y.Z.tar.gz

  5. Copy release files to a public place (usually into public_html in your home directory):

    ssh people.apache.org mkdir public_html/pig-X.Y.Z-candidate-0
scp -p pig-X.Y.Z.tar.gz* people.apache.org:public_html/pig-X.Y.Z-candidate-0
cd ..
scp RELEASE_NOTES.txt people.apache.org:public_html/pig-X.Y.Z-candidate-0

  6. Call a release vote. The initial email should be sent to pig-dev@hadoop.apache.org. Make sure to attache rat report to it. Here is a sample of email: 

    From: Olga Natkovich [mailto:olgan@yahoo-inc.com] 
Sent: Tuesday, November 25, 2008 3:59 PM
To: pig-dev@hadoop.apache.org
Subject: [VOTE] Release Pig 0.1.1 (candidate 0)

Hi,

I have created a candidate build for Pig 0.1.1. This release is almost identical to Pig 0.1.0 with a couple of exceptions:

(1) It is integrated with hadoop 18
(2) It has one small bug fix (PIG-253)
(3) Several UDF were added to piggybank - pig's UDF repository

The rat report is attached.

Keys used to sign the release are available at http://svn.apache.org/viewvc/hadoop/pig/trunk/KEYS?view=markup.

Please download, test, and try it out:

http://people.apache.org/~olga/pig-0.1.1-candidate-0

Should we release this? Vote closes on Wednesday, December 3rd.

Olga

Forward the initial email to private@hadoop.apache.org for Hadoop PMC members to vote.

Publish

Once three PMC members have voted for a release, it may be published.

  1. Tag the release: 

    svn move https://svn.apache.org/repos/asf/hadoop/pig/tags/release-X.Y.Z-rcR \
https://svn.apache.org/repos/asf/hadoop/pig/tags/release-X.Y.Z -m "Pig X.Y.Z release."

  2. Copy release files to the distribution directory and make them writable by the hadoop group.

    ssh people.apache.org
cp -pr public_html/pig-X.Y.Z-candidate-0 /www/www.apache.org/dist/hadoop/pig/pig-X.Y.Z
cd /www/www.apache.org/dist/hadoop/pig
chgrp -R hadoop pig-X.Y.Z
chmod -R g+w pig-X.Y.Z

  3. The release directory usually contains just two releases, the most recent from two branches, with a link named 'stable' to the most recent recommended version.

    ssh people.apache.org
cd /www/www.apache.org/dist/hadoop/pig
rm -rf pig-A.B.C; rm stable
ln -s pig-A.B.D stable
  4. Wait 24 hours for release to propagate to mirrors.

  5. Prepare to edit the website.

    svn co https://svn.apache.org/repos/asf/hadoop/pig/site
  6. Update the documentation links in author/src/documentation/content/xdocs/index.xml.
  7. Update the release news in author/src/documentation/content/xdocs/releases.xml.

  8. Copy in the release specific documentation 

        cd publish
    mkdir docs/rX.Y.Z
    cp -pr <releasebuilddir>/build/docs/* publish/docs/rX.Y.Z/
    svn add publish/docs/rX.Y.Z

  9. Regenerate the site, review it and commit in HowToCommit.

  10. Deploy your site changes.

    ssh people.apache.org
cd /www/hadoop.apache.org/pig
svn up
  11. Wait until you see your changes reflected on the Apache web site.

  12. Send announcements to the user and developer lists as well as (announce@haoop.apache.org) once the site changes are visible. 

    Pig  team is happy to announce Pig X.Y.Z release. 

Pig is Hadoop subproject which provides high-level data-flow language and execution framework for parallel computation on Hadoop clusters.
More details about Pig can be found at http://hadoop.apache.org/pig/.

The highlights of this release are ... The details of the release can be found at http://hadoop.apache.org/pig/releases.html.

[TODO Need to integrate javadoc into this.] [TODO Integrate with JIRA]

HowToRelease (last edited 2009-09-20 23:38:16 by localhost)