This page is to document the release procedure for Pig. Pig is a fairly young Apache project. Its release process is work in progress and is modeled from Hadoop Release Procedure.
Note that only Pig committers can create a release.
TODO:
We only branch for major (X.0.0) and minor(X.Y.0) releases but not for patches (X.Y.Z). Patch is an update to an existing branch created for X.Y.0.
dev@pig.apache.org
to notify that you about to branch the tree and ask to hold off any commits till this is finished.Update CHANGES.txt
to include the release version and date (use Unreleased
for the date if it is unknown) and remove Trunk (unreleased changes)
. Below is the example of the top of the CHANGES.txt
file after the update:
Pig Change Log Release 0.1.1 - Unreleased INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS PIG-253: integration with hadoop-18 BUG FIXES PIG-342: Fix DistinctDataBag to recalculate size after it has spilled. (bdimcheff via gates) |
Commit these changes to trunk:
svn commit -m "Preparing for release X.Y.Z" |
Create a branch for the release series:
svn copy https://svn.apache.org/repos/asf/pig/trunk \ https://svn.apache.org/repos/asf/pig/branches/branch-X.Y -m "Branching for X.Y releases" |
Update CHANGES.txt to add back in Trunk (unreleased changes). Top of the CHANGES.txt
should look like this now:
Pig Change Log Trunk (unreleased changes) INCOMPATIBLE CHANGES IMPROVEMENTS OPTIMIZATIONS BUG FIXES Release 0.1.1 - Unreleased .... |
build.xml
on trunk to X.(Y+1).0-dev.Commit these changes to trunk:
svn commit -m "Preparing for X.(Y+1).0 development" |
The steps in this section are needed for all the releases (major, minor, and patches).
Check out the branch with:
svn co https://svn.apache.org/repos/asf/pig/branches/branch-X.Y |
Run rat report and make sure that all files that can have apache license agreement. Save this rat report.
ant releaseaudit |
build.xml
to be X.YCommit this changes:
svn commit -m "Preparing for release X.Y.Z" |
Tag the release candidate:
svn copy https://svn.apache.org/repos/asf/pig/branches/branch-X.Y \ https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR -m "Pig X.Y.Z-rcR release." |
Building
Build and run unit tests:
ant clean ant jar test |
Check that contrib and tutorial directories compile and tests pass:
cd contrib/piggybank/java ant ant test cd ../../../tutorial ant cd .. |
Build the source release and save the src.tar.gz artifact:
ant -Dversion=X.Y.Z src-release |
ant
building and running tutorial
cd tutorial ant cd .. |
build piggybank
cd contrib/piggybank/java ant ant test cd ../../.. |
ant test-commit
Build the convenience artifacts:
ant clean ant test ant clean ant jar cd contrib/piggybank/java ant cd ../../.. ant -Dversion=X.Y.Z -Dforrest.home=<Forrest installation dir> tar-h12 |
Generate the MD5 checksum of the release artifact and convenience binaries:
cd build md5sum pig-X.Y.Z-src.tar.gz > pig-X.Y.Z-src.tar.gz.md5 md5sum pig-X.Y.Z.tar.gz > pig-X.Y.Z.tar.gz.md5 |
Generating key pair using the following command. You can simply accept all default settings and give your name, email and Passphase.
gpg --gen-key |
Export your public key.
gpg --armor --output pubkey.txt --export 'Your Name' |
Open pubkey.txt, copy the full text and append it to the following files by pasting, then commit these changes:
https://svn.apache.org/repos/asf/pig/branches/branch-X.Y.Z/KEYS https://svn.apache.org/repos/asf/pig/trunk/KEYS |
Upload updated KEYS to Apache.
scp KEYS people.apache.org:/www/www.apache.org/dist/pig/KEYS |
Export your private key, keep it with you.
gpg --export-secret-key -a "Your Name" > private.key |
Sign the release artifact only (see Step-By-Step Guide to Mirroring Releases for more information).
gpg --armor --output pig-X.Y.Z-src.tar.gz.asc --detach-sig pig-X.Y.Z-src.tar.gz |
Verify gpg signature.
gpg --import KEYS (if necessarily) gpg --verify pig-X.Y.Z-src.tar.gz.asc pig-X.Y.Z-src.tar.gz |
Copy release artifact, convenience binaries, release notes and the rat report to a public place (usually into public_html in your home directory):
ssh people.apache.org mkdir public_html/pig-X.Y.Z-rcR scp -p pig-X.Y.Z* people.apache.org:public_html/pig-X.Y.Z-rcR cd .. scp RELEASE_NOTES.txt people.apache.org:public_html/pig-X.Y.Z-rcR scp <rat_report> people.apache.org:public_html/pig-X.Y.Z-rcR |
Create file ~/.m2/settings.xml with following contents (NOTE: It is highly recommended to use Maven's password encryption capabilities for your passwords.):
<settings> <servers> <server> <id>apache.releases.https</id> <username>your apache id</username> <password>your apache password</password> </server> <server> <id>apache.snapshots.https</id> <username>your apache id</username> <password>your apache password</password> </server> <server> <id>apache.staging.https</id> <username>your apache id</username> <password>your apache password</password> </server> </servers> <profiles> <profile> <id>apache-release</id> <properties> <gpg.passphrase>your GPG password</gpg.passphrase> </properties> </profile> </profiles> </settings> |
ant mvn-deploy
to publish Pig artifacts to the apache snapshot repository.ant –Drepo=staging –Dversion=X.Y.Z mvn-deploy
to publish Pig artifacts to the apache staging repository.Call a release vote. The initial email should be sent to dev@pig.apache.org
. Here is a sample of email:
From: Olga Natkovich [mailto:olgan@yahoo-inc.com] Sent: Tuesday, November 25, 2008 3:59 PM To: dev@pig.apache.org Subject: [VOTE] Release Pig 0.1.1 (candidate 0) Hi, I have created a candidate build for Pig 0.1.1. This release is almost identical to Pig 0.1.0 with a couple of exceptions: (1) It is integrated with hadoop 18 (2) It has one small bug fix (PIG-253) (3) Several UDF were added to piggybank - pig's UDF repository Keys used to sign the release are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup. Please download, test, and try it out: http://people.apache.org/~olga/pig-0.1.1-rc0 Release notes and the rat report are available from the same location. Should we release this? Vote closes on Wednesday, December 3rd. Olga |
Forward the initial email to private@pig.apache.org
for Pig PMC members to vote.
Once three PMC members have voted for a release and a majority vote to release and the required 3 days have passed, it may be published.
Tag the release:
svn move https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR \ https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z -m "Pig X.Y.Z release." |
Checkout Pig distribution svn.
svn co https://dist.apache.org/repos/dist/release/pig pig-dist |
Copy release files to the distribution directory add them to svn.
cd pig-dist cp -pr ~/release/pig-X.Y.Z-candidate-0 pig-X.Y.Z svn add pig-X.Y.Z |
The release directory usually contains just three releases, the most recent from three branches, with a link named 'latest' to the most recent recommended version,
svn rm pig-A.B.C svn rm latest link -s pig-X.Y.Z latest |
Commit .the distribution artifacts changes
svn commit --message "Pig X.Y.Z Release" |
Check if svn changes propagates to the staging directory
ssh people.apache.org cd /www/www.apache.org/dist/pig |
After several minutes, you should see your distribution changes here.
Prepare to edit the website.
svn co https://svn.apache.org/repos/asf/pig/site |
Copy in the release specific documentation
mkdir publish/docs/rX.Y.Z cp -pr <releasedir>/docs/* publish/docs/rX.Y.Z/ svn add publish/docs/rX.Y.Z |
Send announcements to the user and developer lists as well as (dev@pig.apache.org; user@pig.apache.org; announce@apache.org; general@hadoop.apache.org) once the site changes are visible. Note that emails sent to announce@apache.org
must be sent from your apache.org
email address and you must be subscribed to each of the lists.
[ANNOUNCE] Apache Pig X.Y.Z released The Pig team is happy to announce the Pig X.Y.Z release. Apache Pig provides a high-level data-flow language and execution framework for parallel computation on Hadoop clusters. More details about Pig can be found at http://pig.apache.org/. The highlights of this release are ... The details of the release can be found at http://pig.apache.org/releases.html. |
Update jdiff for next release (step 16 to 19).
svn co https://svn.apache.org/repos/asf/pig/trunk |
Open build.xml. Change this line:
<property name="jdiff.stable" value="X.Y-1.Z"/> To <property name="jdiff.stable" value="X.Y.Z"/> |
Copy jdiff comparison base to trunk
cp {releasedir}/src/docs/jdiff/pig_X.Y.Z.xml src/docs/jdiff |
svn add src/docs/jdiff/pig_X.Y.Z.xml svn remove src/docs/jdiff/pig_X.Y-1.Z.xml svn commit -m "Jdiff change for X.Y.Z" |
build.xml
in branch to be X.Y.N-SNAPSHOT, where N is one greater than the release being made.svn commit -m "Preparing for X.Y.N release development" |