How to Contribute to Apache Hive
This page describes the mechanics of how to contribute software to Apache Hive. For ideas about what you might contribute, please see open tickets in Jira.
Contents
Getting the source code
First of all, you need the Hive source code.
Get the source code on your local drive using SVN. Most development is done on the "trunk":
svn checkout http://svn.apache.org/repos/asf/hadoop/hive/trunk hive-trunk
Setting up Eclipse Development Environment (Optional)
This is an optional step. Eclipse has a lot of advanced features for Java development, and it makes the life much easier for Hive developers as well.
How to set up Eclipse for Hive development
Making Changes
Before you start, send a message to the Hive developer mailing list, or file a bug report in Jira. Describe your proposed changes and check that they fit in with what others are doing and have planned for the project. Be patient, it may take folks a while to understand your requirements.
Modify the source code and add some (very) nice features using your favorite IDE.
But take care about the following points
All public classes and methods should have informative Javadoc comments.
- Do not use @author tags.
Code should be formatted according to Sun's conventions, with one exception:
- Indent two spaces per level, not four.
- Contributions should pass existing unit tests.
New unit tests should be provided to demonstrate bugs and fixes. JUnit is our test framework:
You must implement a class that extends junit.framework.TestCase and whose class name starts with Test.
Define methods within your class whose names begin with test, and call JUnit's many assert methods to verify conditions; these methods will be executed when you run ant test.
You can run all the unit test with the command ant test, or you can run a specific unit test with the command ant -Dtestcase=<class name without package prefix> test (for example ant -Dtestcase=TestFileSystem test)
Understanding Ant
Hive is built by Ant, a Java building tool.
Good Ant tutorial: http://i-proving.ca/space/Technologies/Ant+Tutorial
Unit Tests
Please make sure that all unit tests succeed before and after applying your patch and that no new javac compiler warnings are introduced by your patch. You can specify the hadoop version with -Dhadoop.version="<your-hadoop-version>" if your hadoop version is not the default.
> cd hive-trunk > ant clean test tar -logfile ant.log
After a while, if you see
BUILD SUCCESSFUL
all is ok, but if you see
BUILD FAILED
then you should fix things before proceeding. Running
> ant testreport
and examining the HTML report in build/test might be helpful.
Add a Unit Test
There are two kinds of unit tests in Hive:
- Normal unit test: These are used by testing a particular component of Hive.
- A new query: If the new feature can be tested using Hive command line, we just need to add a new *.q file and a new *.q.out file:
- If the feature is added in ql
- Add a new XXXXXX.q file in ql/src/test/queries/clientpositive
Run "ant test -Dtestcase=TestCliDriver -Dqfile=XXXXXX.q -Doverwrite=true -Dtest.silent=false". This will generate a new XXXXXX.q.out file in ql/src/test/results/clientpositive.
- If the feature is added in contrib
Do the steps above, replacing "ql" with "contrib", and "TestCliDriver" with "TestContribCliDriver".
- If the feature is added in ql
Debugging
Please see Debugging Hive code in Development Guide.
Creating a patch
Check to see what files you have modified with:
svn stat
Add any new files with:
svn add .../MyNewClass.java svn add .../TestMyNewClass.java svn add .../XXXXXX.q svn add .../XXXXXX.q.out
In order to create a patch, type (from the base directory of hive):
svn diff > HIVE-1234.patch
This will report all modifications done on Hive sources on your local disk and save them into the HIVE-1234.patch file. Read the patch file. Make sure it includes ONLY the modifications required to fix a single issue.
Please do not:
- reformat code unrelated to the bug being fixed: formatting changes should be separate patches/commits.
- comment out code that is now obsolete: just remove it.
- insert comments around each change, marking the change: folks can use subversion to figure out what's changed and by whom.
- make things public which are not required by end users.
Please do:
- try to adhere to the coding style of files you edit;
- comment code whose function or rationale is not obvious;
update documentation (e.g., package.html files, this wiki, etc.)
If you need to rename files in your patch:
- Write a shell script that uses 'svn mv' to rename the original files.
- Edit files as needed (e.g., to change package names).
- Create a patch file with 'svn diff --no-diff-deleted --notice-ancestry'.
- Submit both the shell script and the patch file.
This way other developers can preview your change by running the script and then applying the patch.
Applying a patch
To apply a patch either you generated or found from JIRA, you can issue
patch -p0 < cool_patch.patch
if you just want to check whether the patch applies you can run patch with --dry-run option
patch -p0 --dry-run < cool_patch.patch
If you are an Eclipse user, you can apply a patch by : 1. Right click project name in Package Explorer , 2. Team -> Apply Patch
Contributing your work
Finally, patches should be attached to an issue report in Jira via the Attach File link on the issue's Jira. Please add a comment that asks for a code review following our code review checklist. Please note that the attachment should be granted license to ASF for inclusion in ASF works (as per the Apache License ยง5).
When you believe that your patch is ready to be committed, select the Submit Patch link on the issue's Jira.
Folks should run ant clean test before selecting Submit Patch. Tests should all pass. If your patch involves performance optimizations, they should be validated by benchmarks that demonstrate an improvement.
If your patch creates an incompatibility with the latest major release, then you must set the Incompatible change flag on the issue's Jira 'and' fill in the Release Note field with an explanation of the impact of the incompatibility and the necessary steps users must take.
If your patch implements a major feature or improvement, then you must fill in the Release Note field on the issue's Jira with an explanation of the feature that will be comprehensible by the end user.
A committer should evaluate the patch within a few days and either: commit it; or reject it with an explanation.
Please be patient. Committers are busy people too. If no one responds to your patch after a few days, please make friendly reminders. Please incorporate other's suggestions into your patch if you think they're reasonable. Finally, remember that even a patch that is not committed is useful to the community.
Should your patch receive a "-1" select the Resume Progress on the issue's Jira, upload a new patch with necessary fixes, and then select the Submit Patch link again.
Committers: for non-trivial changes, it is best to get another committer to review your patches before commit. Use Submit Patch link like other contributors, and then wait for a "+1" from another committer before committing. Please also try to frequently review things in the patch queue.
Jira Guidelines
Please comment on issues in Jira, making their concerns known. Please also vote for issues that are a high priority for you.
Please refrain from editing descriptions and comments if possible, as edits spam the mailing list and clutter Jira's "All" display, which is otherwise very useful. Instead, preview descriptions and comments using the preview button (on the right) before posting them. Keep descriptions brief and save more elaborate proposals for comments, since descriptions are included in Jira's automatically sent messages. If you change your mind, note this in a new comment, rather than editing an older comment. The issue should preserve this history of the discussion.
Stay involved
Contributors should join the Hive mailing lists. In particular the dev list (to join discussions of changes) and the user list (to help others).