How to Contribute to Solr

"Contributing" to an Apache project is about more then just writing code -- it's about doing what you can to make the project better. There are lots of ways to contribute....

Be Involved

Contributors should join the Solr mailing lists. In particular:

Please keep discussions about Solr on list so that everyone benefits. Emailing individual committers with questions about specific Solr issues is discouraged. See http://people.apache.org/~hossman/#private_q.

Be A Mailing List Moderator

Being a list moderator is incredibly easy - the basic responsibilities are:

If you'd like to volunteer to be the moderator of a mailing list, just contact listname-owner@lucene... (ie: solr-user-owner@lucene...)

Write/Improve User Documentation

Solr can always use more/better documentation targeted at end users. The Reference Guide is the official documentation, and only committers can modify it, but anyone can comment on it. There is quite a lot of additional documentation in this wiki, which anyone can edit after creating an account and asking for edit permissions via the solr-user mailing list or the #solr IRC channel on freenode.

If you see a gap or problem in the Reference Guide, comment on the page to bring it to the attention of a committer. If you see a gap or problem in the documentation on this wiki, fill it in. Even if you don't know exactly what to say, ask on the user list and you'll probably get a lot of great responses -- talking informally about how Solr works is something lots of people tend to have time for, but aggregating all of that info into concise cohesive documentation takes a little more work and patience.

If there is a patch in Jira that you think is really great, writing some "user guide" style docs about how it works (or is suppose to work) in this wiki is a great way to help the patch get committed: It helps serve as a road map for what the "goal" of the issue is, what should be possible for users to do once the issue is resolved; it helps get people who may not understand the low level details get excited about the new functionality; and it can eventually evolve into the final documentation once the code is committed. (just make sure to link to the issue so people who find your wiki page first know it's not included in Solr's main code line yet).

Contributing Code (Features, Bug Fixes, Tests, etc...)

This section identifies the optimal steps community member can take to submit a changes or additions to the Solr code base. This can be new features, bug fixes optimizations of existing features, or tests of existing code to prove it works as advertised (and to make it more robust against possible future changes).

Please note that these are the "optimal" steps, and community members that don't have the time or resources to do everything outlined on this below should not be discouraged from submitting their ideas "as is" per "Yonik's Law of Patches" ...

A half-baked patch in Jira, with no documentation, no tests
and no backwards compatibility is better than no patch at all.

Just because you may not have the time to write unit tests, or cleanup backwards compatibility issues, or add documentation, doesn't mean other people don't. Putting your patch out there allows other people to try it and possibly improve it.

Getting the source code

First of all, you need the Solr source code.

Get the source code on your local drive using SVN.

To check out code from SVN

for non-committers:
svn checkout http://svn.apache.org/repos/asf/lucene/dev/<branch>

for committers
svn checkout https://svn.apache.org/repos/asf/lucene/dev/<branch>

The <branch> part of the command above needs to be replaced by something concrete - the "code line" you want to get. Examples, and how to interpret what they mean:

SVN revision numbers can be used to obtain branches and tags from their parents. For example, revision 1394844 on branches/lucene_solr_4_0 corresponds to the official 4.0.0 release, so if you add "-r 1394844" to the command line above using branches/lucene_solr_4_0 for <branch>, you will get the code corresponding to 4.0.0. You can also just do a checkout of tags/lucene_solr_4_0_0 to get the same code.

Most development is done on "trunk" and then backported to the other active branches. At some point in the future, a branch_5x will be created and trunk will be updated to reflect a 6.0 version. Shortly after that, branch_4x will deleted and bugfixes for 4.x versions will continue on the newest lucene_solr_4_x branch.

Note that committers have to use https instead of http here, but http is fine for read-only access to the code.

If you prefer you could use the GitHub mirror instead. Note that the drop-down lets you select "trunk" or "branch_4x" (among others).

Exporting to a local repository

When making larger scale (please, try to keep your patches as small as humanly possible) changes, or when working in a team, you might want to be able to keep track of what you are doing locally. One way of doing so, is exporting to a local SVN repository.

A proposed strategy for switching/exporting to a local SVN repository;

* by doing a unix diff, you keep any local version out of the patch file.

Making Changes

Before you start, you should send a message to the Solr developer mailing list (Note: you have to subscribe before you can post), or file a bug in Jira. Describe your proposed changes and check that they fit in with what others are doing and have planned for the project. Be patient, it may take folks a while to understand your requirements.

Modify the source code and add some (very) nice features using your favorite IDE.

But take care about the following points

Notes for Eclipse

To get correct classpath, formatting, encoding, and project settings in Eclipse, simply run ant eclipse and then reload the project in Eclipse. Make sure you are using Java 7 (1.7.x) or later.

Generating a patch

A "patch file" is the format that all good contributions come in. It bundles up everything that is being added, removed, or changed in your contribution.

Unit Tests

Please make sure that all unit tests succeed before constructing your patch.

> cd solr-trunk
> ant clean test

After a while, if you see

BUILD SUCCESSFUL

all is ok. but if you see

BUILD FAILED

please, read carefully the errors messages and check your code. If the test fails you may want to repeatedly rerun a single test as you debug and sort out any problems. In which case you could run

> ant -Dtestcase=TestXXX test

Where "TestXXX" is the name of the particular Junit test you want to run. NOTE: specifying -Dtestcase= must be done at least at the <home>/lucene or <home>/solr level. Executing this at the root level will not run tests.

Frequently failing Tests

There are some tests that fail sometimes on some systems, but run on Jenkins fine. It's always a good idea to be sure you can run "ant test" successfully before you start making code changes. Or keep an un-changed version of the code around to see if your changes are really to blame.

One of the great things about Open Source is so many people run the tests on so many different systems. Occasionally you'll be the lucky person who has the system that wins the prize by having the environment that exposes a new failure mode, see the discussion at SOLR-3846 for an example.

If you do find one of these, here's what you should do:

Pre-commit check

Before constructing your patch, please run the top-level pre-commit check, which finds problems like tabs and @author tags in source files, broken links in javadocs, files not controlled by Subversion (a.k.a. "unversioned files"), etc.

To run the pre-commit checks from ant, run the following from the top-level directory -- the directory containing lucene/ and solr/ -- in your working copy:

ant precommit

Creating the patch file

Check to see what files you have modified with:

svn stat

Add any new files with:

svn add src/.../MyNewClass.java

Subversions "add" command only modifies your local copy, so it does not require commit permissions. By using "svn add", your entire contribution can be included in a single patch file, without needing to submit a separate set of "new" files.

If you have a lot of new files you can do "svn add" for all new files in a single command (on "real" OS's only)

svn stat | grep "^?" | awk '{print $2}' | xargs svn add

Edit the CHANGES.txt file, adding a description of your change, including the bug number it fixes.

In order to create a patch, just type:

svn diff > SOLR-NNNN.patch

This will report all modifications done on Solr sources on your local disk and save them into the SOLR-NNN.patch file. Read the patch file. Make sure it includes ONLY the modifications required to fix a single issue.

Note the SOLR-NNN.patch patch file name. Please use this naming pattern when creating patches for uploading to JIRA. Once you create a new JIRA issue, note its name and use that name when naming your patch file. For example, if you are creating a patch for a JIRA issue named SOLR-123, then name your patch filename SOLR-123.patch. If you are creating a new version of an existing patch, use the existing patch's file name. JIRA will automatically "gray out" the old patch and clearly mark your newly uploaded patch as the latest.

Please do not:

Please do:

Contributing your work

Finally, patches should be attached to a bug report in Jira. If you are revising an existing patch, please re-use the exact same name as the previous attachment, Jira will "gray out" the older versions so it's clear which version is the newest.

Please be patient. Committers are busy people too. If no one responds to your patch after a few days, please make friendly reminders. Please incorporate others suggestions into your patch if you think they're reasonable. Remember that even a patch that is not committed is useful to the community. Supply first patch as early as possible and updated patches as often as possible during your work. This helps the rest of the community and committers to start understanding, help shaping, commenting on etc. your work throughout the entire process. Supplying a patch does not necessarily mean that it is complete and ready to be committed, it might also just be a way of communicating your idea and progress.

JIRA tips (our issue/bug tracker)

The issue tracker we use is a JIRA instance at https://issues.apache.org/jira/browse/SOLR. If you don't yet have an account, just click the "login" link and you'll get the opportunity to create one that will allow you to add tickets, upload patches etc.

Review/Improve Existing Patches

If there's a Jira issue that already has a patch you think is really good, and works well for you -- please add a comment saying so. If there's room for improvement (more tests, better javadocs, etc...) then make the changes and attach it as well. If a lot of people review a patch and give it a thumbs up, that's a good sign for committers when deciding if it's worth spending time on the patch -- and if other people have already put in effort to improve the docs/tests for a patch, that helps even more.

Working With Patches

You can easily download a patch from JIRA and test it by doing the following:

$ cd <your Solr trunk checkout dir>
$ svn up
$ wget <URL of the patch>
$ patch -p0 -i name of the patch --dry-run

(note: --dry-run just pretends to apply a patch, so you can see if it would succeed or fail. Remove --dry-run to *really* apply the patch)

The address for the patch can be obtained from the issue page, under the "File Attachments" section of the issue.

For people who like one-liners, The following should work as well:

$ cd <your Solr trunk checkout dir>
$ svn up
$ wget <URL to the patch> -O - | patch -p0 --dry-run

If you are on Solaris, you should replace 'patch' with 'gpatch' to use GNU Patch instead.

If the patch is created using Git, it has another format, which can be applied using -p1:

patch -p1 -i name of the git formatted patch --dry-run

Reverting to pre-patch state is one line:

svn revert -R .

Though this leaves added files, which can be removed with

svn st | grep '?' | awk '{print $2}' | xargs rm

Another useful trick is to have multiple checkouts of trunk and "bounce" an active changeset from one to another with

svn diff | (cd ../otherbranch; patch -p0)

Helpful Resources

The following resources may prove helpful when developing Solr contributions. (These are not an endorsement of any specific development tools, but Eclipse and IntelliJ seem to be the most popular)

<!> Solr1.3 If you are using eclipse to follow trunk (leading up to the 1.3 release) eclipse will give several errors about not resolving components in the solrj library. This will appear in the org.apache.solr.handler.component package relating to distributed search (sharedrequest.java ...etc) The solution is to compile the solrj library via the dist-solrj target and add them to your eclipse build path. After running the dist-solrj target look in dist/solrj-lib and add apache-solr-solrj-1.3-dev.jar and commons-httpclient-3.1.jar to your buildpath.

Development Environment Tips

Here is a guide for setting up Eclipse, IntelliJ and Netbeans dev environments:

Follow the instructions above to fetch the combined Lucene and SOLR trunk. For the remainder of this document, the installation path is assumed to be ~/apache/trunk/lucene and ~/apache/trunk/solr. NOTE: I'm installing on a Macbook, so this is a *nix style file system etc. These instructions should work for windows as well, but if you try to use them in that environment, feel free to update this page with anything you uncover.

Before fiddling with the IDE, I'd strongly recommend you get the tests to run from the shell. This will insure that your machine has the proper setup for the IDEs to magically find what's necessary. See the instructions above. Hint: Issue 'ant clean test' in the SOLR and Lucene directories and look for "BUILD SUCCESSFUL" minutes later.

Setting things up is actually very smooth when it's smooth, especially if the tests have run <G>.

DO NOT BE SURPRISED IF SOME TESTS FAIL IN THE IDE. There are some anomalies when running Junit tests for these projects in an IDE. Some of them are already cleaned up, but others may still fail when run in an IDE. The definitive case for whether a test fails or not is running it as an Ant task.

Eclipse

See wiki page on configuring Eclipse.

IntelliJ (9.X-13.X)

See the Lucene wiki page on configuring IntelliJ - it also covers Solr configuration.

Netbeans

See the Lucene wiki page on configuring Netbeans.

Maven

There is information about using Maven with Solr and Lucene in the source tree, at dev-tools/maven/README.maven. The information differs slightly by code branch. Here are links to this file for each Solr version:

Solr Version

Link to Lucene/Solr Maven build instructions

4.6

http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_6/dev-tools/maven/README.maven

trunk/5.0 (unreleased)

http://svn.apache.org/repos/asf/lucene/dev/trunk/dev-tools/maven/README.maven

Getting your feet wet: where to begin?

New to Solr? Want to find JIRA issues that you can work on without taking on the whole world?

The Solr/Lucene developers use the "newdev" label to mark issues that developers new to Solr might be interested in working on. The rough criteria used to make this selection are:

To see a list of open Solr and Lucene issues with the newdev label, look at this link http://s.apache.org/newdevlucenesolr

Note: Fixing these issues may require asking questions on the developer list to figure out what they mean - there is no guarantee that any of these will be either quick or easy.

Parsers and JFlex

YOU PROBABLY DON'T HAVE TO DO THIS but if you do for some reason want to regenerate the .java from the .jflex files, here's the cheat-sheet. This is a separate step that people who actually like parsers sometimes have to execute to reflect changes in various grammars.

(1) I was getting OOM errors when running "ant jflex", and got a tip to do this, which made that problem go away. You may not have to be so generous with memory, but this worked... (Thanks Steve!).

One final note

As always, there are gremlins out there. If you find problems with the information here, and especially if you subsequently find solutions to the problems you find, please either write to the solr-user mailing list or update this page directly so others can benefit.

HowToContribute (last edited 2014-12-19 21:16:21 by ErickErickson)