Working with Hadoop under Eclipse
Here are instructions for setting up a development environment for Hadoop under the Eclipse IDE. Please feel free to make additions or modifications to this page. At worst, if you see anything wrong please let me, MarcoNicosia, know.
This document (currently) assumes you already have Eclipse downloaded, installed, and configured to your liking.
Download and install the Subversive plug-in
The easiest way to download and install is to use Eclipse's Update Manager. That process is well described one ![[WWW]](/wiki/modern/img/moin-www.png){kind=small}
the Subversive's site. Think to add the two update sites.
Associate the Hadoop Trunk Repository
Select File > New > Other...
Then SVN > Repository Location wizard
Use
http://svn.apache.org/repos/asf/hadoop/core/ as the Root URL. I set a custom label of "Hadoop".
The repository will show up under "SVN Repositories" Perspective (select "Open Perspective.")
Create a Project
From the SVN Repositories perspective:
Right-click Hadoop > "Trunk" and select "Find/Check Out As..."
Check out as a project configured using the New Project Wizard
Java Project
Project Name: "Hadoop"
Using Subversive with already checkouted projects
Refer to the Subversive FAQ.
Configuring Eclipse to build Hadoop
As of 28 March 2008 there is an ant task for generating the requisite Eclipse files (see HADOOP-1228). Follow these instructions to configure Eclipse:
Set up an ANT_HOME Classpath Variable in Eclipse Preferences. This is a global Eclipse setting so you only need to do this once.
Checkout Hadoop.
Run the eclipse-files ant target (right click build.xml, choose run as Ant Build and choose eclipse-files target)
Refresh the Eclipse project.
Ensure that the Java version used matches the version of the project (currently 1.6 is used). This could be selected for the project by going to Project > Preferences > Builders > Hadoop_Ant_Builders. Go to JRE tab and select an appropriate Java version.
Select Project | Build Project.
Manual Settings
If you want to build all of Hadoop in Eclipse then there are some DDL files used by the tests that need to compiled first. One strategy is to configure Eclipse to call part of the Ant script to build these, and have two build directories, one for the Ant script, the other for Eclipse, as you need to include the classes built by Ant on the Eclipse library path and circular references are forbidden.
In Eclipse, select Project -> Properties -> Java Build Path -> Source
Then ensure the following source directories are on the Java build path:
hadoop/src/examples hadoop/src/java hadoop/src/test
Then if you want to use the contrib directories as well:
hadoop/src/contrib/test hadoop/src/contrib/abacus/examples hadoop/src/contrib/abacus/src/java hadoop/src/contrib/data_join/src/join hadoop/src/contrib/hbase/src/java hadoop/src/contrib/hbase/src/test hadoop/src/contrib/streaming/src/java hadoop/src/contrib/streaming/src/test
and set the output folder to
hadoop/eclipse-build
Then select Project -> Properties -> Java Build Path -> Libraries
Add all the libraries (.jar files) in
hadoop/lib hadoop/lib/jetty-ext
If you are using contrib also add all the libraries (.jar files) in
hadoop/src/contrib/hbase/lib
Then add the classes in
hadoop/build/test/classes
Then select Project->Properties->Builders
Add a new Ant builder. Select the top level build.xml as the build file. Next select the "targets" tab, after clean specify
compile-core-classes, compile-core test
then after manual build specify
compile-core-classes, compile-core-tests, compile
Apply these changes. Hopefully Hadoop should now build successfully in Eclipse without any errors.