Building/Testing Hadoop under IDEA

Hadoop builds with Ant, but you can set it up to work under IDEA for testing and some iterative development. This does not take away the need to run Ant; you just run it side by side.

Before you begin

Building

build.webapps=build/classes/webapps

Creating a new module

Create a new IDEA module for Hadoop.

Source folders

build/src
conf
src/ant
src/contrib/streaming/src/java
src/core
src/examples
src/hdfs
src/mapred
src/native/src
src/tools

test source directories

src/test
build/test/src

Build Paths

Set these to the full path of where Hadoop's Ant build sticks things, such as :

Libraries

Setting up a test run

To run JUnit tests under the IDE, create a new test configuration pointing to the chosen tests.

-Dhadoop.log.dir=/home/user/hadoop-core/build/test/logs

JRockit users: consider editing conf/log4j.properties to

log4j.appender.console.layout.ConversionPattern=%-4r %-5p %c %x - %m%n

This may seem odd, but it eliminated deadlocks in the logging.

HadoopUnderIDEA (last edited 2009-09-20 23:54:21 by localhost)