...
This tutorial requires Hadoop 2.x already correctly installed. If you haven't done this yet, please follow the official documentation https://hadoop.apache.org/docs/stable/
Configuration
Only two properties which are resource manager address and default filesystem uri properties is essentially needed for Hama on YARN. The sample configuration is as follows:
| No Format |
|---|
<!-- Path to your hama-site.xml --> <configuration> <property> <name>yarn.resourcemanager.address</name> <value>'your resource manager address or hostname':'resource manager port'</value> </property> <property> <name>fs.default.name</name> <value>hdfs://'your default file system address or hostname':'default file system port'/</value> </property> </configuration> |
See also configuration page for advanced configurations of Hama.
...
| No Format |
|---|
$HAMA_HOME/bin/hama jar hama-yarn-0.7.0-SNAPSHOT.jar org.apache.hama.bsp.YarnSerializePrinting
|
You should see "Hello BSP Message" Messages which each container spawned in HDFS where you defined output path in your ternimal. If your application is success, you'll be able to got the following message.
| No Format |
|---|
INFO bsp.YARNBSPJobClient: Application has completed successfully. Breaking monitoring loop
Hello BSP from 1 of 4: cluster-0:16004
Hello BSP from 2 of 4: cluster-1:16006
Hello BSP from 3 of 4: cluster-0:16008
Hello BSP from 4 of 4: cluster-1:16010
Job Finished in 14.838 seconds
|
How to write a Hama-YARN job
...
Basically you just need the following code to submit a Hama-YARN job.
| No Format |
|---|
HamaConfiguration conf = new HamaConfiguration();
YARNBSPJob job = new YARNBSPJob(conf);
job.setBspClass(HelloBSP.class);
job.setJarByClass(HelloBSP.class);
job.setJobName("Serialize Printing");
job.setMemoryUsedPerTaskInMb(50);
job.setNumBspTask(2);
job.waitForCompletion(false);
|
...