...
This tutorial requires Hadoop 2.x already correctly installed. If you haven't done this yet, please follow the official documentation https://hadoop.apache.org/docs/stable/
How to
...
TODO this must be revised once the example has moved the jar.
| No Format |
|---|
bin/hama jar hama-yarn-0.7.0-SNAPSHOT.jar org.apache.hama.bsp.YarnSerializePrinting
|
Once running, you should see in the spawned application master log that it is launching containers. When the containers launched, you can see in the logs that there is a little "Hello World" from the other tasks.
How to write a Hama-YARN job
The BSPModel hasn't changed, but the way to submit a job has.
...
| No Format |
|---|
HamaConfiguration conf = new HamaConfiguration();
conf.set("yarn.resourcemanager.address", "0.0.0.0:8040");
YARNBSPJob job = new YARNBSPJob(conf);
job.setBspClass(HelloBSP.class);
job.setJarByClass(HelloBSP.class);
job.setJobName("Serialize Printing");
job.setMemoryUsedPerTaskInMb(50);
job.setNumBspTask(2);
job.waitForCompletion(false);
|
...
There are some configuration values that the job needs to have in order to submit sucessfully to YARN infrastructure.
The importantest configuration is configurations are the yarn.resourcemanager.address and fs.default.name. This should point to the address (hostname+port) where your ResourceManager and HDFS Namonode runs, for example localhost:8040, hdfs://localhost:9000.
Another important configuration value is the amount of memory used by the BSPApplicationMaster. You can configure a base amount of memory for the application master with this configuration key
...
to submit a Hama job. You can just change the BSPJob to YARNBSPJob.
Hama-YARN example
TODO this must be revised once the example has moved the jar.
| No Format |
|---|
bin/hama jar hama-yarn-0.7.0-SNAPSHOT.jar org.apache.hama.bsp.YarnSerializePrinting
|
Once running, you should see in the spawned application master log that it is launching containers. When the containers launched, you can see in the logs that there is a little "Hello World" from the other tasks.