Quickly Launch A Hama Cluster In Cloud Using Whirr
The following commands install Whirr and start a 5 node Hama cluster on Amazon EC2 in 5 minutes or less.
| No Format |
|---|
% curl -O http://www.apache.org/dist/whirr/whirr-0.x.0/whirr-0.x.0.tar.gz % tar zxf whirr-0.x.0.tar.gz; cd whirr-0.x.0 % export AWS_ACCESS_KEY_ID=YOUR_ID % export AWS_SECRET_ACCESS_KEY=YOUR_SECKEY % ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa_whirr // Launch a Hama cluster on EC2 % bin/whirr launch-cluster --config recipes/hama-ec2.properties --private -key-file ~/.ssh/id_rsa_whirr |
Upon success you should see imok echoed to the console, indicating that ZooKeeper is running.
Now you can run an BSP examples as below:
| No Format |
|---|
edward@domU-12-31-39-0C-7D-41:/usr/local/hama-0.3.0-incubating$ bin/hama jar hama-examples-0.3.0-incubating.jar
An example program must be given as the first argument.
Valid program names are:
bench: Random Communication Benchmark
pagerank: PageRank
pi: Pi Estimator
sssp: Single Source Shortest Path
test: Serialize Printing Test
edward@domU-12-31-39-0C-7D-41:/usr/local/hama-0.3.0-incubating$ bin/hama jar hama-examples-0.3.0-incubating.jar pi
11/12/25 11:48:11 INFO bsp.BSPJobClient: Running job: job_201112251143_0001
11/12/25 11:48:14 INFO bsp.BSPJobClient: Current supersteps number: 0
11/12/25 11:48:17 INFO bsp.BSPJobClient: Current supersteps number: 1
11/12/25 11:48:20 INFO bsp.BSPJobClient: The total number of supersteps: 1
Estimated value of PI is 3.147866666666667
Job Finished in 9.635 seconds
|
And shut down the cluster with:
| No Format |
|---|
... Namenode web UI available at http://ec2-184-73-29-237.compute-1.amazonaws.com:50070 Jobtracker web UI available at http://ec2-184-73-29-237.compute-1.amazonaws.com:50030 Wrote Hadoop site file /home/edward/.whirr/hama/hadoop-site.xml Wrote Hadoop proxy script /home/edward/.whirr/hama/hadoop-proxy.sh Completed configuration of hama BSPMaster web UI available at http://ec2-184-73-29-237.compute-1.amazonaws.com:40013 Wrote Hama site file /home/edward/.whirr/hama/hama-site.xml Wrote Hama proxy script /home/edward/.whirr/hama/hama-proxy.sh Wrote instances file /home/edward/.whirr/hama/instances Started cluster of 6 instances ... // Destroy % bin/whirr destroy-cluster --config recipes/hama-ec2.properties |
The various options are explained in more detail in the Whirr website