This effort is still a "work in progress". Please feel free to add comments.
Introduction
HAMA is a distributed framework on Hadoop for massive matrix and graph computations, currently being incubated as one of the incubator project by the Apache Software Foundation.
Goal
The Hama project goal is to provide easy matrix/graph computing programming environment on the Hadoop (distributed system). We are focusing on are as follows:
- Compatibility
- Scalability
- Flexibility
- Usability and Applicability
The overall architecture of HAMA
Below diagram is illustrates the overall architecture of HAMA.
+--------------------------------------+ | Matrix/Graph Computing Program | User Applications +--------------------------------------+ +------------------------------------------+ | HAMA : BSP, ............, etc | Computing Engines +------------------------------------------+ +----------------------------------------------------+ | ZooKeeper | Distributed Locking Service +----------------------------------------------------+ +----------------------------------------------------+ | Hadoop : HDFS, HBase, ..., etc | Distributed Storage Systems +----------------------------------------------------+
BSP framework
- TODO: explain what is the BSP? and Hama BSP?
BSP Architecture/Design
Job Manager System
- work in progress
Hama Examples
- TODO: explain how to programming BSP program with Hama.
- work in progress HAMA-244
// BSP job configuration BSPJob bsp = new BSPJob(); // Set the job name bsp.setJobName("BSP test job"); // Set in/output path and formatter bsp.setInputPath(conf, new Path("input path")); bsp.setOutputPath(conf, new Path("output path")); bsp.setInputFormat(MyInputFormat.class); bsp.setOutputFormat(MyOutputFormat.class); // Set the BSP code bsp.setBSPCode(BSPProgram.class); BSPJobClient.runJob(bsp);
- Matrix Inversion
- Pi Estimator
- PageRank
Shell/DSL
- Hama DSL (Domain Specific Language) in Groovy – Work in progress
- Hama Shell – Work in progress