You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 456 Next »

This effort is still a "work in progress". Please feel free to add comments.

Introduction

HAMA is a distributed framework on Hadoop for massive matrix and graph computations, currently being incubated as one of the incubator project by the Apache Software Foundation.

Goal

The Hama project goal is to provide easy matrix/graph computing programming environment on the Hadoop (distributed system). We are focusing on are as follows:

  • Compatibility
  • Scalability
  • Flexibility
  • Usability and Applicability

The overall architecture of HAMA

Below diagram is illustrates the overall architecture of HAMA.

               +--------------------------------------+
               |    Matrix/Graph Computing Program    |  User Applications
               +--------------------------------------+
           +------------------------------------------+
           |       HAMA : BSP, ............, etc      |  Computing Engines
           +------------------------------------------+ 
 +----------------------------------------------------+
 |                     ZooKeeper                      |  Distributed Locking Service
 +----------------------------------------------------+
 +----------------------------------------------------+
 |           Hadoop : HDFS, HBase, ..., etc           |  Distributed Storage Systems
 +----------------------------------------------------+

BSP framework

  • TODO: explain what is the BSP? and Hama BSP?

BSP Architecture/Design

Job Manager System

  • work in progress

BSP Example

  • TODO: explain how to programming BSP program.

BSP Job Configuration

    // BSP job configuration 
    BSPJob bsp = new BSPJob(); 
    // Set the job name 
    bsp.setJobName("BSP test job"); 

    // Set in/output path and formatter 
    bsp.setInputPath(conf, new Path("input path")); 
    bsp.setOutputPath(conf, new Path("output path")); 
    bsp.setInputFormat(MyInputFormat.class); 
    bsp.setOutputFormat(MyOutputFormat.class); 
     
    // Set the BSP code 
    bsp.setBSPCode(BSPProgram.class); 
    BSPJobClient.runJob(bsp);

Shell/DSL

  • Hama DSL (Domain Specific Language) in Groovy – Work in progress
  • Hama Shell – Work in progress
  • No labels