| Table of Contents |
|---|
...
Design of Graph Module
Hama includes the Graph module for vertex-centric graph computations. Hama's Graph APIs allows you to program Google's Pregel style applications with simple programming interface.
...
The VertexInputReader is the user-defined interface for parsing and extracting the Vertex structure from arbitrary text and binary data. Internally, the loadVertices() method reads the records from assigned split, and then loads the converted Vertex objects by the user-defined VertexInputReader.parseVertex() method into memory Vertices storage.
...
storage
...
.
GraphJob
GraphJob provides some additional Get/Set methods extending the core BSPJob interface for supporting the Graph specific configurations, such as setMaxIteration, setAggregatorClass, setVertexInputReaderClass, and setVertexOutputWriterClass. Rest APIs e.g., InputFormat, OutputFormat etc. are the same with core BSPJob interface.
...
- setup() phase: the initialization phase for vertex computations.
- bsp() phase: the main computations of the vertices. The message communications among vertices are also handled by BSP communication interface in this phase.
- cleanup() phase: output write phase after completing the computations of the vertices.
More specifically, below two methods LoadVertices and doSuperstep() are used for loading and processing vertices.
loadVertices
As you can guess, the loadVertices() is in the setup() initialization phase. It reads assigned split data, parses and loads Vertex into VerticesInfo. The current implementation of Vertex computations assumes that Vertices are already sorted by vertexID, for processing memory-efficiently.
doInitialSuperstep and doSuperstep
List of Future Ideas and Challenges
- Currently, we use ListVerticesInfo and Collections.sort(vertices). It need to be improved.
- With this, HBase's Scanner or disk-based vertices storage should be considered in the future
- .