Architecture of Graph Module
Hama includes the Graph module for vertex-centric graph computations. Hama's Graph APIs allows you to program Google's Pregel style applications with simple programming interface.
Internals
The Graph APIs are implemented on top of Hama BSP framework. It consists of three major classes: VertexInputReader, GraphJob, and GraphJobRunner.
- VertexInputReader: it is used for parsing and extracting the Vertex structure from arbitrary text and binary data.
- GraphJob: the primary interface for a user to describe a Graph job to the Hama BSP framework for execution.
- GraphJobRunner: the BSP program for performing the Vertex's compute() method.
VertexInputReader
The VertexInputReader is the user-defined interface for parsing and extracting the Vertex structure from arbitrary text and binary data. Internally, the loadVertices() method reads the records from assigned split, and then loads the converted Vertex objects by the user-defined VertexInputReader.parseVertex() method into memory Vertices storage.
- TODO: disk-based vertices storage should be considered in the future.
GraphJob
GraphJob provides some additional Get/Set methods extending the core BSPJob interface for supporting the Graph specific configurations, such as setMaxIteration, setAggregatorClass, setVertexInputReaderClass, and setVertexOutputWriterClass. Rest APIs e.g., InputFormat, OutputFormat etc. are the same with core BSPJob interface.
GraphJobRunner
The GraphJobRunner is the BSP program which is performs assigned vertex computations and creates output.
LoadVertices
doInitialSuperstep and doSuperstep
Future Ideas and Challenges
- List of Future Ideas and Challenges.