...
The number of total edges | Hama-0.7.0 | Giraph-1.2.0 |
300M | 115 97 seconds | 130 seconds |
270M | 106 91 seconds | 124 seconds |
240M | 100 85 seconds | 113 seconds |
210M | 97 73 seconds | 104 seconds |
180M | 82 67 seconds | 89 seconds |
150M | 67 52 seconds | 79 seconds |
120M | 61 49 seconds | 72 seconds |
90M | 52 43 seconds | 61 seconds |
60M | 40 31 seconds | 50 seconds |
30M | 28 19 seconds | 40 seconds |
What are the major changes from the last release?
The major improvement changes are in the queue and messaging systems. We now use own outgoing/incoming message manager instead of using Java's built-in queues. It stores messages in serialized form in a set of bundles (or a single bundle) to reduce the memory usage and RPC overhead. Kryo serializer is used to serialize objects more quickly. Another important improvement is the enhanced graph package. Instead of sending each message individually, we package the messages per vertex and send a packaged message to their assigned destination nodes. The thread-pool executor service also used for each vertex computation. With this, we achieve better performance.https://lh6.googleusercontent.com/-C2a7xlwPLdo/VS4HBRzkt1I/AAAAAAAAE9U/VG5ShajJSyo/w795-h609-no/hama.png
Graph Package: PageRank Benchmarks 0.6.4 vs 0.7.0-SNAPSHOT
...