Short-term Issues
Re-factoring issues
- Move current code related to matrix operations to the 'examples' package
- A design of structure of the matrix/graph
Long-term Issues
We have a plan to redesign Hama to be based on BSP model and be specified to shared nothing systems consisting of several thousands commodity servers, which is generally called cloud computing environments.
Why BSP?
(Working)
Design Considerations
- Fault Tolerance - Hama aims at running on a several thousands of commodity servers, so it is subject to some fault. In addition, Hama is for large-scale processing that generally takes long time ranging from few minutes to several hours. Therefore, it is important for Hama to finish some given jobs although faults occur during processing. If not, Hama has to restart all jobs.
- Heterogeneity -
- Efficiency
- Easy to Use
(Working)
TODO
- A survey on matrix and graph processing algorithms based on BSP programming model.
- Developing a fault-tolerant mechanism for BSP model.
- Developing a struggling mechanism for BSP model.
- Implement BSP frameworks based on the source code that we have done.
- A selection of the primitive operations for matrix processing and linear algebra.
- Implement the primitive operations for matrix and linear algebra.
- Develop operation models based on the primitive operations developed above.
- Implement processing framework for matrix and linear algebra.
- Design domain-specific language that well reflects to algebraic characteristics.
- A selection of the primitive operations for graph processing.
- Develop operation models based on the above primitive operation for large-scale graph processing.
(Working)
Idea Generating and Research Tasks
This section deals with the types of tasks that includes overall architecture/performance related abstract issues. – edyoon
- Regarding in-memory/cache operations (e.g., a single reducer), are there any possiblilies to accelate the performance? – smiler
- As an alternative, utilize hardware supports on Hama, and do empirical works.