This effort is still a "work in progress". Please feel free to add comments. BRBut please make the content less visible by using smaller fonts. – Edward J. Yoon
Overview
Hama use a [http://hadoop.apache.org/hbase/ Hbase] to store the matrices – Matrices are basically tables. They are ways of storing numbers and other things. Typical matrix has rows and columns. Actually called a 2-way matrix because it has two dimensions. For example, you might have respondents-by-attitudes. Of course, you might collect the same data on the same people at 5 points in time. In that case, you either have 5 different 2-way matrices, or you could think of it as a 3-way matrix, that is respondent-by-attitude-by-time.
Dense Matrix
For dense matrix computations, The block-partitioned algorithms used to minimize data movement and network cost. Dense Matrix and Blocked Dense Matrix are both stored in one table with other metadata. But, Blocked dense matrix can't be synchronized by dense matrix update.
// Generate matrix with random elements DenseMatrix a = DenseMatrix.random(conf, 1000, 1000); DenseMatrix b = DenseMatrix.random(conf, 1000, 1000); // The type of the Matrix to be blocked, must be dense. a.blocking(2); b.blocking(2); DenseMatrix c = a.mult(b);
For example, The matrix multiplication of the original arrays can be transformed into matrix multiplication of blocks as describe below.
C_block(1,1)=A_block(1,1)*B_block(1,1) + A_block(1,2)*B_block(2,1) C A B +-----+-----+ +-----+-----+ +-----+-----+ | x x | | | --> | --> | | | | | | | x x | | | --> | --> | | ↓ ↓ | | +-----+-----+ = +-----+-----+ * +-----+-----+ | | | | | | | | | | | | | | | | | | ↓ ↓ | | +-----+-----+ +-----+-----+ +-----+-----+
– To be statically sized blocks, What should we do? – Edward