...
You can implement your own outputformat. It is similar to Hadoop MapReduce's output formats, so you can use existing literature to get into it.
Implementation notes
Internal implementation details
BSPJobClient
- Create the splits for the job 2. writeNewSplits() 3. job.set("bsp.job.split.file", submitSplitFile.toString()); 4. Sets the number of peers to split.lenth
- Receives splitFile 2. Add split argument to TaskInProgress constructor
Task
- Gets his split from Groom 2. Initializes everything in BSPPeerImpl
Communication Model
Within the bsp() function, you can use the powerful communication functions for many purposes using BSPPeer. We tried to follow the standard library of BSP world as much as possible. The following table describes all the functions you can use:
Function | Description |
send(String peerName, BSPMessage msg) | Send a message to another peer. |
getCurrentMessage() | Get a received message from the queue. |
getNumCurrentMessages() | Get the number of messages currently in the queue. |
sync() | Starts the barrier synchronization. |
getPeerName() | Get the peer name of this task. |
getPeerName(int index) | Gets the n-th peer name. |
getNumPeers() | Get the number of peers. |
getAllPeerNames() | Get all peer names (including "this" task). (Hint: These are always sorted in ascending order) |
Communication Model
Within the bsp() function, you can use the powerful communication functions for many purposes using BSPPeer. We tried to follow the standard library of BSP world as much as possible. The following table describes all the functions you can use:
Function | Description |
send(String peerName, BSPMessage msg) | Send a message to another peer. |
getCurrentMessage() | Get a received message from the queue. |
getNumCurrentMessages() | Get the number of messages currently in the queue. |
sync() | Starts the barrier synchronization. |
getPeerName() | Get the peer name of this task. |
getPeerName(int index) | Gets the n-th peer name. |
getNumPeers() | Get the number of peers. |
getAllPeerNames() | Get all peer names (including "this" task). (Hint: These are always sorted in ascending order) |
The send() The send() and all the other functions are very flexible. Here is an example that sends a message to all peers:
...
| No Format |
|---|
public static class SumCombiner extends Combiner {
@Override
public BSPMessageBundle combine(Iterable<BSPMessage> messages) {
BSPMessageBundle bundle = new BSPMessageBundle();
int sum = 0;
Iterator<BSPMessage> it = messages.iterator();
while (it.hasNext()) {
sum += ((IntegerMessage) it.next()).getData();
}
bundle.addMessage(new IntegerMessage("Sum", sum));
return bundle;
}
}
|
Implementation notes
Internal implementation details
BSPJobClient
- Create the splits for the job 2. writeNewSplits() 3. job.set("bsp.job.split.file", submitSplitFile.toString()); 4. Sets the number of peers to split.lenth
- Receives splitFile 2. Add split argument to TaskInProgress constructor
Task
- Gets his split from Groom 2. Initializes everything in BSPPeerImpl