DataProcessingBenchmarks

1. Hadoop Map/Reduce Data Processing Benchmarks
   1. Group/Sort
      1. MapReduce Flow
      2. Benchmarks
2. Hbase Matrix computations Benchmarks
   1. MapReduce Flow

---

• Reporter : Edward Yoon

Hadoop Map/Reduce Data Processing Benchmarks

Group/Sort

• Finds the most connected networks.

SQL > select ipaddress, count(*) from access_log group by ipaddress order by count(*) desc limit 0,100;
σ _{count. ipaddress} (τ _count (γ _{count(ipaddress). ipaddress} (access_log)))

MapReduce Flow

• Map was used for extract the IP address of the client requesting the web page.

• Reduce was used for summation.

• 1 more Map/Reduce was used for sort by count.

Benchmarks

1.5 GB access_log on 10 node cluster

This test should include the data load time for the MySql column, not just the SQL time.

---

Hbase Matrix computations Benchmarks

• You can download the Hbase Matrix Package for Map/Reduce-based Parallel Matrix Computations (still under development)

MapReduce Flow

• The Multiplication requires (n + 1) table full scan irrespective of the number of mapper.

• Each map processor requires O(n2) for the communication and O(n3/mappers) the computation.

---