Differences between revisions 15 and 16
Revision 15 as of 2008-12-16 15:22:09
Size: 4441
Comment:
Revision 16 as of 2009-09-20 21:45:51
Size: 4441
Editor: localhost
Comment: converted to 1.6 markup
No differences found!

Goals

System should be :

  • Stable & Reliable

  • Fault tolerant
  • Scalability
  • Efficient
  • Cost-effective
  • Maintainable

Table Indices

                                         +-------------+
                   +---------------+     | USER TABLEs |
     +-------+     | META 1 ~ 2^17 |     | ...         |
     | META0 +-----+ ...           +-----+             |
     +-------+     |               |     |             |
                   +---------------+     |             |
                                         +-------------+
         1                2                     3
         ↑                ↑                     ↑
         |                |                     |
     +---+-------+--------+                     |
     |  Client   +------------------------------+
     +-----------+

Block Diagram

- MDA : Mail Delivery Agent

                                            +-----------------------------+
                          +-----------+     |  Mail Frontend Web servers  |                 * Client layer *
                          |  Watcher  |     |   (web UI, send an mails)   |
                          +-----+-----+     +------+---------------+------+
                                |                  |       ↑       |
                               SMTP              read     read  sent mail
                                |               requests  reply   log
         +--------------------------------------------------------------------------------------+ 
         |                      ↓                  ↓       |       ↓                            |
         |              +--------------------------+----------------------+                     |
         |              | Broker Servers (MDA & Spam Filtering, ..., etc) +←--+             * Broker layer *
         |              +--------------------------+----------------------+   |                 |            
         |                    ↑                    ↑               |          |                 |
         +--------------------+--------------------+---------------+----------+-----------------+
                              |                    |               |          |
                     Reads user profile       Read contents   Update contents |
                              |                    |               ↓          |
    +-----+-------------------+-----+    +---------+----------------------+   |         
    |       Hbase:UserTable         |    |        Hbase:MIMETable         |   |
    |     (user profile, logs)      |    |          (MIME data)           |   |             * Storage layer *
    +-------------------------------+    +--------------------------------+   |
                              ↑                    ↑                          |
+-----------------------------+--------------------+----------------------+   |
|  Hadoop: MapReduce Jobs (e.g. Log Analysis, SPAM calculation, .., etc)  |←--+
+-------------------------------------------------------------------------+

Table Schema

Row Key is a email address, a unique identifier and a Sender & Receiver. Column families are meta data. (e.g. from, to, cc, bcc, reply-to, ..., etc)

                         From:                                              To: 
==============================================================================================================~~
edwardyoon@apache.org    From:edward@udanax.org <"Edward J. Yoon">   

                                                                            To:chanwit@gmail.com <"Chanwit">
edward@udanax.org        From:hama-dev@incubator.apache.org <"hama-dev">
                                 



   Subject:               Content:                 Address-Book:                                       ...
~~==============================================================================================================
   <"Hello, I'm ed">      <"This is a content">    Address-Book:doug@apache.org  <"Doug Cutting">      ...
                                                   Address-Book:channy@gmail.com <"Channy Yoon">
                                                   ...

SPAM filtering

  • Sender Reputation
  • Social Graph
  • Text Pattern

HbaseForWebMail (last edited 2009-09-20 21:45:51 by localhost)