Goals
System should be :
- Stable & Reliable
- Fault tolerant
- Scalability
- Efficient
- Cost-effective
- Maintainable
Hbase Index
+-------------+ +---------------+ | USER TABLEs | +-------+ | META 1 ~ 2^17 | | ... | | META0 +-----+ ... +-----+ | +-------+ | | | | +---------------+ | | +-------------+ 1 2 3 ↑ ↑ ↑ | | | +---+-------+--------+ | | Client +------------------------------+ +-----------+
Block Diagram
- MDA : Mail Delivery Agent +-----------------------------+ +-----------+ | Mail Frontend Web servers | * Client layer * | Watcher | | (web UI, send an mails) | +-----+-----+ +------+---------------+------+ | | ↑ | SMTP read read sent mail | requests reply log +--------------------------------------------------------------------------------------+ | ↓ ↓ | ↓ | | +--------------------------+----------------------+ | | | Broker Servers (MDA & Spam Filtering, ..., etc) +←--+ * Broker layer * | +--------------------------+----------------------+ | | | ↑ ↑ | | | +--------------------+--------------------+---------------+----------+-----------------+ | | | | Reads user profile Read contents Update contents | | | ↓ | +-----+-------------------+-----+ +---------+----------------------+ | | Hbase:UserTable | | Hbase:MIMETable | | | (user profile, logs) | | (MIME data) | | * Storage layer * +-------------------------------+ +--------------------------------+ | ↑ ↑ | +-----------------------------+--------------------+----------------------+ | | Hadoop: MapReduce Jobs (e.g. Log Analysis, SPAM calculation, .., etc) |←--+ +-------------------------------------------------------------------------+
Table Schema
Row Key is a email address, a unique identifier and a Sender & Receiver. Column families are meta data. (e.g. from, to, cc, bcc, reply-to, ..., etc)
From: To: ==============================================================================================================~~ edwardyoon@apache.org From:edward@udanax.org <"Edward J. Yoon"> To:chanwit@gmail.com <"Chanwit"> edward@udanax.org From:hama-dev@incubator.apache.org <"hama-dev"> Subject: Content: Address-Book: ... ~~============================================================================================================== <"Hello, I'm ed"> <"This is a content"> Address-Book:doug@apache.org <"Doug Cutting"> ... Address-Book:channy@gmail.com <"Channy Yoon"> ...