HBase/Zookeeper Integration Documentation

Introduction

Apache ZooKeeper is for HBase what Chubby is for Bigtable. The goal of using ZK is that a coordination manager is useful to decentralize the master's job and it improves availability. In Bigtable's paper, Chubby is used to :

• Ensure there is at most 1 active master at any time
• Store the bootstrap location
• Discover tablet servers and finalize tablet server death
• Store the schema information
• Store access control lists

It has been discussed that, for the moment, only the three first bullets will be implemented. Also, in order to have a lesser impact on current users, ZK will be optional and a basic implementation will be provided in HBase. For those that will use ZK, the overhead will be like using HDFS.

For updates, see HBASE-546

The Interface

The interface must support macro operations so that dirty implementation details are hidden.

The current interface would be :

...

public interface DistributedLockInterface extends VersionedProtocol {

  public static final long versionID = 1L;

  /**
   * Used to bootstrap. Used to find a tablet location when a client doesn't have
   * META information
   * 
   * @return root region information
   */
  public byte[] getRootRegion() throws IOException;

  /**
   * Used by regionserver at startup or by master if a regionserver is down
   * 
   * @param filename File name of the lock
   */
  public void acquireLock(String filename) throws IOException;

  /**
   * Used by regionserver at shutdown or by master if a regionserver is down
   * 
   * @param filename File name of the lock
   */
  public void releaseLock(String filename) throws IOException;

  /**
   * Used by master at startup
   */
  public void acquireMasterLock() throws IOException;

  /**
   * Used by master at startup to discover live servers
   * @return All the addresses of the region servers
   */
  public HServerAddress[] getLiveRegionServers() throws IOException;

}

The HBase Implementation

This implementation of the interface would be master-based, this means that it doesn't support master failover and won't provide better availability.

See HBASE-708

The ZK Integration

Not so soon...