Hadoop and IPv6

Apache Hadoop is not currently supported on IPv6 networks. It has only been tested and developed on IPv4 stacks. Hadoop needs IPv4 to work, and only IPv4 clients can talk to the cluster.If your organisation moves to IPv6 only, you will encounter problems.

Some things to be aware of:

  1. Some Linux releases default to being IPv6 only. That means unless the systems are configured to re-enable IPv4, some machines will break. As of Jan 2010, this was causing problems in Debian 1, 2, which is then leading to bug reports in other programs Sun bug database, Apache Jira.

  2. Binding Hadoop to an IPv6-only interface/name will not work.
  3. If you see NoRouteToHost error messages, this may be the cause.

Fixing the Debian Problem

To fix this, you need to change your system to allow IPv4 addressing.

The Fix recommended on the debian bug list is to patch the file /etc/sysctl.d/bindv6only.conf to set the configuration option net.ipv6.bindv6only to 0 and restart networking. The automated command to do this is:

sudo sed -i 's/net.ipv6.bindv6only\ =\ 1/net.ipv6.bindv6only\ =\ 0/' \
/etc/sysctl.d/bindv6only.conf && sudo invoke-rc.d procps restart

HadoopIPv6 (last edited 2012-11-29 05:22:27 by nat)