Mapper/Reducer Configure method

Both the Mapper and Reducer classes contain a method that can be overridden: public void configure(JobConf conf).

This is called when the map/reduce task is instantiated rather than once per record. Data structures that need to be persistent across Map or Reduce calls can be created here, but note that they will NOT be shared across map or reduce tasks.

It is also possible to pass data to this method via the JobConf object. When the JobConf object is created the method JobConf.set(String name, Object value) can be used to set values for the mapper and reducer. During the configure method simply use JobConf.get(String name) to access the value previously set.

  • No labels