MapReduce Indexing

Here is an example of the typical usage of the BlurOutputFormat. The Blur table has to be created before the MapReduce job is started. The setupJob method configures the following:

Example Usage

Iface client = BlurClient.getClient("controller1:40010");

TableDescriptor tableDescriptor = client.describe(tableName);

Job job = new Job(jobConf, "blur index");
job.setJarByClass(BlurOutputFormatTest.class);
job.setMapperClass(CsvBlurMapper.class);
job.setInputFormatClass(TextInputFormat.class);

FileInputFormat.addInputPath(job, new Path(input));
CsvBlurMapper.addColumns(job, "cf1", "col");

BlurOutputFormat.setupJob(job, tableDescriptor);
BlurOutputFormat.setIndexLocally(job, true);
BlurOutputFormat.setOptimizeInFlight(job, false);

job.waitForCompletion(true);

Options

MapReduce (last edited 2013-06-14 01:41:03 by AaronMcCurry)