Monday, September 4, 2017

Specify the name of output files in MR

E.g.
public class WordCount {
 
    public static void main(String[] args) throws Exception {
        JobConf conf = new JobConf(WordCount.class);
        conf.setOutputKeyClass(Text.class);
        conf.setOutputValueClass(Text.class);
        conf.setOutputFormat(CustomMultipleTextOutputFormat.class);
        FileInputFormat.setInputPaths(conf, new Path(args[0]));
        FileOutputFormat.setOutputPath(conf, new Path(args[1]));
        JobClient.runJob(conf);
    }
}
 
class CustomMultipleTextOutputFormat extends MultipleTextOutputFormat<Object, Object> {
    protected String generateFileNameForKeyValue(Object key, Object value, String name) {
        return key.toString() + "-" + name;
    }
}

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.