What combiners are and when you should use a combiner in a mapreduce job?
Answer / Akhilesh Kumar Kushwaha
Combiners are a feature of the MapReduce programming model that allows intermediate results from map tasks to be partially reduced before they are written to disk. This can help reduce the amount of data that needs to be shuffled between the map and reduce phases, thereby improving the overall performance of the job. Combiners should be used when the reduce function is commutative and associative, meaning that the order in which map output pairs are processed does not affect the final result and the operation can be performed in multiple steps without changing the outcome. For example, if you have a MapReduce job to find the sum of values for each key, using a combiner would help reduce the number of intermediate calculations before the actual reduction.
| Is This Answer Correct ? | 0 Yes | 0 No |
What is Hadoop Map Reduce ?
What is the function of mapreduce partitioner?
How to write MapReduce Programs?
mapper or reducer?
What is shuffling in mapreduce?
What is difference between a MapReduce InputSplit and HDFS block
Where sorting is done in Hadoop MapReduce Job?
What are the primary phases of a Reducer?
Is it necessary to write a mapreduce job in java?
Define MapReduce?
What is a "map" in Hadoop?
What does conf.setmapper class do?
Apache Hadoop (394)
MapReduce (354)
Apache Hive (345)
Apache Pig (225)
Apache Spark (991)
Apache HBase (164)
Apache Flume (95)
Apache Impala (72)
Apache Cassandra (392)
Apache Mahout (35)
Apache Sqoop (82)
Apache ZooKeeper (65)
Apache Ambari (93)
Apache HCatalog (34)
Apache HDFS Hadoop Distributed File System (214)
Apache Kafka (189)
Apache Avro (26)
Apache Presto (15)
Apache Tajo (26)
Hadoop General (407)