Coalesce and Repartition are operations used to change the number of partit

What is the difference between coalesce and repartition in spark?

Question Posted / Prem Shankar Jha

1 Answers
337 Views
I also Faced
E-Mail Answers

Answer Posted / Prem Shankar Jha

Coalesce and Repartition are operations used to change the number of partitions of a DataFrame or RDD. Repartition re-distributes data across new partitions, while Coalesce consolidates existing partitions (and potentially reducing the overall partition count) by combining smaller ones into larger ones. The main difference is that Repartition may shuffle the data more than Coalesce as it needs to redistribute the data among different partitions.

Is This Answer Correct ?

0 Yes

0 No

Post New Answer View All Answers

Please Help Members By Posting Answers For Below Questions

List the advantage of Parquet file in Apache Spark?

474

What is the latest version of spark?

288

Explain how RDDs work with Scala in Spark

355

What is meant by Transformation? Give some examples.

328