Answer / Raj Ratna Singh

Coalesce operation in Apache Spark is used to reduce the total number of partitions of an RDD, DataFrame, or Dataset. This can help save memory when dealing with large datasets as it consolidates smaller partitions into larger ones. However, coalescing to a larger number of partitions than necessary should be avoided because it can lead to slower performance due to increased shuffle operations. Coalescing is useful when the application needs to reduce the overhead of managing many small partitions but still wants to maintain a reasonable number of partitions for efficient computation.

What do you understand by Transformations in Spark?

1 Answers  

---

How does broadcast join work in spark?

1 Answers  

---

What is rdd in spark with example?

1 Answers  

---

Do you need to install Spark on all nodes of Yarn cluster while running Spark on Yarn?

1 Answers  

---

What is the key difference between textfile and wholetextfile method?

1 Answers  

---

What is spark master?

1 Answers  

---

What is spark architecture?

1 Answers  

---

Why spark is faster than hadoop?

1 Answers  

---

Is there any API available for implementing graphs in Spark?

1 Answers  

---

How can you trigger automatic clean-ups in Spark to handle accumulated metadata?

1 Answers  

---

What does map transformation do? Provide an example.

1 Answers  

---

Which is the best spark certification?

1 Answers  

---