"Caching the DataFrame or DataSet can significantly improve the perfor

If there is certain data that we want to use again and again in different transformations, what should improve the performance?

Question Posted / Mahima Singh

1 Answers
323 Views
I also Faced
E-Mail Answers

Answer Posted / Mahima Singh

"Caching the DataFrame or DataSet can significantly improve the performance as it stores the data in memory for reuse across multiple actions. Another approach could be using persist() method with MEMORY_ONLY, MEMORY_ONLY_SER, or MEMORY_AND_DISK storage levels depending on the required level of persistence and memory footprint."

Is This Answer Correct ?

0 Yes

0 No

Post New Answer View All Answers

Please Help Members By Posting Answers For Below Questions

What is the latest version of spark?

288

Explain how RDDs work with Scala in Spark

355

What is meant by Transformation? Give some examples.

328

List the advantage of Parquet file in Apache Spark?

474