Manual partitioning of RDDs can be achieved by using the glom() and mapPart

How can you manually partition the rdd?

Question Posted / Mohit Kumar Sharma

1 Answers
317 Views
I also Faced
E-Mail Answers

Answer Posted / Mohit Kumar Sharma

Manual partitioning of RDDs can be achieved by using the glom() and mapPartitions() transformation functions. For example: rdd.glom().mapPartitions(iter => iter.toList.splitAll(partitionSize)).flatMap(_.iterator).collect()

Is This Answer Correct ?

0 Yes

0 No

Post New Answer View All Answers

Please Help Members By Posting Answers For Below Questions

Explain how RDDs work with Scala in Spark

355

What is the latest version of spark?

288

List the advantage of Parquet file in Apache Spark?

474

What is meant by Transformation? Give some examples.

328