Interested to Buy Any Domain ? << Click Here >> for more details...
What is the reason behind Transformation being a lazy operation in Apache Spark RDD? How is it useful?
- 1 Answers
- 510 Views
- I also Faced
- E-Mail Answers
Answer Posted / Deepak Kumar Tiwari
In Apache Spark, transformations are designed to be lazily evaluated. This means that transformations do not immediately execute when applied; instead, they are stored as logical operations on the RDD until an action (like collect, count, saveAsTextFile) is called. The benefits of lazy evaluation include: (1) Improved performance: Transformations can be optimized and batched together for efficient execution before applying the action. (2) Fault-tolerance: If a task fails, only the failed task needs to be recomputed instead of the entire lineage of data. (3) Reduced network communication: Data is only sent between nodes when necessary, which reduces the amount of data transferred and improves overall performance.
| Is This Answer Correct ? | 0 Yes | 0 No |
Post New Answer View All Answers
