Answer Posted / Shobhit Asthana
Shuffle Spill in Spark occurs when the size of intermediate data exceeds the memory capacity available for shuffle operations. During these operations, data is spilled to disk, which can result in slower performance due to disk I/O operations. To mitigate this issue, Spark provides options such as increasing worker memory or enabling more nodes.
| Is This Answer Correct ? | 0 Yes | 0 No |
Post New Answer View All Answers