Interested to Buy Any Domain ? << Click Here >> for more details...
How will you calculate the number of executors required to do real-time processing using Apache Spark? What factors need to be considered for deciding on the number of nodes for real-time processing?
- 1 Answers
- 321 Views
- I also Faced
- E-Mail Answers
Answer Posted / Vishal Prayani
To determine the number of executors needed for real-time processing in Apache Spark, consider the following factors: data size, available hardware resources (CPU and memory), task complexity, and desired latency. Generally, a good starting point is to allocate one executor per CPU core on each worker node with appropriate memory allocations based on the specific application requirements. Increasing the number of nodes can further improve parallelism and throughput.
| Is This Answer Correct ? | 0 Yes | 0 No |
Post New Answer View All Answers
