Answer / Yogendra Pal Singh

In Apache Spark, a "Stage" represents a collection of tasks that process the same dataset. A stage can be thought of as a unit of work that can run on one worker node at a time. On the other hand, a "Task" is the smallest unit of work that gets executed by workers in Spark. Each task performs some operation on a subset of data.

What is a parquet file?

1 Answers  

---

What is map in spark?

1 Answers  

---

What is a worker node in Apache Spark?

1 Answers  

---

Do I need to learn scala for spark?

1 Answers  

---

Which file systems does Spark support?

1 Answers  

---

What is a tuple in spark?

1 Answers  

---

Explain distnct(),union(),intersection() and substract() transformation in Spark?

1 Answers  

---

What are the roles and responsibilities of worker nodes in the Apache Spark cluster? Is Worker Node in Spark is same as Slave Node?

1 Answers  

---

What are broadcast variables in Apache Spark? Why do we need them?

1 Answers  

---

How can I speed up my spark?

1 Answers  

---

What is sparksession and sparkcontext?

1 Answers  

---

What is RDD lineage graph? How does it enable fault-tolerance in Spark?

1 Answers  

---