Answer / Romi Awasthi

Transformations are lazy in Spark to optimize resource utilization. When a transformation is called, it doesn't immediately execute the action on all data; instead, it creates a logical plan outlining the steps required to process the data. The physical execution starts only when an action like collect(), count(), or save() is triggered.

What is the difference between dataset and dataframe in spark?

1 Answers  

---

State the difference between Spark SQL and Hql

1 Answers  

---

Explain the lookup() operation in Spark?

1 Answers  

---

Define RDD?

1 Answers  

---

How many partitions are created by default in Apache Spark RDD?

1 Answers  

---

Should I install spark on all nodes of yarn cluster?

1 Answers  

---

What are the various libraries available on top of Apache Spark?

1 Answers  

---

Define parquet file format? How to convert data to parquet format?

1 Answers  

---

Is a distributed machine learning framework on top of spark?

1 Answers  

---

What are the ways in which Apache Spark handles accumulated Metadata?

1 Answers  

---

Can you define yarn?

1 Answers  

---

How is rdd fault?

1 Answers  

---