Golgappa.net | Golgappa.org | BagIndia.net | BodyIndia.Com | CabIndia.net | CarsBikes.net | CarsBikes.org | CashIndia.net | ConsumerIndia.net | CookingIndia.net | DataIndia.net | DealIndia.net | EmailIndia.net | FirstTablet.com | FirstTourist.com | ForsaleIndia.net | IndiaBody.Com | IndiaCab.net | IndiaCash.net | IndiaModel.net | KidForum.net | OfficeIndia.net | PaysIndia.com | RestaurantIndia.net | RestaurantsIndia.net | SaleForum.net | SellForum.net | SoldIndia.com | StarIndia.net | TomatoCab.com | TomatoCabs.com | TownIndia.com
Interested to Buy Any Domain ? << Click Here >> for more details...

  1. Categories >> Software >> Big Data >> Hadoop >> Apache Spark
  2. Suggest New Category

How does Apache Spark handles accumulated Metadata?

Question Posted / Peeyush Tripathi

Answer Posted / Peeyush Tripathi

Apache Spark manages accumulated metadata using a structure called the lineage. Each RDD in the computation has a lineage, which is a record of all its ancestors. This allows Spark to trace back the history of data and recalculate any RDD that is needed again if it or one of its dependencies fails. Additionally, Spark periodically prunes the lineage graph to reduce memory usage.

Is This Answer Correct ?    0 Yes 0 No



Post New Answer       View All Answers


Please Help Members By Posting Answers For Below Questions

What is meant by Transformation? Give some examples.

328

What is the latest version of spark?

288

List the advantage of Parquet file in Apache Spark?

474

Explain how RDDs work with Scala in Spark

355