In a very huge text file, you want to just check if a particular keyword exists. How would you do this using Spark?



In a very huge text file, you want to just check if a particular keyword exists. How would you do th..

Answer / Vikas Kumar Pal

"To check if a specific keyword exists in a large text file using Apache Spark, you can use the filter() transformation with regex. Here's an example: nn val textFile = spark.textFile("path/to/your/file")n val containsKeyword = textFile.filter(line => line.matches("[w]+"+keyword+"[w]+"))n val result = containsKeyword.count()n if (result > 0) {n println("Keyword found!")n } else {n println("Keyword not found.")n }n"n

Is This Answer Correct ?    0 Yes 0 No

Post New Answer

More Apache Spark Interview Questions

What are the various storages from which Spark can read data?

1 Answers  


Explain different transformations in DStream in Apache Spark Streaming?

1 Answers  


What is the future of apache spark?

1 Answers  


How can I improve my spark performance?

1 Answers  


What are shared variables in spark?

1 Answers  


Is rdd type safe?

1 Answers  


Explain about the common workflow of a Spark program?

1 Answers  


List few benefits of spark over map reduce?

1 Answers  


What is spark deploy mode?

1 Answers  


What is apache spark architecture?

1 Answers  


What is spark yarn executor memoryoverhead?

1 Answers  


Explain parquet file?

1 Answers  


Categories