Please explain what is robots.txt?



Please explain what is robots.txt?..

Answer / Vineet Bajpai

Robots.txt is a text file that webmasters can create to instruct web robots (most notably search engine robots) how to crawl pages on their website. It's used for specifying which sections of the site should be ignored or accessible, thus controlling the behavior of these robots and conserving bandwidth.

Is This Answer Correct ?    0 Yes 0 No

Post New Answer

More Search Engine Optimization SEO Interview Questions

What is meant by paid result?

1 Answers  


The following robots Meta tag directs the search engine bots: <META NAME="robots" CONTENT="noindex,nofollow"> ► a. Not to index the homepage and not to follow the links in the page ► b. Not to index the page and not to follow the links in the page ► c. To index the page and not to follow the links in the page ► d. Not to index the page but to follow the links in the page

1 Answers  


Please explain how will you cross-check whether your seo campaign is working or not?

1 Answers  


Can you optimize the website which has pages in millions?

1 Answers  


What is the maximum number of links a sitemap can have?

1 Answers  


How does keyword selection relates to my business?

1 Answers  


Tell me what do you mean by backlink?

1 Answers  


What will the following robots.txt file do? User-agent:Googlebot Disallow:/*? User-agent:Scooter Disallow: ► a. It will allow Google to crawl any of the dynamically generated pages. It will also allow the AltaVista scooter bot to access every page ► b. It will disallow Google from crawling any of the dynamically generated pages. It will also disallow the AltaVista scooter bot from accessing any page ► c. It will disallow Google from crawling any of the dynamically generated pages. It will allow the AltaVista scooter bot to access every page ► d. None of the above

1 Answers  


What do you have to say about the Google Caffeine Update?

1 Answers  


What is monthly search volume of a keyword and how it's calculated?

1 Answers  


Tell me what is canonical url?

1 Answers  


What mistakes you should avoid while optimizing the website?

1 Answers  


Categories