Answer Posted / Vineet Bajpai
Robots.txt is a text file that webmasters can create to instruct web robots (most notably search engine robots) how to crawl pages on their website. It's used for specifying which sections of the site should be ignored or accessible, thus controlling the behavior of these robots and conserving bandwidth.
| Is This Answer Correct ? | 0 Yes | 0 No |
Post New Answer View All Answers
