Robots.txt is a text tell search engine to avoid crawling of secret pages like the personal images folder, website administration folder, customerís test folder of a web developer ect.
Results 1 to 2 of 2
Thread: What Is Robot.txt?
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)