 |
|
|
This is Topic: Web Design
Following are the News Items published under this Topic.
|
|
 |
Posted by reina on Oct 14, 2004 - 12:22 AM (1866 Reads)
|
 |
 |
The robots.txt file is an ASCII text file that has specific instructions for search engine robots about specific content that they are not allowed to index. These instructions are the deciding factor of how a search engine indexes your website’s pages. The universal address of the robots.txt file is: www.domain.com/robots.txt . This is the first file that a robot visits. It picks up instructions for indexing the site content and follows them. This file contains two text fields.
|
Read more... (13053 bytes more) comments?  |
|
 |
|
|
|
|
