ad_728x90

2013年6月24日 星期一

Robots.txt Specifications

URL matching based on path values Example path matches
[path]Matches Does not matchComments
/any valid URL Matches the root and any lower level URL
/*equivalent to / equivalent to / Equivalent to "/" -- the trailing wildcard is ignored.
/fish/fish
/fish.html
/fish/salmon.html
/fishheads
/fishheads/yummy.html
/fish.php?id=anything
/Fish.asp
/catfish
/?id=fish
Note the case-sensitive matching.
/fish*/fish
/fish.html
/fish/salmon.html
/fishheads
/fishheads/yummy.html
/fish.php?id=anything
/Fish.asp
/catfish
/?id=fish
Equivalent to "/fish" -- the trailing wildcard is ignored.
/fish//fish/
/fish/?id=anything
/fish/salmon.htm
/fish
/fish.html
/Fish/Salmon.asp
The trailing slash means this matches anything in this folder.
fish/equivalent to /fish/ equivalent to /fish/ equivalent to /fish/
/*.php/filename.php
/folder/filename.php
/folder/filename.php?parameters
/folder/any.php.file.html
/filename.php/
/ (even if it maps to /index.php)
/windows.PHP
/*.php$/filename.php
/folder/filename.php
/filename.php?parameters
/filename.php/
/filename.php5
/windows.PHP
/fish*.php/fish.php
/fishheads/catfish.php?parameters
/Fish.PHP

COMMENTS HAVE BEEN DISABLED FOR THIS POST [文章的評論已被禁用]

Ratings and Recommendations by outbrain