LorDong's Blog

首先我们来观摸一下淘宝网的做法：http://www.taobao.com/robots.txt

依葫芦画瓢，这是我的写法：

User-agent: Baiduspider
Disallow: /
User-agent: baiduspider
Disallow: /
User-agent: *
Disallow: /down/
#更多要屏蔽的目录罗列在这里

参考内容：在Google里搜索“robots 屏蔽 百度”关键词组即可找到相关内容。

Tags: Google, 安全

This entry was posted on 星期三, 4月 27th, 2011 at 17:33 and is filed under 东东博客. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.