Sogou的蜘蛛怎么这么疯狂？堪比DDoS了

2013-12-24 23:11:30 +08:00
 xiaop
66.249.64.1 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /redirect.php?goto=findpost&pid=567887 HTTP/1.0" 302 20 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
218.30.103.97 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/144666.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.89.164 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/6636/last/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
218.30.103.87 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /member/154806.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
61.135.189.124 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/167728.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
218.30.103.157 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/140326/p2.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
218.30.103.148 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/29477.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
66.249.64.2 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/178887.html HTTP/1.0" 200 7465 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
218.30.103.132 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/22570.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
66.249.64.4 - - [24/Dec/2013:23:10:57 +0800] m.quxizang.net "GET /bbs/t7730.html HTTP/1.0" 200 2151 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
218.30.103.154 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/119864.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
61.135.189.70 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/112342.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
218.30.103.152 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/29939/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
5541 次点击
所在节点    站长
7 条回复
xiaop
2013-12-24 23:15:36 +08:00
其中一台小服务器已经挂掉，赶紧屏蔽了搜狗蜘蛛，日志是屏蔽后的。
zhouzb889
2013-12-24 23:22:24 +08:00
曾经被这鬼东西消耗完流量，果断拒绝。
它的Agent包括：Sogou web spider，Sogou News Spider，New-Sogou-Spider

在DNS设置中解析到一个不存在的IP貌似也是可行的。

另外，Sosospider也很疯狂！
binux
2013-12-24 23:28:05 +08:00
又到年底覆盖率评估的时候了
juicy
2013-12-24 23:38:56 +08:00
。。。。我最近还刻意去添加了搜狗的SEO。。。。。悲剧。。。。
katekiss
2013-12-24 23:42:48 +08:00
哎，不行就是不行
solo7net
2013-12-25 09:34:15 +08:00
我的带宽被百度跑满过
jjplay
2013-12-25 10:41:13 +08:00
藏ICP备很少见的啊，LZ西藏汉子啊

这是一个专为移动设备优化的页面（即为了让你能够在 Google 搜索结果里秒开这个页面），如果你希望参与 V2EX 社区的讨论，你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/94438

V2EX 是创意工作者们的社区，是一个分享自己正在做的有趣事物、交流想法，可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX