爬虫思路，这个有趣的网站，问一下思路怎么来做？

网站： http://www.exporivaschuh.it/catalogue/15ES2/search.html

我需要做的事找出所有的中国企业，cn, 爬出来他们的公司名字，电话号码，邮箱。

让我吐槽的是，似乎数据时存储在javascript里。

这个怎么分析。。思路怎么来弄，，求大家给个方案。

我一般都是beautifulsoup。

Supplement 1 · Jun 17, 2015

p[18] = new e ("AIMINER LEATHER PRODUCTS CO., LTD.","","","CN", "AIMINERLEATHERPRODUCTSCOLTD","NO.258 WENCHANG ROAD,JIN HUA QIAO STREET,WUHOU DIS","610043", "CHENGDU", "PALAFIERE - HALL B2 - Stand D08", "+86 28 85017357", "[email protected]";, "www.aiminer.net", "09012799","00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000");

p[17] = new e ("AIDER COMPANY LIMITED","","","CN", "AIDERCOMPANYLIMITED","9/F XINGHU COMMERCIAL BLDG., NO. 46 OF HU LI DA DA","", "XIAMEN", "PALAFIERE - HALL C4 - Stand A23", "+86-592-5699286", "[email protected]";, "",

p[32] = new e ("ALLROUNDER SARL","","","FR", "ALLROUNDERSARL","BP 60007 - ROUTE DE SARREGUEMINES ","57400", "SARREBOURG", "PALAZZO DEI CONGRESSI 1^ - Stand 15", "+33 387 233000", "[email protected]";, "www.allrounder.com", "16012812","00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000");

数据存储是上面这个样子的，，大家给个代码看看吧。。。
我不懂js, 目前只是自学到PYTHON

我需要的是，如：

前期是确认是 cn (中国），然后找到

AIMINER LEATHER PRODUCTS CO., LTD
+86 28 85017357
[email protected]
www.aiminer.net

思路

爬虫

JavaScript

6 replies • 2016-07-09 06:39:47 +08:00

mutoulbj

Jun 17, 2015

只要数据是可以获取到，再javascript里也没关系，自己处理下文本，再从中提取就可以了。

mhycy

Jun 17, 2015

分析JS逻辑，最简单直接用正则表达式抓取后重建索引

hiboshi

Jun 17, 2015

在js里面就更简单了直接正则匹配js文件

fangjinmin

Jun 17, 2015

import urllib2
import re
from bs4 import BeautifulSoup

url="http://www.exporivaschuh.it/catalogue/15ES2/search.html"
soup = BeautifulSoup(urllib2.urlopen(url).read())
script = soup.findAll('script')[0].string
p1 = re.compile('new e \(.*\)')
arrEs = p1.findall(script)
f = open('companysofchina.csv', 'w')
for e in arrEs:
e = e.replace('new e (','').replace(')', '')
arrItems = eval('[' + e + ']')
if arrItems[3] == 'CN':
company = arrItems[0]
tel = arrItems[9]
email = arrItems[10]
f.write(company + ',' + tel + ',' + email + '\n')

f.close()

redhatping

Jun 17, 2015

@fangjinmin 没错，搞定。。。非常感谢，

aeshfawre

Jul 9, 2016

@redhatping 已发邮件

爬虫思路，这个有趣的网站， 问一下思路怎么来做？

爬虫思路，这个有趣的网站，问一下思路怎么来做？