Python requests 爬虫结果不正确

2016-09-05 18:27:15 +08:00
 icy20088
还是之前那个问题哈, 我改成了 POST, 结果还是报错, 麻烦大虾们帮忙看看.

import requests
from bs4 import BeautifulSoup

myPost={"version":"1.1","method":"PmData.getDataSet","params":[[{"operation":"fetchpm","filter_starttime":"","filter_endtime":"","filter_extendtime":"","base_time":"","select_counters":"auto","filter_string":"nodename='BB1_C050E_C00323OD_BALOK'","prefix_class":0,"select_time":"total","base_object":"object","select_object":"nodename"},{"operation":"evalformula","formulasource":"all","formulafile":"etijian/etijian_New_BB","formulas":["C01_Data_CSSR","C02_RRC_Setup_Success_Rate","C03_RRC_Connection_Setup_SR_Service","C04_ERAB_Call_Setup_Success_Rate","C05_CSFB_Setup_Success_Rate","C06_eRAB_Drop_Call_Rate","C07_DL_Packet_Loss_Rate","C08_UL_Packet_Loss_Rate","C09_Intra_LTE_Handover_Success_Rate","C10_Network_Availability_Cell_Level","C11_IRATHO_Success_Rate","C12_VolTe_CSSR"]},{"operation":"output","fields":["C01_Data_CSSR","C02_RRC_Setup_Success_Rate","C03_RRC_Connection_Setup_SR_Service","C04_ERAB_Call_Setup_Success_Rate","C05_CSFB_Setup_Success_Rate","C06_eRAB_Drop_Call_Rate","C07_DL_Packet_Loss_Rate","C08_UL_Packet_Loss_Rate","C09_Intra_LTE_Handover_Success_Rate","C10_Network_Availability_Cell_Level","C11_IRATHO_Success_Rate","C12_VolTe_CSSR"],"top":"","filter":"","sortexpr":"time,object"}],{"repeat_merge":"0","repeat_timeprefix":["TimeSer"],"repeat_timefilter":[{"time_display":"From 2016-08-29 00:00 to 2016-09-05 00:00","time_start":"7d","time_end":"","time_unit":"d","time_extend_filter":""}],"repeat_objectprefix":[],"repeat_objectfilter":[],"repeat_objectexpand":"0"}],"itkdebug":{"view":"/itk/web/views/itk_standard/pm_dynaview.xml","client":"itk-web-client-3.0.5-1"}}

myHder1={
"Host":"4g.coldice.me",
"User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0",
"Accept":"application/json",
"Accept-Language": "en-US,en;q=0.5",
"Accept-Encoding":"gzip, deflate",
"Content-Type":"application/json; charset=UTF-8",
"Cookie":"itksession=5b4lj0Ai",
"Referer":"https://4g.coldice.me/itk/web/",
"Connection":"keep-alive",
"Pragma":"no-cache",
"Cache-Control":"no-cache"
}
myRep=requests.post("https://4g.coldice.me/itk/rpc",headers=myHder1,verify=False,data=myPost)

print(myRep.content)

回复:
b'{"jsonrpc":"2.0","error":{"message":"Error parsing JSON: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before \\"method=PmData.getDat...\\") at /opt/ericsson/itk/lib/Itk/WebService/JsonRpcHandler.pm line 271.\\n\\nView name: No view information available\\nClient name: No client information available","code":"-32700"},"id":null}'

可能是什么原因呢? requst的信息都是按照httpfox 抓到post请求写的.
2137 次点击
所在节点    Python
8 条回复
paulw54jrn
2016-09-05 18:36:07 +08:00
这个缩进让别人很难做啊..

网页也打不开, 被逆向墙了?
binux
2016-09-05 18:48:21 +08:00
myRep=requests.post("https://4g.coldice.me/itk/rpc",headers=myHder1,verify=False,json=myPost)
aeshfawre
2016-09-05 18:50:11 +08:00
楼上正解。
matrix67
2016-09-05 18:53:53 +08:00
json 没 dump
icy20088
2016-09-05 19:14:10 +08:00
@binux 哈哈,多谢了! 终于出来了, 你是咋一样看出 data 是 json 格式的呀!
binux
2016-09-05 19:17:55 +08:00
@icy20088
Error parsing JSON: malformed JSON string
at character offset 0 (before \\"method=PmData.getDat...\\")
Chyroc
2016-09-05 22:11:29 +08:00
@binux 请问 data 和 json 有什么区别。
我做的话,都是构造 post 的字典 data ,然后 requests.post(url, data),这是你说的哪一种情况
aeshfawre
2016-09-05 22:39:19 +08:00
@Chyroc 区别在于你 post 给服务器的数据格式不同,
data 是这样 user=aaa&pass=ffff
json 是这样{"user":"aaa","pass":"fff"}
服务器那边解析是按他写前端的时候约定好的,后端按哪种格式解析你 post 过来的数据,你提交的数据如果和约定不同,那会解析失败的。

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/304110

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX