liwenbest's recent timeline updates liwenbest's recent timeline updates |
liwenbestV2EX member #398105, joined on 2019-04-03 11:49:51 +08:00 |
| 舆情爬虫项目框架及存储选择 Python • liwenbest • Jan 18, 2021 • Lastly replied by tisswb | 16 |
| Python soap client 使用 suds 错误:suds.TypeNotFound: Type not found: 'faultcode',如何修复 Python • liwenbest • Jan 8, 2020 |
| 小白用 flask 做了个接口，运行一段时间，就会报“由于系统缓冲区空间不足或队列已满，不能执行套接字上的操作” Python • liwenbest • Sep 24, 2019 • Lastly replied by celeron533 | 6 |
| 求助： pyppeteer 是如何修改默认的浏览器下载路径的吗 Python • liwenbest • Sep 23, 2019 • Lastly replied by liwenbest | 2 |
| 求助 v 站大佬 如何利用 Python 迁移数据 Python • liwenbest • Aug 16, 2019 • Lastly replied by liwenbest | 14 |
| 有没有更好的图片验证码识别方式 Python • liwenbest • Apr 26, 2019 • Lastly replied by deepall | 3 |
liwenbest's recent replies
Jan 14, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@tisswb 我目前也是 scrapy_redis 但是存储用 mongodb
Jan 7, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@smgui 非常感谢 我看看
Jan 7, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@czfy 要自己开发的
Jan 7, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@Keyes 公司接了个舆情项目 要开发的 开发周期一年 要自己搞了
Jan 7, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@wzwwzw 沙滩车
Jan 7, 2021 Replied to a topic by liwenbest › Python › 舆情爬虫项目框架及存储选择 |
@AntoniotheFuture 加我 QQ986636628 私聊
Mar 30, 2020 Replied to a topic by wuwukai007 › Python › fastsql 1.2.19 发布 (个人项目,有点丑陋) |
挺好的，我也经常用 python 来做些 ETL 方面的工作，有个问题是 是表类型 oracle-mysql 字段类型不一致 会报错
Sep 24, 2019 Replied to a topic by liwenbest › Python › 小白用 flask 做了个接口，运行一段时间，就会报“由于系统缓冲区空间不足或队列已满，不能执行套接字上的操作” |
V 站这里贴代码有些乱
Sep 24, 2019 Replied to a topic by liwenbest › Python › 小白用 flask 做了个接口，运行一段时间，就会报“由于系统缓冲区空间不足或队列已满，不能执行套接字上的操作” |
#coding:utf-8
from flask import Flask
from flask import jsonify
from flask import request
import re
import time,random
from util import time_to_date,time_to_day
import json
from tyc_search import Tycsearch
class Apibase:
def __init__(self):
self.mongodb = Mongobase(GsmongoDB)
self.mongodb.connect()
def get_mongoone(self,condition,tname):
return self.mongodb.select(condition,tname)
def get_mongoall(self,tname):
return self.mongodb.all_dt(tname)
def close_mongo(self):
return self.mongodb.close()
class ComplexEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime):
return obj.strftime('%Y-%m-%d %H:%M:%S')
elif isinstance(obj, date):
return obj.strftime('%Y-%m-%d')
else:
return json.JSONEncoder.default(self, obj)
app = Flask(__name__)
apibase = Apibase()
tycseach = Tycsearch()
@app.route('/cha/search', methods=['GET'])
def get_tasks7():
#global m
kw = request.args.get("key")
kw = kw.replace(' (','(').replace(') ',')').replace(' ','')
print(f'请求信息:{kw}......')
try:
_ = apibase.mongodb.select({'searchkey': {'=':kw}},tname=GsmongoDB['tyc_records'])
if len(_)==0:
dic = tycseach.search(kw)
#print(dic)
return json.dumps(dic,ensure_ascii=False)
else:
dic = _[-1]
del dic['_id']
#apibase.close_mongo()
return json.dumps(dic,ensure_ascii=False)
except Exception as E:
print(f'请求失败:原因是:{E}..')
dic = {}
dic['code'] = '405'
dic['urlid'] = ""
dic['pingfen'] = ''
dic['searchkey'] = kw
dic['qymc'] = ''
dic['请求失败原因'] = '接口调用异常..'
now_time,now_day = time_to_day(time.time())
dic['climbdate'] = now_day
return json.dumps(dic,ensure_ascii=False)
@app.route('/')
def index():
return "Hello, World!"
if __name__ == '__main__':
#app.run(debug=True)
app.run(host="0.0.0.0",port=8090,threaded=True)#h
from flask import Flask
from flask import jsonify
from flask import request
import re
import time,random
from util import time_to_date,time_to_day
import json
from tyc_search import Tycsearch
class Apibase:
def __init__(self):
self.mongodb = Mongobase(GsmongoDB)
self.mongodb.connect()
def get_mongoone(self,condition,tname):
return self.mongodb.select(condition,tname)
def get_mongoall(self,tname):
return self.mongodb.all_dt(tname)
def close_mongo(self):
return self.mongodb.close()
class ComplexEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime):
return obj.strftime('%Y-%m-%d %H:%M:%S')
elif isinstance(obj, date):
return obj.strftime('%Y-%m-%d')
else:
return json.JSONEncoder.default(self, obj)
app = Flask(__name__)
apibase = Apibase()
tycseach = Tycsearch()
@app.route('/cha/search', methods=['GET'])
def get_tasks7():
#global m
kw = request.args.get("key")
kw = kw.replace(' (','(').replace(') ',')').replace(' ','')
print(f'请求信息:{kw}......')
try:
_ = apibase.mongodb.select({'searchkey': {'=':kw}},tname=GsmongoDB['tyc_records'])
if len(_)==0:
dic = tycseach.search(kw)
#print(dic)
return json.dumps(dic,ensure_ascii=False)
else:
dic = _[-1]
del dic['_id']
#apibase.close_mongo()
return json.dumps(dic,ensure_ascii=False)
except Exception as E:
print(f'请求失败:原因是:{E}..')
dic = {}
dic['code'] = '405'
dic['urlid'] = ""
dic['pingfen'] = ''
dic['searchkey'] = kw
dic['qymc'] = ''
dic['请求失败原因'] = '接口调用异常..'
now_time,now_day = time_to_day(time.time())
dic['climbdate'] = now_day
return json.dumps(dic,ensure_ascii=False)
@app.route('/')
def index():
return "Hello, World!"
if __name__ == '__main__':
#app.run(debug=True)
app.run(host="0.0.0.0",port=8090,threaded=True)#h
Sep 23, 2019 Replied to a topic by liwenbest › Python › 求助： pyppeteer 是如何修改默认的浏览器下载路径的吗 |
非常感谢
Select Language