V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
lywane
V2EX  ›  Python

请教使用 gunicorn 部署 flask 时遇到的 redis 连接数飙升问题

  •  
  •   lywane · 2018-01-08 11:27:51 +08:00 · 3241 次点击
    这是一个创建于 2511 天前的主题,其中的信息可能已经有所发展或是发生改变。
    小弟写了个简单的 flask 接口,其中使用了 redis,连接 redis 的语句写在全局里了,如下:
    cache = redis.StrictRedis('127.0.0.1', 6379)

    然后使用 gunicorn 部署,部署命令是:
    gunicorn --workers=9 --worker-class=gevent t_fl:app -b 0.0.0.0:5002

    在高并发情况下(并发两千至五千)执行十万次请求,使用 info clients 命令查看 redis 的客户端连接数飙升到 3000 左右,严重拖慢了查询速度。请问有什么解决办法。
    4 条回复    2018-06-05 15:49:56 +08:00
    myyou
        1
    myyou  
       2018-01-08 11:58:12 +08:00
    使用 Redis 连接池 ConnectionPool 设置最大连接数量
    lywane
        2
    lywane  
    OP
       2018-01-08 12:32:03 +08:00
    @myyou
    如下声明之后,
    rdp = redis.ConnectionPool(host='127.0.0.1', port=6379,max_connections=10)
    cache = redis.StrictRedis(connection_pool=rdp)

    请求过来时会报错:
    [2018-01-08 12:06:10,117] ERROR in app: Exception on / [POST]
    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
    File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
    File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
    File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
    File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
    File "/opt/t_fl.py", line 54, in index
    timeout, endtime = get_timeout_endtime(GID)
    File "/opt/t_fl.py", line 20, in get_timeout_endtime
    timeout_endtime = cache.get(gid)
    File "/usr/lib/python2.7/site-packages/redis/client.py", line 976, in get
    return self.execute_command('GET', name)
    File "/usr/lib/python2.7/site-packages/redis/client.py", line 665, in execute_command
    connection = pool.get_connection(command_name, **options)
    File "/usr/lib/python2.7/site-packages/redis/connection.py", line 965, in get_connection
    connection = self.make_connection()
    File "/usr/lib/python2.7/site-packages/redis/connection.py", line 981, in make_connection
    raise ConnectionError("Too many connections")
    myyou
        3
    myyou  
       2018-01-08 14:54:08 +08:00
    gunicorn 使用 gevent 时--worker-connections 默认 1000,你可以把这个值设置小一些
    liuxingpu
        4
    liuxingpu  
       2018-06-05 15:49:56 +08:00
    我也碰到这个问题,请问楼主解决了没?
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   961 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 28ms · UTC 20:49 · PVG 04:49 · LAX 12:49 · JFK 15:49
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.