阅读 Tornado 源码过程中的一个疑惑,求解答

2016-01-05 19:50:34 +08:00
 kidlj

下面是 tornado.httpclient.AsyncHTTPClient类的 fetch()方法的源代码。我没有在里面找到任何"fetch"的动作,它是怎么实现 “ Executes a request, asynchronously returning an HTTPResponse”的?

完整代码在: https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py

def fetch(self, request, callback=None, raise_error=True, **kwargs):
        """Executes a request, asynchronously returning an `HTTPResponse`.
        The request may be either a string URL or an `HTTPRequest` object.
        If it is a string, we construct an `HTTPRequest` using any additional
        kwargs: ``HTTPRequest(request, **kwargs)``
        This method returns a `.Future` whose result is an
        `HTTPResponse`. By default, the ``Future`` will raise an
        `HTTPError` if the request returned a non-200 response code
        (other errors may also be raised if the server could not be
        contacted). Instead, if ``raise_error`` is set to False, the
        response will always be returned regardless of the response
        code.
        If a ``callback`` is given, it will be invoked with the `HTTPResponse`.
        In the callback interface, `HTTPError` is not automatically raised.
        Instead, you must check the response's ``error`` attribute or
        call its `~HTTPResponse.rethrow` method.
        """
        if self._closed:
            raise RuntimeError("fetch() called on closed AsyncHTTPClient")
        if not isinstance(request, HTTPRequest):
            request = HTTPRequest(url=request, **kwargs)
        else:
            if kwargs:
                raise ValueError("kwargs can't be used if request is an HTTPRequest object")
        # We may modify this (to add Host, Accept-Encoding, etc),
        # so make sure we don't modify the caller's object.  This is also
        # where normal dicts get converted to HTTPHeaders objects.
        request.headers = httputil.HTTPHeaders(request.headers)
        request = _RequestProxy(request, self.defaults)
        future = TracebackFuture()
        if callback is not None:
            callback = stack_context.wrap(callback)

            def handle_future(future):
                exc = future.exception()
                if isinstance(exc, HTTPError) and exc.response is not None:
                    response = exc.response
                elif exc is not None:
                    response = HTTPResponse(
                        request, 599, error=exc,
                        request_time=time.time() - request.start_time)
                else:
                    response = future.result()
                self.io_loop.add_callback(callback, response)
            future.add_done_callback(handle_future)

        def handle_response(response):
            if raise_error and response.error:
                future.set_exception(response.error)
            else:
                future.set_result(response)
        self.fetch_impl(request, handle_response)
        return future

def fetch_impl(self, request, callback):
        raise NotImplementedError()
9182 次点击
所在节点    Tornado
6 条回复
phithon
2016-01-05 20:26:06 +08:00
![]( )
impl 看名字应该是个接口,等着子类去实现的。
你可以看 AsyncHTTPClient 的 `__new__` 方法,实际上执行 `new AsyncHTTPClient()` ,其返回的对象是一个 SimpleAsyncHTTPClient 实例, SimpleAsyncHTTPClient 是对 AsyncHTTPClient 的实现。
所以,最后执行的 fetch_impl 方法,实际上在这里:
https://github.com/tornadoweb/tornado/blob/6b312f7a73dc78de4b1be2a71a5bbfa51acc725d/tornado/simple_httpclient.py#L122
kidlj
2016-01-05 20:29:09 +08:00
@phithon 非常感谢!
sujin190
2016-01-05 20:45:30 +08:00
https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py#L148
这设置了生成实例时是哪个实现的实例
sujin190
2016-01-05 20:47:41 +08:00
不过比较坑的是 SimpleAsyncHTTPClient 本身不支持连接池,每次都是打开新连接,除此之外还有一个实现是 CurlAsyncHTTPClient ,支持连接池,不过在 pypy 下似乎会出现栈溢出错误
kidlj
2016-01-05 21:33:08 +08:00
@sujin190 谢谢。我的疑问得到了解答。
mqingyn616
2016-01-06 23:50:35 +08:00
@sujin190 这个问题有一个比较靠谱的解决方案,可以看 github 上的这个 pr , https://github.com/tornadoweb/tornado/pull/1622

给 SimpleAsyncHTTPClient 增加连接池,亲测可用。

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/248581

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX