Scrapy 的 RetryMiddleware 不生效，求教

按照惯例，我先贴出来代码

class CustomerRetryMiddleware(RetryMiddleware):
    """
        去掉了一些判断，增加了 log
    """

    def process_response(self, request, response, spider):
        if response.status in self.retry_http_codes:
            retries = request.meta.get('retry_times', 0) + 1
            reason = response_status_message(response.status)
            spider.logger.warn('下载中收到错误 http code:{0}，丢入重试，第{1}次重试'.format(reason, retries))
            return self._retry(request, reason, spider) or response
        else:
            return response

    """
        去掉了 EXCEPTIONS_TO_RETRY 判断，无脑重试
    """

    def process_exception(self, request, exception, spider):
        retries = request.meta.get('retry_times', 0) + 1
        spider.logger.warn('下载中收到:{0}，丢入重试，第{1}次重试'.format(exception.reasons, retries))
        return self._retry(request, 'CustomerRetryError', spider)

    def _retry(self, request, reason, spider):
        retries = request.meta.get('retry_times', 0) + 1

        retry_times = self.max_retry_times

        if 'max_retry_times' in request.meta:
            retry_times = request.meta['max_retry_times']

        stats = spider.crawler.stats
        if retries <= retry_times:
            spider.logger.debug("Retrying %(request)s (failed %(retries)d times): %(reason)s",
                                {'request': request, 'retries': retries, 'reason': reason},
                                extra={'spider': spider})
            retryreq = request.copy()
            retryreq.meta['retry_times'] = retries
            retryreq.dont_filter = True
            retryreq.priority = request.priority + self.priority_adjust

            if isinstance(reason, Exception):
                reason = global_object_name(reason.__class__)

            stats.inc_value('retry/count')
            stats.inc_value('retry/reason_count/%s' % reason)
            return retryreq
        else:
            stats.inc_value('retry/max_reached')
            spider.logger.warn('超过重试次数，停止重试')

process_response 方法主要是为了加一点 log process_exception 方法是为了出现任何 Exception 都能重试同样，重写 _retry 方法也是为了加点 log

我遇到的问题是，如果我 settings.py 中设定了 5 次重试，那么，log 中一定只打印第 6 次重试的 log，前 5 次的 log 并没有打出来，我很疑惑，求解答

另外，我为了去除次数的问题，我还改配置改到 100 次，然而 log 中只打印第 101 次重试，然后就是 _retry 方法中的 else 语句的 log "超过重试次数，停止重试"

warcraft1236

2019-02-04 18:28:13 +08:00

@meik2333 我重新试了一下，我上边这段代码确实打印不出来 if retries <= retry_times: 判断里边的 log

重新看了一下，发现

```
INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'New91Crawler.middlewares.CustomerRetryMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']

```

我自定义的 retry 和 Scrapy 自带的 retry 都加载了，理论上不应该用我的覆盖系统的吗？