elasticsearch 搜索错误

[wico77]

服务器配置 16G 内存， 500G 硬盘，分了 8G 给 ES 用。网站搭配 flask+nginx+mongodb+elasticsearch,最近 google webmaster 里老出现 500 和 502 错误，见前一个帖子 http://v2ex.com/t/247625#reply9 ，现在找到原因是 ES 全文搜索有错误，我在 ES 日志里看到不少这种搜索错误，请问是什么原因造成的？内存还是小吗？
[2016-01-19 23:42:38,283][DEBUG][action.search.type ] [Razorback] [exta][2], node[7B-RkEI5Rq2uLL2yqg3CPA], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@5a7adcd4] lastShard [true]
org.elasticsearch.search.SearchParseException: [exta][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:721)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:557)
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:529)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:291)
at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.index.query.QueryParsingException: [exta] Failed to parse query [gymnast torture]
at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:250)
at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:302)
at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:382)
at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:281)
at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:276)
at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:705)
... 9 more
Caused by: org.apache.lucene.queryparser.classic.ParseException: Cannot parse 'gymnast torture\': Lexical error at line 1, column 17. Encountered: <EOF> after : ""
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:137)
at org.apache.lucene.queryparser.classic.MapperQueryParser.parse(MapperQueryParser.java:887)
at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:233)
... 15 more
Caused by: org.apache.lucene.queryparser.classic.TokenMgrError: Lexical error at line 1, column 17. Encountered: <EOF> after : ""
at org.apache.lucene.queryparser.classic.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1133)
at org.apache.lucene.queryparser.classic.QueryParser.jj_scan_token(QueryParser.java:601)
at org.apache.lucene.queryparser.classic.QueryParser.jj_3R_2(QueryParser.java:484)
at org.apache.lucene.queryparser.classic.QueryParser.jj_3_1(QueryParser.java:491)
at org.apache.lucene.queryparser.classic.QueryParser.jj_2_1(QueryParser.java:477)
at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:228)
at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:214)
at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:172)
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:127)
... 17 more

[windfarer]

错误很明显嘛
Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]
Cannot parse 'gymnast torture\': Lexical error at line 1, column 17. Encountered: <EOF> after : ""

传过去的查询 DSL 不正确

[Zzzzzzzzz]

报错很明显, 你传的 json 有问题啊.....

org.elasticsearch.search.SearchParseException: [exta][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]

[wico77]

@Zzzzzzzzz 你的意思是 query 查询的词后面有个'\'?
我使用 pyes ，获取查询词之后的语句是这样的：
bt = QueryStringQuery(keyword.lower())
s = Search(bt)
videos =conn.search(s, size=32, indices="exta",start=(page-1)*32)
请问传过去的查询如何修正？

[wico77]

@windfarer 请指点如何修改

[windfarer]

@wico77 pyes 我没用过啊，那几个 python 的库选来选去，我们最后选了官方的 elasticsearch-dsl-py

[Zzzzzzzzz]

@wico77 我没复现你的问题, 我猜是不是低版本 simplejson 导致的, 印象里以前 simplejson 确实有过 slash 问题, 先升级一下重启应用试试.

[wico77]

@Zzzzzzzzz pip install simplejson
Collecting simplejson
/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading simplejson-3.8.1.tar.gz (76kB)
100% |████████████████████████████████| 77kB 520kB/s
Building wheels for collected packages: simplejson
Running setup.py bdist_wheel for simplejson
Stored in directory: /root/.cache/pip/wheels/4e/25/10/2ff5f7347927f4816794414d599a5383c7f8dd9a9ff729af28
Successfully built simplejson
Installing collected packages: simplejson
Successfully installed simplejson-3.8.1
我安装了下 simplejson ，难道是默认没安装 simplejson ？

[Zzzzzzzzz]

@wico77 从 2.6 开始引入的自带 json 包就是 simplejson, 不过考虑到最新的 simplejson 包会做一些优化和解决一些 bug 一般都优先导入 simplejson 包, 没有再回退到 python 自带的 json 库, 我看 pyes 也是这样做的, 用了最新的 simplejson 还出错么?

[wico77]

@Zzzzzzzzz 等明天看 ES 的 log 再回复你

[wico77]

@Zzzzzzzzz 用了最新的 simplejson 如果 keyword 末端有\也会出错，我把这些 escape 去掉就好了