es7 在实际的项目中使用java es 客户端 scroll 报错了

Question

{"error":{"root_cause":[{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."},{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"collect_data_store_customer_flow_fact_201910","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}},{"shard":0,"index":"collect_data_store_customer_flow_fact_201911","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}}]},"status":500}
                at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:920)
                at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227)
                at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433)
                ... 62 common frames omitted
        Caused by: org.elasticsearch.client.ResponseException: method [POST], host [http://172.18.32.181:9200], URI [/collect_data_store_customer_flow_fact*/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&scroll=60000ms&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."},{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"collect_data_store_customer_flow_fact_201910","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}},{"shard":0,"index":"collect_data_store_customer_flow_fact_201911","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}}]},"status":500}
                at org.elasticsearch.client.RestClient$1.completed(RestClient.java:540)
                at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529)
                at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
                at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
                at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
                at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
                at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
                at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
                at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
                at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
                at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
                at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
                at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
                at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
                ... 1 common frames omitted

请问 是如何解决的？ 是否是 需要 配置

curl -X PUT http://10.12.42.153:9200/_cluster/settings -H 'Content-Type: application/json' -d'{
    "persistent" : {
        "search.max_open_scroll_context": 5000
    },
    "transient": {
        "search.max_open_scroll_context": 5000
    }
}
'
————————————————
百度到的是这样：https://blog.csdn.net/weixin_39031707/article/details/93125516

而且我的 java client scroll num 已经是 500 了。
但是我上面的错误是 更新了 数据之后， 查询报错了， 过了一段时间又没有这个错误了

更新的语句 


POST /collect_data_store_customer_flow_fact*/_update_by_query
{
  
   "query": {
    "range": {
      "enterStoreNum": {
        "gt": 0
      
      }
    }
  },
  
  "script" : "ctx._source.enterStoreNum=0"
  
}
然后 导入 新的数据 之后，查询爆了上面错误，而且 查询的 数据 也和导入的实际数据不一致。

可以查看我写的BUG 文章： 错误描述

java 客户端代码：

@Override
    public Object search(FaceSearchDto faceSearchDto) throws IOException {
        String faceCode = faceSearchDto.getFaceCode();
        String indexName = null;
        if (faceSearchDto.getOnlyThisMonth()) {
            String suffixIndex = DateUtil.formatIndexDate(System.currentTimeMillis());
            indexName = esConfig.getDataIndex() + faceCode + suffixIndex;
        } else {
            indexName = esConfig.getDataIndex() + faceCode + "*";
        }
        QueryBuilder queryBuilder = buildCondition(faceSearchDto);

        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        if (queryBuilder != null) {
            searchSourceBuilder.query(queryBuilder);
        }
        if (faceSearchDto.getSortFieldDto() != null) {
            SortFieldDto sortFieldDto = faceSearchDto.getSortFieldDto();
            searchSourceBuilder.sort(sortFieldDto.getMetricCode(), sortFieldDto.getAsc() ? SortOrder.ASC : SortOrder.DESC);
        }
        if (faceSearchDto.isNeedPager()) {
            searchSourceBuilder.size(faceSearchDto.getSize());
            searchSourceBuilder.from(faceSearchDto.getFrom());
            return SearchUtil.search(searchSourceBuilder, indexName, client.getClient());
        } else {
            searchSourceBuilder.size(FaceConstant.SCROLL_NUM);
            return SearchUtil.searchAndScroll(searchSourceBuilder, indexName, client.getClient());
        }
    }


public static List<Map<String, Object>> searchAndScroll(SearchSourceBuilder searchSourceBuilder,
                                                            String indexName, RestHighLevelClient client) throws IOException {

        SearchRequest searchRequest = new SearchRequest();
        searchRequest.source(searchSourceBuilder);
        searchRequest.indices(indexName);
        searchRequest.scroll(new TimeValue(60000));

        SearchResponse scrollResp = client.search(searchRequest, RequestOptions.DEFAULT);
        int totalSize = (int) scrollResp.getHits().getTotalHits().value;
        List<Map<String, Object>> resultList = new ArrayList<>(totalSize);

        while (scrollResp.getHits().getHits().length != 0) {
            for (SearchHit hit : scrollResp.getHits().getHits()) {
                Map<String, Object> map = hit.getSourceAsMap();
                resultList.add(map);
            }
            SearchScrollRequest searchScrollRequest = new SearchScrollRequest();
            searchScrollRequest.scroll(new TimeValue(60000)).scrollId(scrollResp.getScrollId());
            scrollResp = client.scroll(searchScrollRequest, RequestOptions.DEFAULT);
        }
        return resultList;
    }

FaceConstant.SCROLL_NUM=500

Answer 1

rockybean 2019-11-03 17:59:34

这个是你创建的 scroll 太多了，估计是你代码层面有问题。每一次调用 scroll 的创建 api，都会创建一个 scroll 使用的 context，其数目可以通过如下 api 查看：

GET /_nodes/stats/indices/search

其中有一个 open_contexts 的指标。

可以通过 delete api 删除无用的 scroll 来释放

DELETE /_search/scroll

一般不会用到这么多 scroll 的，应该是你代码哪里写的有问题，导致并发创建了很多 search context，你可以检查下。调大配置不是一个好的解决方案

• 提问者 懂渊 #1

  谢谢了。我的API第一个就是没有 去主动清理 scroll 即没有调用Clear-Scroll API 去 清理 。 第二个 你说的并发，我调用的时候确实是有并发调用的，不少于5个并发去调用。