We use cookies to improve your experience with our site.

缓存文档标识符以加速搜索引擎中的查询处理

Caching Document Identifiers to Speedup Query Processing in Search Servers

  • 摘要:
    研究背景 现代搜索引擎在处理海量信息时,面临着高查询负载和严格的时间约束挑战。为了提升效率,搜索引擎采用了多种技术,如动态剪枝和缓存策略。然而,现有的缓存技术在资源消耗和性能提升上仍有优化空间。
    目的 本文的主要目标是提出并评估一种基于文档标识符的缓存策略,旨在通过缓存查询结果中的最低和最高文档标识符,减少后续查询中需要处理的倒排列表范围,从而提升查询处理效率。
    方法 本文通过利用查询结果中文档标识符的最小值(Min)和最大值(Max)信息,确定搜索范围,下次查询时,利用该范围跳过无关文档,从而加速动态剪枝技术的处理速度。本文提出的缓存策略包括静态和动态缓存方案:静态缓存根据查询频率、成本等标准选择查询;动态缓存使用LRU、LFU等策略。此外,本文定义基于成本和跳过信息的访问策略,增强缓存策略性能。
    结果 使用真实数据集(ClueWeb12和AOL查询日志),对比Maxscore(MS)和Block-Max WAND(BW)算法在不同缓存策略下的性能。结果显示:Min/Max缓存使查询处理速度最高提升1.23倍,尾延迟(如P99)最高降低2.0倍;成本感知策略表现最佳,优于传统的频率或近期策略(如LFU、LRU);短查询受益更显著,缓存策略在不同查询长度下均能有效减少处理时间;动态策略在大缓存规模下表现更优,而静态策略在小缓存规模下更具优势。
    结论 Min/Max缓存策略能有效提高查询处理速度,最高可达1.23倍加速,并显著降低高百分位尾延迟。成本感知策略在多数场景下表现更好,动态策略更适合实际应用。该研究为搜索系统的缓存策略提供了新的思路和方法。未来可探索基于术语的Min/Max剪枝策略,以进一步优化性能。

     

    Abstract: Modern search systems have become a fundamental tool for accessing the massive amount of information stored in different repositories. These systems use sophisticated techniques to efficiently process a high volume of queries (thus optimizing energy consumption). One of these techniques is caching, which is implemented at different levels of a search architecture. In this work, we propose a novel caching strategy that speeds up dynamic pruning techniques (such as Maxscore) by exploiting the information of the lowest (Min) and highest (Max) document identifiers that appear as the result of a previously submitted query. We name this technique as Min/Max caching. The idea is to use Min/Max information for pruning the terms’ posting lists in the query before executing the ranking algorithm in a document-at-a-time (DAAT) approach. The proposed technique uses low memory resources, returns safe results, and complements other levels of caching (if present). We also combine the approach with different access policies. Extensive experimentation on real-world data shows that the proposed method increases query processing speedup up to 1.23x and can also reduce high-percentile tail latency (up to 2.0x speedup), an essential requirement for operational scenarios. We evaluate different access and eviction cache policies based on different decision criteria. Our findings confirm that considering the cost of the cached items (cost-aware policies) allows more computation savings.

     

/

返回文章
返回