Lyirs的小屋

主题

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models ​

MemoryRAGContinual LearningKnowledge Graph3.9k+ICML 2025CCF-A俄亥俄州立大学伊利诺伊大学厄巴纳-香槟分校

Gutiérrez B J, Shu Y, Qi W, et al. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. ICML, 2025.

arXiv · PMLR · GitHub

从 RAG 到记忆：面向大语言模型的非参数持续学习 ​

论文脉络HippoRAG 系列
前置论文2024NeurIPS 2024
HippoRAGNeurobiologically Inspired Long-Term Memory for Large Language Models 查看前置

Abstract ​

Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG.

持续获取、组织和利用知识的能力是人类智能的一项关键特征，AI 系统必须逼近这种能力，才能释放其全部潜力。 鉴于大语言模型（LLM）持续学习面临的挑战，检索增强生成（RAG）已经成为引入新信息的主导方式。 然而，RAG 对向量检索的依赖阻碍了它模拟人类长期记忆动态且相互连接的特性。 近期的 RAG 方法使用知识图谱等多种结构来增强向量嵌入，以弥补其中一些缺口，即意义建构与联想性。 然而，它们在更基础的事实记忆任务上的性能明显低于标准 RAG。

We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

我们针对这种意外的性能退化提出 HippoRAG 2；该框架在事实记忆、意义建构记忆和联想记忆任务上全面优于标准 RAG。 HippoRAG 2 建立在 HippoRAG 使用的个性化 PageRank 算法之上，并通过更深入的段落整合以及更有效地在线使用 LLM 对其加以增强。 这种组合使该 RAG 系统更接近人类长期记忆的效能：它在联想记忆任务上比最先进的嵌入模型提升 7%，同时还表现出更强的事实知识与意义建构记忆能力。 这项工作为 LLM 的非参数持续学习铺平了道路。 代码和数据发布于 https://github.com/OSU-NLP-Group/HippoRAG。

1. Introduction ​

HippoRAG 2 evaluation across factual memory, sense-making, and associativity
图1：跨三个关键维度评估持续学习能力：事实记忆（NaturalQuestions、PopQA）、意义建构（NarrativeQA）和联想性（MuSiQue、2Wiki、HotpotQA 与 LV-Eval）。HippoRAG 2 在所有基准类别上均优于其他方法，使其向真正的长期记忆系统更进一步。

In an ever-evolving world, the ability to continuously absorb, integrate, and leverage knowledge is one of the most important features of human intelligence. From lawyers navigating shifting legal frameworks to researchers tracking multifaceted scientific progress, much of our productivity relies on this incredible capacity for continual learning. It is imperative for AI systems to approximate this capability in order to become truly useful human-level assistants.

在不断演变的世界中，持续吸收、整合和利用知识的能力是人类智能最重要的特征之一。 从应对不断变化法律框架的律师，到追踪多方面科学进展的研究人员，我们的大部分生产力都依赖这种非凡的持续学习能力。 AI 系统必须逼近这种能力，才能成为真正有用的人类水平助手。

In recent years, large language models (LLMs) have made remarkable progress in many aspects of human intelligence. However, efforts to endow these models with our evolving long-term memory capabilities have faced significant challenges in both fully absorbing new knowledge and avoiding catastrophic forgetting, due to the complex distributional nature of their parametric knowledge. Retrieval-augmented generation (RAG) has emerged as a way to circumvent these obstacles and allow LLMs to access new information in a non-parametric fashion without altering an LLM's parametric representation. Due to their simplicity and robustness, RAG has quickly become the de facto continual learning solution for production LLM systems. However, their reliance on simple vector retrieval results in the inability to capture two vital aspects of our interconnected long-term memory system: sense-making (the ability to interpret larger, more complex, or uncertain contexts) and associativity (the capacity to draw multi-hop connections between disparate pieces of knowledge).

近年来，大语言模型（LLM）在人类智能的许多方面取得了显著进展。 然而，由于参数知识具有复杂的分布性质，为这些模型赋予不断演化的长期记忆能力时，无论是充分吸收新知识，还是避免灾难性遗忘，都面临重大挑战。 检索增强生成（RAG）成为绕过这些障碍的一种方式，使 LLM 能够在不改变其参数表示的情况下，以非参数方式访问新信息。 凭借简单性和鲁棒性，RAG 很快成为生产级 LLM 系统持续学习的事实标准方案。 然而，对简单向量检索的依赖使其无法捕捉相互连接的长期记忆系统中两个至关重要的方面：意义建构（解释更大、更复杂或不确定上下文的能力）和联想性（在分散知识之间建立多跳联系的能力）。

Several RAG frameworks that engage an LLM to explicitly structure its retrieval corpus have been recently proposed to address these limitations. To enhance sense-making, such structure-augmented RAG methods allow an LLM to either generate summaries or a knowledge graph (KG) structure to link groups of disparate but related passages, thereby improving the RAG system's ability to understand longer and more complex discourse such as long stories. To address the associativity gap, the authors of HippoRAG use the Personalized PageRank algorithm and an LLM's ability to automatically construct a KG and endow the retrieval process with multi-hop reasoning capabilities.

近期提出了若干让 LLM 显式组织检索语料库的 RAG 框架，以解决这些局限。 为了增强意义建构能力，这类结构增强 RAG 方法允许 LLM 生成摘要或知识图谱（KG）结构，将分散但相关的段落组连接起来，从而提高 RAG 系统理解长篇故事等更长、更复杂语篇的能力。 为了弥合联想性缺口，HippoRAG 使用个性化 PageRank 算法和 LLM 自动构建知识图谱的能力，为检索过程赋予多跳推理能力。

Although these methods demonstrate strong performance in both of these more challenging memory tasks, bringing RAG truly closer to human long-term memory requires robustness across simpler memory tasks as well. In order to understand whether these systems could achieve such robustness, we conduct comprehensive experiments that not only simultaneously evaluate their associativity and sense-making capacity through multi-hop QA and large-scale discourse understanding, but also test their factual memory abilities via simple QA tasks, which standard RAG is already well-equipped to handle.

虽然这些方法在上述两类更具挑战性的记忆任务上表现强劲，但要使 RAG 真正接近人类长期记忆，还必须在更简单的记忆任务上具备鲁棒性。 为了了解这些系统能否达到这种鲁棒性，我们开展了全面实验：不仅通过多跳问答和大规模语篇理解同时评估其联想性与意义建构能力，还通过标准 RAG 已经十分擅长处理的简单问答任务测试其事实记忆能力。

As shown in Figure 1, our evaluation reveals that all previous structure-augmented methods underperform against the strongest embedding-based RAG methods available on all three benchmark types. Perhaps unsurprisingly, we find that each method type experiences the largest performance decay in tasks outside its own experimental setup. For example, HippoRAG's performance drops most on large-scale discourse understanding due to its lack of query-based contextualization, while RAPTOR's performance deteriorates substantially on the simple and multi-hop QA tasks due to the noise introduced into the retrieval corpora by its LLM summarization mechanism.

如图1所示，我们的评估表明，此前所有结构增强方法在三类基准上都不及现有最强的基于嵌入的 RAG 方法。 并不令人意外的是，我们发现每类方法在其自身实验设置之外的任务上都会经历最大的性能下降。 例如，由于缺少基于查询的上下文化，HippoRAG 在大规模语篇理解上的性能下降最大；而 RAPTOR 的 LLM 摘要机制会向检索语料库引入噪声，使其在简单和多跳问答任务上的性能显著恶化。

In this work, we leverage this experimental setting to help us address the robustness limitations of these innovative approaches while avoiding the pitfalls of focusing too narrowly on just one task. Our proposed method, HippoRAG 2, leverages the strength of HippoRAG's OpenIE and Personalized PageRank (PPR) methodologies while addressing its query-based contextualization limitations by integrating passages into the PPR graph search process, involving queries more deeply in the selection of KG triples as well as engaging an LLM in the online retrieval process to recognize when retrieved triples are irrelevant.

在这项工作中，我们利用上述实验设置来解决这些创新方法的鲁棒性局限，同时避免过度聚焦单一任务的弊端。 我们提出的 HippoRAG 2 发挥了 HippoRAG 的 OpenIE 与个性化 PageRank（PPR）方法的优势，同时通过将段落整合进 PPR 图搜索过程来解决其基于查询的上下文化局限：查询会更深入地参与知识图谱三元组选择，在线检索过程还会借助 LLM 识别检索到的三元组何时不相关。

Through extensive experiments, we find that this design provides HippoRAG 2 with consistent performance improvements over the most powerful standard RAG methods across the board. More specifically, our approach achieves an average 7 point improvement over standard RAG in associativity tasks while showing no deterioration and even slight improvements in factual memory and sense-making tasks. Furthermore, we show that our method is robust to different retrievers as well as to the use of strong open-source and proprietary LLMs, allowing for a wide degree of usage flexibility. All of these results suggest that HippoRAG 2 is a promising step in the development of a more human-like non-parametric continual learning system for LLMs.

通过大量实验，我们发现这一设计使 HippoRAG 2 在各方面都能持续优于最强的标准 RAG 方法。 更具体地说，我们的方法在联想性任务上平均比标准 RAG 提升 7 分，同时在事实记忆与意义建构任务上没有退化，甚至略有提升。 此外，我们还表明，该方法对不同检索器以及强大的开源和专有 LLM 都具有鲁棒性，从而具备广泛的使用灵活性。 所有这些结果都表明，HippoRAG 2 是构建更类人的 LLM 非参数持续学习系统的一项很有前景的进展。

2. Related Work ​

2.1 Continual Learning for LLMs ​

As the use of LLMs in real-world applications grows, it becomes increasingly important for them to acquire and integrate new knowledge over time while preserving past information---as evidenced by the many benchmarking efforts in this direction. Given the high computational cost of full-scale LLM pretraining, various techniques have been leveraged to endow these models with this continual learning capacity. These approaches generally fall into three categories: continual fine-tuning, model editing, and RAG.

随着 LLM 在现实应用中的使用不断增长，使其能够随时间获取和整合新知识，同时保留既有信息，变得愈发重要；该方向的大量基准工作也印证了这一点。 鉴于完整 LLM 预训练的计算成本很高，人们采用了多种技术为这些模型赋予持续学习能力。 这些方法通常分为三类：持续微调、模型编辑和 RAG。

Continual fine-tuning involves periodically training an LLM on new data. This can be achieved through methods like continual pretraining, instruction tuning, and alignment fine-tuning. While effective in incorporating new linguistic patterns and reasoning skills, continual fine-tuning suffers from catastrophic forgetting, where previously learned knowledge is lost as new data is introduced. Moreover, its computational expense makes frequent updates impractical for real-world applications.

持续微调是指定期使用新数据训练 LLM。 这可以通过持续预训练、指令微调和对齐微调等方法实现。 持续微调虽然能有效纳入新的语言模式与推理技能，却会遭受灾难性遗忘，即在引入新数据时丢失此前学到的知识。 此外，其计算开销使频繁更新在现实应用中并不实际。

Model editing techniques provide a more lightweight alternative by directly modifying specific parameters in the model to update its knowledge. However, these updates have been found to be highly localized, having little effect on information associated with the update that should also be changed.

模型编辑技术通过直接修改模型中的特定参数来更新知识，提供了一种更轻量的替代方案。 然而，研究发现这些更新高度局部化，对本应随之改变的关联信息几乎没有影响。

RAG has emerged as a scalable and practical alternative for continual learning. Instead of modifying the LLM itself, RAG retrieves relevant external information at inference time, allowing for real-time adaptation to new knowledge. We will discuss several aspects of this non-parametric continual learning solution for LLMs in the next section.

RAG 已成为一种可扩展且实用的持续学习替代方案。 RAG 不修改 LLM 本身，而是在推理时检索相关外部信息，从而实时适应新知识。 下一节将讨论这一 LLM 非参数持续学习方案的几个方面。

HippoRAG 2 offline indexing and online retrieval methodology
图2：HippoRAG 2 方法。离线索引阶段使用 LLM 从段落中抽取开放知识图谱三元组，并对短语节点应用同义词检测，短语与段落共同组成开放知识图谱。在线检索阶段，嵌入模型同时为段落和三元组评分，以确定个性化 PageRank（PPR）算法所需的两类种子节点；识别记忆使用 LLM 过滤排名靠前的三元组，PPR 随后在知识图谱上执行基于上下文的检索，为最终问答提供最相关的段落。图中节点颜色表示概率质量，颜色越深，PPR 过程赋予的概率越高。

2.2 Non-Parametric Continual Learning for LLMs ​

Encoder model improvements, particularly with LLM backbones, have significantly enhanced RAG systems by generating high-quality embeddings that better capture semantic relationships, improving retrieval quality for LLM generation. Recent models leverage LLMs, large corpora, improved architectures, and instruction fine-tuning for notable retrieval gains. NV-Embed-v2 serves as the primary comparison in this paper.

编码器模型改进，尤其是采用 LLM 主干后，通过生成能够更好捕捉语义关系的高质量嵌入，显著增强了 RAG 系统，并提高了面向 LLM 生成的检索质量。 近期模型利用 LLM、大规模语料库、改进的架构和指令微调，取得了显著的检索增益。 本文以 NV-Embed-v2 作为主要比较对象。

Sense-making is the ability to understand large-scale or complex events, experiences, or data. Standard RAG methods are limited in this capacity since they require integrating information from disparate passages, and thus, several RAG frameworks have been proposed to address it. RAPTOR and GraphRAG both generate summaries that integrate their retrieval corpora. However, they follow distinct processes for detecting what to summarize and at what granularity. While RAPTOR uses a Gaussian Mixture Model to detect document clusters to summarize, GraphRAG uses a graph community detection algorithm that can summarize documents, entity clusters with relations, or a combination of these elements. LightRAG employs a dual-level retrieval mechanism to enhance comprehensive information retrieval capabilities in both low-level and high-level knowledge, integrating graph structures with vector retrieval.

意义建构是理解大规模或复杂事件、经历或数据的能力。 标准 RAG 方法在这方面存在局限，因为它们需要整合分散段落中的信息，因此人们提出了若干 RAG 框架来解决这一问题。 RAPTOR 和 GraphRAG 都会生成整合检索语料库的摘要。 不过，它们在检测摘要对象及其粒度时采用不同流程。 RAPTOR 使用高斯混合模型检测需要摘要的文档簇，而 GraphRAG 使用图社区检测算法，可以对文档、带关系的实体簇或这些元素的组合进行摘要。 LightRAG 采用双层检索机制，将图结构与向量检索结合，以增强低层与高层知识的综合信息检索能力。

Although both GraphRAG and LightRAG use a KG just like our HippoRAG 2 approach, our KG is used to aid in the retrieval process rather than to expand the retrieval corpus itself. This allows HippoRAG 2 to introduce less LLM-generated noise, which deteriorates the performance of these methods in single and multi-hop QA tasks.

虽然 GraphRAG 和 LightRAG 与 HippoRAG 2 一样都使用知识图谱，但我们的知识图谱用于辅助检索过程，而不是扩展检索语料库本身。 因此，HippoRAG 2 引入的 LLM 生成噪声更少；这类噪声会降低上述方法在单跳和多跳问答任务上的性能。

Associativity is the capacity to draw multi-hop connections between disparate facts for efficient retrieval. It is an important part of continual learning, which standard RAG cannot emulate due to its reliance on independent vector retrieval. HippoRAG is the only RAG framework that has addressed this property by leveraging the PPR algorithm over an explicitly constructed open KG. HippoRAG 2 is closely inspired by HippoRAG, which allows it to perform very well on multi-hop QA tasks. However, its more comprehensive integration of passages, queries, and triples allows it to have a more comprehensive performance across sense-making and factual memory tasks as well.

联想性是为了高效检索而在分散事实之间建立多跳联系的能力。 它是持续学习的重要组成部分；标准 RAG 依赖彼此独立的向量检索，因而无法模拟这种能力。 HippoRAG 是唯一通过在显式构建的开放知识图谱上运行 PPR 算法来处理这一特性的 RAG 框架。 HippoRAG 2 深受 HippoRAG 启发，因此在多跳问答任务上表现出色。 不过，它对段落、查询和三元组进行了更全面的整合，因此在意义建构与事实记忆任务上也能取得更全面的表现。

3. HippoRAG 2 ​

3.1 Overview ​

HippoRAG is a neurobiologically inspired long-term memory framework for LLMs in which each component is inspired by its neurobiological analog for human memory. The framework consists of three primary components: 1) an LLM that acts as an artificial neocortex, 2) a KG and the Personalized PageRank algorithm to mirror the auto-associative qualities of the hippocampus and 3) a retrieval encoder that links these two components, reflecting one of the functions of the parahippocampal regions. These components collaborate to replicate the interactions observed in human long-term memory.

HippoRAG 是一个受神经生物学启发的 LLM 长期记忆框架，其中每个组件都借鉴了人类记忆中相应的神经生物学结构。 该框架由三个主要组件构成：1）充当人工新皮层的 LLM；2）用于模拟海马体自联想特性的知识图谱与个性化 PageRank 算法；3）连接前两个组件、对应海马旁区某项功能的检索编码器。 这些组件协同工作，以复现人类长期记忆中的交互。

HippoRAG's offline indexing process uses an LLM to process passages into KG triples, which are then incorporated into the KG, our artificial hippocampal index. Meanwhile, the retrieval encoder is responsible for detecting synonymy to interconnect information. In HippoRAG's online retrieval process, the LLM neocortex extracts named entities from a query while the retrieval encoder finds their most similar counterparts in the KG. Then, the nodes in the KG corresponding to these entities, which we refer to as seed nodes, are used to run the Personalized PageRank (PPR) algorithm. More specifically, these seed nodes are used to assign the reset probabilities within PPR, which alter the original PageRank algorithm to distribute probability towards the seed nodes and their neighborhoods, enabling HippoRAG's context-based retrieval. Although HippoRAG seeks to construct memory from non-parametric RAG, its effectiveness is hindered by a critical flaw: an entity-centric approach that causes context loss during both indexing and inference, as well as difficulties in semantic matching.

HippoRAG 的离线索引过程使用 LLM 将段落处理为知识图谱三元组，再把它们纳入作为人工海马体索引的知识图谱。 与此同时，检索编码器负责检测同义关系，以连接不同信息。 在 HippoRAG 的在线检索过程中，LLM 新皮层从查询中抽取命名实体，而检索编码器在知识图谱中寻找与其最相似的对应项。 随后，知识图谱中与这些实体对应的节点，即我们所称的种子节点，被用于运行个性化 PageRank（PPR）算法。 更具体地说，这些种子节点用于分配 PPR 中的重置概率；该机制修改原始 PageRank 算法，使概率向种子节点及其邻域分布，从而实现 HippoRAG 的基于上下文检索。 虽然 HippoRAG 试图从非参数 RAG 构建记忆，但其有效性受到一个关键缺陷的阻碍：以实体为中心的方法会在索引和推理期间造成上下文丢失，并带来语义匹配困难。

Built on the neurobiologically inspired long-term memory framework proposed in HippoRAG, the structure of HippoRAG 2 follows a similar two-stage process: offline indexing and online retrieval, as shown in Figure 2. Additionally, however, HippoRAG 2 introduces several key refinements that improve its alignment with human memory mechanisms:

HippoRAG 2 建立在 HippoRAG 提出的神经生物学启发式长期记忆框架之上，其结构遵循类似的两阶段过程：离线索引与在线检索，如图2所示。 除此之外，HippoRAG 2 还引入了若干关键改进，使其与人类记忆机制更加一致：

  1. It seamlessly integrates conceptual and contextual information within the KG, enhancing the comprehensiveness and atomicity of the constructed index (Section 3.2).
  2. It facilitates more context-aware retrieval by leveraging the KG structure beyond isolated KG nodes (Section 3.3).
  3. It incorporates recognition memory to improve seed node selection for graph search (Section 3.4).

In the following sections, we introduce the pipeline in more detail and elaborate on each of these refinements.

  1. 它在知识图谱内无缝整合概念信息与上下文信息，提升所构建索引的全面性和原子性（第 3.2 节）。
  2. 它不再局限于孤立的知识图谱节点，而是利用知识图谱结构实现更具上下文感知能力的检索（第 3.3 节）。
  3. 它引入识别记忆，以改进图搜索的种子节点选择（第 3.4 节）。

下面几节将更详细地介绍整个流程，并逐一阐述这些改进。

Offline Indexing.

离线索引。

  1. HippoRAG 2, just as HippoRAG, leverages an LLM to extract triples from each passage using OpenIE, which allows the relations and entities to be generated without any constraints or schema.

These triples are then arranged into our schema-less KG or hippocampal index. We call the subject or object of these triples phrases and the edge connecting them relation edge.

  1. Next, the retrieval encoder identifies synonyms by evaluating phrase pairs within the KG, detecting those with vector similarity above a predefined threshold, and adding synonym edge between such pair.

This process enables the KG to link synonyms across different passages, facilitating the integration of both old and new knowledge during learning.

  1. Finally, this phrase-based KG is combined with the original passages, allowing the final open KG to incorporate both conceptual and contextual information (Section 3.2).
  1. HippoRAG 2 与 HippoRAG 一样，利用 LLM 通过 OpenIE 从每个段落中抽取三元组，使关系和实体能够不受任何约束或模式限制地生成。

随后，这些三元组被组织成无模式知识图谱，即海马体索引。 我们把这些三元组的主语或宾语称为短语，把连接它们的边称为关系边。

  1. 接着，检索编码器评估知识图谱中的短语对，检测向量相似度高于预定义阈值的短语，并在这类短语对之间添加同义边，以识别同义关系。

该过程使知识图谱能够连接不同段落中的同义词，促进学习过程中旧知识与新知识的整合。

  1. 最后，这个基于短语的知识图谱与原始段落结合，使最终的开放知识图谱同时包含概念信息和上下文信息（第 3.2 节）。

Online Retrieval.

在线检索。

  1. The query is linked to relevant triples and passages using the encoder, identifying nodes that could be used as seed nodes for graph search (Section 3.3).
  2. During triple linkage, the recognition memory functions as a filter, ensuring only relevant triples are retained from the retrieved set as the final seed nodes (Section 3.4).
  3. These final seed nodes are then used to assign reset probabilities within the PPR algorithm, enabling its context-aware retrieval and refining the linking results to retrieve the most relevant passages.
  4. Finally, the retrieved passages serve as contextual inputs for the final QA task.

Next, we describe each of the improvements in HippoRAG 2 in more detail.

  1. 使用编码器把查询链接到相关三元组和段落，并识别可用作图搜索种子节点的节点（第 3.3 节）。
  2. 在三元组链接期间，识别记忆充当过滤器，确保仅从检索集合中保留相关三元组，作为最终种子节点（第 3.4 节）。
  3. 随后使用这些最终种子节点分配 PPR 算法中的重置概率，实现上下文感知检索，并细化链接结果以检索最相关的段落。
  4. 最后，检索到的段落作为最终问答任务的上下文输入。

接下来，我们将更详细地介绍 HippoRAG 2 的各项改进。

3.2 Dense-Sparse Integration ​

The nodes in the HippoRAG KG primarily consist of phrases describing concepts, which we refer to as phrase nodes in this paper. This graph structure introduces limitations related to the concept-context tradeoff. Concepts are concise and easily generalizable but often entail information loss. In contrast, context provide specific circumstances that shape the interpretation and application of these concepts, enriching semantics but increasing complexity.

HippoRAG 知识图谱中的节点主要由描述概念的短语构成，本文称其为短语节点。 这种图结构带来了与概念—上下文权衡有关的局限。 概念简洁且容易泛化，但通常会造成信息损失。 相比之下，上下文提供塑造这些概念解释和应用方式的具体情境，虽然丰富了语义，却也增加了复杂性。

However, in human memory, concepts and contexts are intricately interconnected. The dense and sparse coding theory offers insights into how the brain represents and processes information at different granularities. Dense coding encodes information through the simultaneous activation of many neurons, resulting in a distributed and redundant representation. Conversely, sparse coding relies on minimal neural activation, engaging only a small subset of neurons to enhance efficiency and storage compactness.

然而，在人类记忆中，概念与上下文彼此紧密连接。 稠密编码与稀疏编码理论为理解大脑如何在不同粒度上表示和处理信息提供了启示。 稠密编码通过同时激活许多神经元来编码信息，形成分布式且冗余的表示。 相反，稀疏编码依赖最少的神经激活，仅调动少量神经元，以提高效率和存储紧凑性。

Inspired by the dense-sparse integration observed in the human brain, we treat the phrase node as a form of sparse coding for the extracted concepts, while incorporating dense coding into our KG to represent the context from which these concepts originate. First, we adopt an encoding approach similar to how phrases are encoded, using the embedding model. These two types of coding are then integrated in a specific manner within the KG. Unlike the document ensemble in HippoRAG, which simply aggregates scores from graph search and embedding matching, we enhance the KG by introducing passage nodes, enabling more seamless integration of contextual information. This approach retains the same offline indexing process as HippoRAG while enriching the graph structure with additional nodes and edges related to passages during construction. Specifically, each passage in the corpus is treated as a passage node, with the context edge labeled “contains” connecting the passage to all phrases derived from this passage.

受人脑中稠密—稀疏整合的启发，我们把短语节点视为抽取概念的一种稀疏编码形式，同时将稠密编码纳入知识图谱，以表示这些概念所源自的上下文。 首先，我们使用嵌入模型，采用与短语编码相似的编码方式。 随后，这两类编码以特定方式在知识图谱中整合。 HippoRAG 的文档集成只是聚合图搜索和嵌入匹配的分数；与此不同，我们通过引入段落节点来增强知识图谱，从而更无缝地整合上下文信息。 该方法保留 HippoRAG 的离线索引过程，同时在构图期间使用与段落相关的额外节点和边丰富图结构。 具体而言，语料库中的每个段落都被视为一个段落节点，并通过标记为“contains”的上下文边连接到由该段落派生的所有短语。

3.3 Deeper Contextualization ​

Building upon the discussion of the concept-context tradeoff, we observe that query parsing in HippoRAG, which relies on Named Entity Recognition (NER), is predominantly concept-centric, often overlooking the contextual alignment within the KG. This entity-focused approach to extraction and indexing introduces a strong bias toward concepts, leaving many contextual signals underutilized. To address this limitation, we explore and evaluate different methods for linking queries to the KG, aiming to more effectively align query semantics with the starting nodes of graph searches. Specifically, we consider three approaches:

基于前面对概念—上下文权衡的讨论，我们观察到，HippoRAG 依赖命名实体识别（NER）的查询解析主要以概念为中心，常常忽略知识图谱中的上下文对齐。 这种聚焦实体的抽取与索引方法对概念产生了强烈偏置，使许多上下文信号未被充分利用。 为解决这一局限，我们探索并评估不同的查询—知识图谱链接方法，旨在更有效地将查询语义与图搜索的起始节点对齐。 具体而言，我们考虑三种方法：

  1. NER to Node: This is the original method used in HippoRAG, where entities are extracted from the query and subsequently matched with nodes in the KG using text embeddings.
  2. Query to Node: Instead of extracting individual entities, we leverage text embeddings to match the entire query directly to nodes in the KG.
  3. Query to Triple: To incorporate richer contextual information from the KG, we match the entire query to triples within the graph using text embeddings.

Since triples encapsulate fundamental contextual relationships among concepts, this method provides a more comprehensive understanding of the query's intent. By default, HippoRAG 2 adopts the query-to-triple approach, and we evaluate all three methods later (Section 6.1).

  1. NER to Node：这是 HippoRAG 使用的原始方法，从查询中抽取实体，再使用文本嵌入将其与知识图谱节点匹配。
  2. Query to Node：不抽取单个实体，而是利用文本嵌入把整个查询直接与知识图谱节点匹配。
  3. Query to Triple：为了纳入知识图谱中更丰富的上下文信息，我们使用文本嵌入把整个查询与图内三元组匹配。

三元组封装了概念之间的基本上下文关系，因此该方法能够更全面地理解查询意图。 HippoRAG 2 默认采用 Query to Triple 方法，稍后将评估全部三种方法（第 6.1 节）。

3.4 Recognition Memory ​

Recall and recognition are two complementary processes in human memory retrieval. Recall involves actively retrieving information without external cues, while recognition relies on identifying information with the help of external stimuli. Inspired by this, we model the query-to-triple retrieval as a two-step process.

回忆与识别是人类记忆检索中两个互补过程。 回忆是在没有外部线索时主动检索信息，而识别则依靠外部刺激来辨认信息。 受此启发，我们把查询到三元组的检索建模为两步过程。

  1. Query to Triple: We use the embedding model to retrieve the top-k triples T of the graph as described in Section 3.3.
  2. Triple Filtering: We use LLMs to filter retrieved T and generate triples T′⊆T.

The detailed prompts are shown in the appendix.

  1. Query to Triple：如第 3.3 节所述，我们使用嵌入模型检索图中排名前 k 的三元组 T。
  2. Triple Filtering：我们使用 LLM 过滤检索到的 T，并生成三元组集合 T′⊆T。

详细提示见附录。

3.5 Online Retrieval ​

We summarize the online retrieval process in HippoRAG 2 after introducing the above improvements. The task involves selecting seed nodes and assigning reset probabilities for retrieval. HippoRAG 2 identifies phrase nodes from filtered triples generated by query-to-triple and recognition memory. If no triples are available, it directly retrieves top-ranked passages using the embedding model. Otherwise, up to k phrase nodes are selected based on their average ranking scores across filtered triples they originate.

介绍完上述改进后，我们总结 HippoRAG 2 的在线检索过程。 该任务需要选择种子节点，并为检索分配重置概率。 HippoRAG 2 从 Query to Triple 与识别记忆生成的过滤后三元组中识别短语节点。 如果没有可用三元组，它就使用嵌入模型直接检索排名靠前的段落。 否则，系统根据短语节点在其来源过滤三元组中的平均排名分数，最多选择 k 个短语节点。

All passage nodes are also taken as seed nodes, as broader activation improves multi-hop reasoning. Reset probabilities are assigned based on ranking scores for phrase nodes, while passage nodes receive scores proportional to their embedding similarity, adjusted by a weight factor (Section 6.2) to balance the influence between phrase nodes and passage nodes. The PPR search is then executed, and passages are ranked by their PageRank scores, with the top-ranked passages used for downstream QA. An example of the pipeline is in the appendix and the PPR initialization is detailed in the appendix.

所有段落节点也都被作为种子节点，因为更广泛的激活有助于多跳推理。 短语节点的重置概率依据排名分数分配；段落节点获得与其嵌入相似度成比例的分数，再使用权重因子（第 6.2 节）调整，以平衡短语节点和段落节点的影响。 随后执行 PPR 搜索，并根据 PageRank 分数对段落排序，将排名靠前的段落用于下游问答。 流程示例见附录，PPR 初始化细节也见附录。

表1：数据集统计。
NQPopQAMuSiQue2WikiHotpotQALV-EvalNarrativeQA
Num of queries1,0001,0001,0001,0001,000124293
Num of passages9,6338,67611,6566,1199,81122,8494,111

4. Experimental Setup ​

4.1 Baselines ​

We select three different types of baselines for comparison. We include three simple baselines: the classic BM25 baseline as well as Contriever and GTR, two popular dense embedding retrievers.

我们选择三类不同的基线进行比较。 简单基线包括经典的 BM25，以及 Contriever 和 GTR 两个常用的稠密嵌入检索器。

Our second baseline category includes some of the largest embedding models available (7B) that demonstrate strong performance on the BEIR leaderboard: Alibaba-NLP/GTE-Qwen2-7B-Instruct, GritLM/GritLM-7B, and nvidia/NV-Embed-v2.

第二类基线包括若干现有规模最大的嵌入模型（7B），它们在 BEIR 排行榜上表现强劲：Alibaba-NLP/GTE-Qwen2-7B-Instruct、GritLM/GritLM-7B 和 nvidia/NV-Embed-v2。

In our final baseline category, we include four structure-augmented RAG methods. RAPTOR organizes the retrieval corpus into a hierarchical structure based on semantic similarity. GraphRAG and LightRAG leverage a KG structure like ours to generate high-level summaries of the concepts present in the corpus. Finally, HippoRAG uses a KG as well but integrates knowledge using PPR rather than summarization.

最后一类基线包括四种结构增强 RAG 方法。 RAPTOR 根据语义相似度把检索语料库组织成层次结构。 GraphRAG 和 LightRAG 与我们一样利用知识图谱结构，生成语料库所含概念的高层摘要。 最后，HippoRAG 也使用知识图谱，但通过 PPR 而非摘要来整合知识。

4.2 Datasets ​

To evaluate how well RAG systems retain factual memory while enhancing associativity and sense-making, we select datasets that correspond to three critical challenge types.

为了评估 RAG 系统在增强联想性与意义建构能力的同时，能够多好地保留事实记忆，我们选择了对应三种关键挑战类型的数据集。

  1. Simple QA primarily evaluates the ability to recall and retrieve factual knowledge accurately.
  2. Multi-hop QA measures associativity by requiring the model to connect multiple pieces of information to derive an answer.
  3. Discourse understanding evaluates sense-making by testing the capability to interpret and reason over lengthy, complex narratives.
  1. 简单问答主要评估准确回忆和检索事实知识的能力。
  2. 多跳问答要求模型连接多条信息以推导答案，从而衡量联想性。
  3. 语篇理解通过测试解释和推理长篇复杂叙事的能力来评估意义建构。

We will now list the datasets chosen for each category and describe them in detail. The statistics for our sampled datasets are summarized in Table 1.

下面列出每类任务所选择的数据集并作详细介绍。 采样数据集的统计信息汇总于表1。

Simple QA. This common type of QA task primarily involves questions centered around individual entities, making it particularly well-suited for embedding models to retrieve relevant contextual information intuitively. We randomly collect 1,000 queries from the NaturalQuestions (NQ) dataset, which contains real user questions with a wide range of topics. Additionally, we select 1,000 queries from PopQA, with the corpus derived from the December 2021 Wikipedia dump. Both datasets offer straightforward QA pairs, enabling evaluation of single-hop QA capabilities in RAG systems. Notably, PopQA from Wikipedia is especially entity-centric, with entities being less frequent than NaturalQuestions, making it an excellent resource for evaluating entity recognition and retrieval in simple QA tasks.

简单问答。 这类常见问答任务主要围绕单个实体提出问题，因此特别适合评估嵌入模型能否直观地检索相关上下文信息。 我们从 NaturalQuestions（NQ）数据集中随机收集 1,000 个查询；该数据集包含主题广泛的真实用户问题。 此外，我们从 PopQA 中选择 1,000 个查询，其语料库来自 2021 年 12 月的 Wikipedia 转储。 两个数据集都提供直接的问答对，可用于评估 RAG 系统的单跳问答能力。 值得注意的是，源自 Wikipedia 的 PopQA 尤其以实体为中心，所含实体的出现频率低于 NaturalQuestions，因此非常适合评估简单问答任务中的实体识别与检索。

Multi-hop QA. We randomly collect 1,000 queries from MuSiQue, 2WikiMultihopQA, and HotpotQA following HippoRAG, all requiring multi-passage reasoning. Additionally, we include all 124 queries from LV-Eval (hotpotwikiqa-mixup 256k), a challenging dataset designed to minimize knowledge leakage and reduce overfitting through keyword and phrase replacements. Thus, unlike Wikipedia-based datasets, LV-Eval better evaluates the model's ability to synthesize knowledge from different sources effectively. For corpus collection, we segment long-form contexts of LV-Eval into shorter passages while maintaining the same RAG setup as other multi-hop datasets.

多跳问答。 我们沿用 HippoRAG 的设置，从 MuSiQue、2WikiMultihopQA 和 HotpotQA 中各随机收集 1,000 个查询；这些查询都需要跨段落推理。 此外，我们纳入 LV-Eval（hotpotwikiqa-mixup 256k）的全部 124 个查询；这是一个通过替换关键词和短语来尽量减少知识泄漏并降低过拟合的高难度数据集。 因此，与基于 Wikipedia 的数据集不同，LV-Eval 能更好地评估模型有效综合不同来源知识的能力。 在收集语料库时，我们把 LV-Eval 的长篇上下文切分为较短段落，同时保持与其他多跳数据集相同的 RAG 设置。

Discourse Understanding. This category consists of only NarrativeQA, a QA dataset that contains questions requiring a cohesive understanding of a full-length novel. This dataset's focus on large-scale discourse understanding allows us to leverage it in our evaluation of sense-making in our chosen baselines and our own method. We randomly select 10 lengthy documents and their corresponding 293 queries from NarrativeQA and collect a retrieval corpus just as in the above LV-Eval dataset.

语篇理解。 该类别仅包含 NarrativeQA；这个问答数据集的问题需要对整部长篇小说形成连贯理解。 该数据集聚焦大规模语篇理解，因此可用于评估所选基线与我们方法的意义建构能力。 我们从 NarrativeQA 随机选择 10 篇长文档及其对应的 293 个查询，并采用与上述 LV-Eval 数据集相同的方式收集检索语料库。

表2：RAG 基准上的问答性能（F1 分数），问答阅读器为 Llama-3.3-70B-Instruct。No retrieval 表示评估阅读器的参数知识。所有结构增强 RAG 基线和 HippoRAG 2 均使用 Llama-3.3-70B-Instruct 生成结构，并使用 NV-Embed-v2 作为检索器。最佳结果以粗体表示，次佳结果以下划线表示。我们使用 bootstrap 统计检验评估显著性；† 表示 HippoRAG 2 显著优于最佳 NV-Embed-v2 基线（p < 0.05）。
RetrievalSimple QAMulti-Hop QADiscourse
Understanding		Avg
NQPopQAMuSiQue2WikiHotpotQALV-EvalNarrativeQA
Simple Baselines
None54.932.526.142.847.36.012.938.4
Contriever58.953.131.341.962.38.119.746.9
BM2559.049.928.851.263.45.918.347.7
GTR (T5-base)59.956.234.652.862.87.119.950.4
Large Embedding Models
GTE-Qwen2-7B-Instruct62.056.340.960.071.07.121.354.9
GritLM-7B61.355.844.860.673.39.823.956.1
NV-Embed-v2 (7B)61.955.745.761.575.39.825.757.0
Structure-Augmented RAG
RAPTOR50.756.228.952.169.55.021.448.8
GraphRAG46.948.138.558.668.611.223.049.6
LightRAG16.62.41.611.62.41.03.76.6
HippoRAG55.355.935.171.863.58.416.353.1
HippoRAG 263.3†56.248.6†71.0†75.512.9†25.959.8
表3：RAG 基准上的检索性能（passage recall@5）。* 表示原论文报告的结果。为公平比较，所比较的结构增强 RAG 方法均使用与我们相同的 LLM 和检索器复现。GraphRAG 与 LightRAG 不直接产生段落检索结果，因此未列出。
RetrievalSimple QAMulti-Hop QAAvg
NQPopQAMuSiQue2WikiHotpotQA
Simple Baselines
BM2556.135.743.565.374.855.1
Contriever54.643.246.657.575.355.4
GTR (T5-base)63.449.449.167.973.960.7
Large Embedding Models
GTE-Qwen2-7B-Instruct74.350.663.674.889.170.5
GritLM-7B76.650.165.976.092.472.2
NV-Embed-v2 (7B)75.451.069.776.594.573.4
Structure-Augmented RAG
RAPTOR68.348.757.866.286.965.6
HippoRAG*--51.989.177.7-
HippoRAG (reproduced)44.453.853.290.477.363.8
HippoRAG 278.051.774.790.496.378.2

4.3 Metrics ​

Following HippoRAG, we use passage recall@5 to evaluate the retrieval task. For the QA task, we follow evaluation metrics from MuSiQue to calculate token-based F1 scores.

沿用 HippoRAG，我们使用 passage recall@5 评估检索任务。 对于问答任务，我们沿用 MuSiQue 的评估指标，计算基于 token 的 F1 分数。

4.4 Implementation Details ​

For HippoRAG 2, we use the open-source Llama-3.3-70B-Instruct as both the extraction (NER and OpenIE) and triple filtering model, and we use nvidia/NV-Embed-v2 as the retriever. We also reproduce the compared structure-augmented RAG methods using the same extractor and retriever for a fair comparison. For the triple filter, we use DSPy MIPROv2 optimizer and Llama-3.3-70B-Instruct to tune the prompt, including the instructions and demonstrations. The resulting prompt is shown in the appendix.

对于 HippoRAG 2，我们使用开源 Llama-3.3-70B-Instruct 同时作为抽取模型（NER 与 OpenIE）和三元组过滤模型，并使用 nvidia/NV-Embed-v2 作为检索器。 为了公平比较，我们还使用相同的抽取器和检索器复现所比较的结构增强 RAG 方法。 对于三元组过滤器，我们使用 DSPy MIPROv2 优化器和 Llama-3.3-70B-Instruct 调整提示，包括指令和示例。 所得提示见附录。

We use top-5 triples ranked by retriever for filtering. Our QA module uses the top-5 retrieved passages as context for an LLM (GPT-4o-mini or Llama-3.3-70B-Instruct) to generate the final answer. For hyperparameters, we follow the default settings from HippoRAG. More implementation and hyperparameter details can be found in the appendix.

我们使用检索器排名前 5 的三元组进行过滤。 问答模块把检索到的前 5 个段落作为 LLM（GPT-4o-mini 或 Llama-3.3-70B-Instruct）的上下文，以生成最终答案。 超参数沿用 HippoRAG 的默认设置。 更多实现和超参数细节见附录。

5. Results ​

We now present our main QA and retrieval experimental results, where the QA process uses retrieved results as its context. More detailed experimental results are presented in the appendix. The statistics for all constructed KGs are shown in the appendix.

下面给出问答与检索的主要实验结果，其中问答过程使用检索结果作为上下文。 更详细的实验结果见附录。 所有已构建知识图谱的统计信息也见附录。

QA Performance. Table 2 presents the QA performance of various retrievers across multiple RAG benchmarks using Llama-3.3-70B-Instruct as the QA reader. HippoRAG 2 achieves the highest average F1 score, demonstrating robustness across different settings. Large embedding models outperform smaller ones, with NV-Embed-v2 (7B) scoring 6.6% higher on average than GTR (T5-base). These models also surpass structure-augmented RAG methods with lower computational costs but excel mainly in simple QA while struggling in complex cases.

问答性能。 表2展示了以 Llama-3.3-70B-Instruct 作为问答阅读器时，不同检索器在多个 RAG 基准上的问答性能。 HippoRAG 2 取得最高的平均 F1 分数，展现出跨不同设置的鲁棒性。 大型嵌入模型优于小型模型，其中 NV-Embed-v2（7B）的平均分比 GTR（T5-base）高 6.6%。 这些模型还以较低计算成本超越结构增强 RAG 方法，但它们主要擅长简单问答，在复杂场景中仍面临困难。

Notably, HippoRAG 2 outperforms NV-Embed-v2 by 9.5% F1 on 2Wiki and by 3.1% on the challenging LV-Eval dataset. Compared to HippoRAG, HippoRAG 2 shows even greater improvements, validating its neuropsychology-inspired approach. These results highlight HippoRAG 2 as a state-of-the-art RAG system that enhances both retrieval and QA performance while being effectively powered by an open-source model. The table in the appendix presents additional QA results (EM and F1) using Llama or GPT-4o-mini as the QA reader, along with an extractor or triple filter. GPT-4o-mini follows Llama's trend, with NV-Embed-v2 outperforming structure-augmented methods in most cases, except for HippoRAG in multi-hop QA. HippoRAG 2 consistently outperforms all other methods across nearly all settings. An analysis of the computational resources (tokens, time and memory) required for each method can be found in the appendix.

值得注意的是，HippoRAG 2 在 2Wiki 上的 F1 比 NV-Embed-v2 高 9.5%，在高难度 LV-Eval 数据集上高 3.1%。 与 HippoRAG 相比，HippoRAG 2 的提升更大，验证了其受神经心理学启发的方法。 这些结果表明，HippoRAG 2 是一个最先进的 RAG 系统，它能同时提升检索与问答性能，并且可由开源模型有效驱动。 附录中的表格给出了使用 Llama 或 GPT-4o-mini 作为问答阅读器并结合抽取器或三元组过滤器时的额外问答结果（EM 和 F1）。 GPT-4o-mini 呈现出与 Llama 相同的趋势：除多跳问答中的 HippoRAG 外，NV-Embed-v2 在多数情况下优于结构增强方法。 HippoRAG 2 在几乎所有设置中都持续优于其他全部方法。 各方法所需计算资源（token、时间和内存）的分析见附录。

Retrieval Performance. We report retrieval results for datasets with supporting passage annotations and models that explicitly retrieve passages in Table 3. Large embedding models (7B) significantly outperform classic smaller LM-based models like Contriever and GTR, achieving at least a 9.8% higher F1 score. While our reproduction of HippoRAG using Llama-3.3-70B-Instruct and NV-Embed-v2 shows slight improvements over the original paper, the gains are minimal, with only a 1.3% increase in F1.

检索性能。 表3报告了带支持段落标注的数据集以及显式检索段落的模型的检索结果。 大型嵌入模型（7B）显著优于 Contriever 和 GTR 等经典的小型语言模型方法，F1 至少高 9.8%。 虽然我们使用 Llama-3.3-70B-Instruct 和 NV-Embed-v2 复现的 HippoRAG 比原论文略有提升，但增益很小，F1 仅提高 1.3%。

Although HippoRAG excels in entity-centric retrieval, achieving the highest recall@5 on PopQA, it generally lags behind recent dense retrievers and HippoRAG 2. Notably, HippoRAG 2 achieves the highest recall scores across most datasets, with substantial improvements of 5.0% and 13.9% in Recall@5 on MuSiQue and 2Wiki, respectively, compared to the strongest dense retriever, NV-Embed-v2.

虽然 HippoRAG 擅长以实体为中心的检索，在 PopQA 上取得最高 recall@5，但总体上落后于近期的稠密检索器和 HippoRAG 2。 值得注意的是，HippoRAG 2 在大多数数据集上取得最高召回率；与最强稠密检索器 NV-Embed-v2 相比，它在 MuSiQue 和 2Wiki 上的 Recall@5 分别显著提高 5.0% 和 13.9%。

表4：消融实验。我们在多跳问答基准上报告 passage recall@5，并比较最终设计在图链接、图构建和三元组过滤方面的若干替代方案。
MuSiQue2WikiHotpotQAAvg
HippoRAG 274.790.496.387.1
w/ NER to node53.891.278.874.6
w/ Query to node44.965.568.359.6
w/o Passage Node63.790.388.981.0
w/o Filter73.090.795.486.4
表5：重置概率因子。不同段落节点权重因子下，MuSiQue 开发集与 NaturalQuestions（NQ）开发集的 passage recall@5；每个集合均包含 1,000 个查询。
Weight0.010.050.10.30.5
MuSiQue79.980.579.878.477.9
NQ75.676.976.976.776.4

6. Discussions ​

6.1 Ablation Study ​

We design ablation experiments for the proposed linking method, graph construction method, and triple filtering method, with the results reported in Table 4. Each introduced mechanism boosts HippoRAG 2. First, the linking method with deeper contextualization leads to significant performance improvements.

我们针对所提出的链接方法、图构建方法和三元组过滤方法设计了消融实验，结果见表4。 引入的每项机制都会提升 HippoRAG 2。 首先，具有更深上下文化能力的链接方法带来了显著的性能提升。

Notably, we do not apply a filtering process to the NER-to-node or query-to-node methods; however, the query-to-triple approach, regardless of whether filtering is applied, consistently outperforms the other two linking strategies. On average, query-to-triple improves Recall@5 by 12.5% compared to NER-to-node. Moreover, query-to-node does not provide an advantage over NER-to-node, as queries and KG nodes operate at different levels of granularity, whereas both NER results and KG nodes correspond to phrase-level representations.

值得注意的是，我们没有对 NER-to-node 或 Query to Node 方法应用过滤过程；然而，无论是否应用过滤，Query to Triple 都持续优于另外两种链接策略。 平均而言，与 NER-to-node 相比，Query to Triple 将 Recall@5 提高了 12.5%。 此外，Query to Node 并不优于 NER-to-node，因为查询与知识图谱节点处于不同粒度，而 NER 结果和知识图谱节点都对应短语级表示。

6.2 Controlling Reset Probabilities ​

When setting the reset probability before starting PPR, we find that it is necessary to balance the reset probabilities between two types of nodes: phrase nodes and passage nodes. Specifically, the reset probability of all passage nodes is multiplied by a weight factor to balance the importance of two types of nodes during PPR. Here, we present the results obtained on the validation set in Table 5, which shows that this factor is crucial for the PPR results. Considering the model performance across different scenarios, we set the factor to be 0.05 by default.

在启动 PPR 前设置重置概率时，我们发现有必要在两类节点之间平衡重置概率：短语节点与段落节点。 具体而言，所有段落节点的重置概率都会乘以一个权重因子，以平衡 PPR 过程中两类节点的重要性。 表5给出了验证集结果，表明该因子对 PPR 结果至关重要。 综合不同场景下的模型性能，我们默认把该因子设为 0.05。

6.3 Robustness to Corpus Expansion ​

As RAG systems become more widely adopted in the real-world, they must increasingly adapt to continual learning scenarios in which the retrieval corpora grow continuously. To understand how HippoRAG 2's capacity to handle this setting compared to standard RAG, we design an experiment in which we partition NQ and MuSiQue into four equal segments, each containing the gold documents and distractors for approximately 250 questions. We then select one segment for evaluation and incrementally add the remaining segments, measuring how performance evolves as new knowledge is added, allowing us to simulate a continual learning setting. We show the F1 score for HippoRAG 2 and NV-Embed-v2, our strongest baseline, in Figure 3.

随着 RAG 系统在现实世界中得到更广泛采用，它们必须越来越多地适应检索语料库持续增长的持续学习场景。 为了比较 HippoRAG 2 与标准 RAG 处理这种设置的能力，我们设计了一项实验：把 NQ 和 MuSiQue 各自划分为四个等大分段，每段包含约 250 个问题的正确文档和干扰文档。 随后，我们选择一个分段进行评估，并逐步加入其余分段，测量新知识加入后性能如何演变，从而模拟持续学习设置。 图3展示了 HippoRAG 2 与最强基线 NV-Embed-v2 的 F1 分数。

HippoRAG 2 continual learning under corpus expansion
图3：持续学习实验：我们把 NQ 和 MuSiQue 数据集划分为 4 个分段，在逐步将另外 3 个分段引入检索语料库、以模拟持续演化语料库的过程中，报告随机选定分段上的 F1 分数。
表6：HippoRAG 2 与 NV-Embed-v2 在不同类型问题上的代表性检索结果（段落标题）。粗体表示支持段落的标题。
QuestionNV-Embed-v2 ResultsHippoRAG 2 Filtered TriplesHippoRAG 2 Results
Simple QAIn what city was I.P. Paul born?1. I. P. Paul
2. Yinka Ayefele - Early life
3. Paul Parker (singer)		(I. P. Paul, from, Thrissur)
(I. P. Paul, was mayor of, Thrissur municipal corporation)		1. I. P. Paul
2. Thrissur
3. Yinka Ayefele
Multi-Hop QAWhat county is Erik Hort's birthplace a part of?1. Erik Hort
2. Horton Park (Saint Paul, Minnesota)
3. Hertfordshire		(Erik Hort, born in, Montebello)
(Erik Hort, born in, New York)		1. Erik Hort
2. Horton Park (Saint Paul, Minnesota)
3. Monstebello, New York
表7：对不同稠密检索器的鲁棒性。MuSiQue 子集上的 passage recall@5。
RetrieverDense RetrievalHippoRAG 2
GTE-Qwen2-7B-Instruct63.668.8
GritLM-7B66.071.6
NV-Embed-v2 (7B)69.774.7

As we can see in Figure 3, HippoRAG 2's improvements over NV-Embed-v2 remain remarkably consistent in both simple (NQ) and associative (MuSiQue) continual learning settings. We also note that, while both methods retain strong performance on simple QA (solid lines) as more knowledge is introduced, their performance in the more complex associative task (dotted lines) degrades at a similar rate as more information is introduced. This divergence underscores the importance of incorporating varied task complexities into future continual learning benchmarks.

如图3所示，在简单（NQ）和联想（MuSiQue）持续学习设置中，HippoRAG 2 相比 NV-Embed-v2 的提升都保持得非常稳定。 我们还注意到，随着更多知识被引入，两种方法在简单问答（实线）上都保持强劲性能，但它们在更复杂的联想任务（虚线）上的性能会以相近速度下降。 这种分化凸显了未来持续学习基准纳入不同任务复杂度的重要性。

6.4 Dense Retriever Flexibility ​

As demonstrated in Table 7, HippoRAG 2 consistently surpasses direct dense retrieval across various retrievers. Notably, these performance gains remain robust regardless of the specific dense retriever used.

如表7所示，HippoRAG 2 在不同检索器上都持续优于直接稠密检索。 值得注意的是，无论使用哪种具体的稠密检索器，这些性能增益都保持稳健。

6.5 Qualitative Analysis ​

We show examples from PopQA and MuSiQue in Table 6. For the first example, “In what city was I. P. Paul born?”, NV-Embed-v2 ranks the entity mentioned in the query “I. P. Paul” as the top 1, where the passage is enough to answer this question. But HippoRAG 2 does even better. It directly finds the answer “Thrissur” when linking the triples, and during the subsequent graph search, it places the passage corresponding to that entity in the second position, which is a perfect retrieval result.

表6展示了 PopQA 和 MuSiQue 中的示例。 对于第一个示例“I. P. Paul 出生在哪座城市？”，NV-Embed-v2 把查询中提到的实体“I. P. Paul”排在第一位，该段落足以回答这个问题。 但 HippoRAG 2 做得更好。 它在链接三元组时直接找到答案“Thrissur”，并在随后的图搜索中把该实体对应的段落排在第二位，形成完美的检索结果。

For the second multi-hop question, “What county is Erik Hort's birthplace a part of?” NV-Embed-v2 also easily identifies the person mentioned, “Erik Hort.” However, since this question requires two-step reasoning, it is not sufficient to fully answer the question. In contrast, HippoRAG 2 retrieves a passage titled “Montebello” during the query-to-triple step, which contains geographic information that implies the answer to the question. In the subsequent graph search, this passage is also ranked at the top. Apart from this, the error analysis of HippoRAG 2 is detailed in the appendix.

对于第二个多跳问题“Erik Hort 的出生地属于哪个县？”，NV-Embed-v2 也很容易识别出所提到的人物“Erik Hort”。 然而，这个问题需要两步推理，因此仅有该信息不足以完整回答问题。 相比之下，HippoRAG 2 在 Query to Triple 步骤中检索到标题为“Montebello”的段落，其中包含能够推出答案的地理信息。 在随后的图搜索中，该段落也排在前列。 除此之外，HippoRAG 2 的错误分析详见附录。

7. Conclusion ​

We introduced HippoRAG 2, a novel framework designed to address the limitations of existing RAG systems in approximating the dynamic and interconnected nature of human long-term memory. It combining the strengths of the Personalized PageRank algorithm, deeper passage integration, and effective online use of LLMs. HippoRAG 2 opens new avenues for research in continual learning and long-term memory for LLMs by achieving comprehensive improvements over standard RAG methods across factual, sense-making, and associative memory tasks, showing capabilities that previous methods have either overlooked or been incapable of achieving in a thorough evaluation. Future work could consider leveraging graph-based retrieval methods to further enhance the episodic memory capabilities of LLMs in long conversations.

我们提出了 HippoRAG 2，这是一个新型框架，旨在解决现有 RAG 系统逼近人类长期记忆动态且相互连接特性时的局限。 它结合了个性化 PageRank 算法、更深入的段落整合以及有效在线使用 LLM 的优势。 HippoRAG 2 在事实记忆、意义建构记忆和联想记忆任务上全面优于标准 RAG 方法，展现了此前方法在全面评估中忽略或无法实现的能力，从而为 LLM 的持续学习与长期记忆研究开辟了新方向。 未来工作可以考虑利用基于图的检索方法，进一步增强 LLM 在长对话中的情景记忆能力。

Impact Statement ​

This paper presents work on Retrieval-Augmented Generation (RAG) to advance the field of long-term memory for large language models. While our work may have various societal implications, we do not identify any concerns that warrant specific emphasis beyond those generally associated with large language models and information retrieval systems.

本文研究检索增强生成（RAG），以推进大语言模型长期记忆领域。 虽然我们的工作可能具有多种社会影响，但除大语言模型和信息检索系统通常涉及的问题外，我们没有发现需要特别强调的其他风险。