---

RET-LLM: Towards a General Read-Write Memory for Large Language Models

MemoryLLM120+ICLR 2024 AGI Workshop Poster慕尼黑大学慕尼黑机器学习中心Microsoft

Modarressi A, Imani A, Fayyaz M, Schütze H. RET-LLM: Towards a General Read-Write Memory for Large Language Models[J]. arXiv preprint arXiv:2305.14322v2, 2023.

https://openreview.net/forum?id=Z7tBs47cSH

---

RET-LLM：迈向面向大语言模型的通用读写记忆

Abstract

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization. However, existing LLMs lack a dedicated memory unit, limiting their ability to explicitly store and retrieve knowledge for various tasks. In this paper, we propose RET-LLM a novel framework that equips LLMs with a general write-read memory unit, allowing them to extract, store, and recall knowledge from the text as needed for task performance. Inspired by Davidsonian semantics theory, we extract and save knowledge in the form of triplets. The memory unit is designed to be scalable, aggregatable, updatable, and interpretable. Through qualitative evaluations, we demonstrate the superiority of our proposed framework over baseline approaches in question answering tasks. Moreover, our framework exhibits robust performance in handling temporal-based question answering tasks, showcasing its ability to effectively manage time-dependent information.

大语言模型（LLM）通过其庞大参数和综合数据利用，显著推动了自然语言处理（NLP）领域的发展。 然而，现有 LLM 缺少专用记忆单元，限制了它们在各种任务中显式存储和检索知识的能力。 在本文中，我们提出 RET-LLM，这是一个新框架，为 LLM 配备通用写读记忆单元，使它们能够根据任务表现需要，从文本中抽取、存储并回忆知识。 受 Davidsonian 语义理论启发，我们以三元组形式抽取并保存知识。 该记忆单元被设计为可扩展、可聚合、可更新且可解释。 通过定性评估，我们展示了所提出框架在问答任务中相较基线方法的优越性。 此外，我们的框架在处理基于时间的问题回答任务时表现稳健，展示了它有效管理时间依赖信息的能力。

1. Introduction

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) in recent years. With their vast parameter count and access to extensive data, LLMs have demonstrated remarkable accuracy across various tasks. However, current state-of-the-art LLMs lack a dedicated memory unit. Instead, they are trained to predict words based on context, encoding knowledge implicitly in their parameters, which differs from the ideal memory function.

近年来，大语言模型（LLM）显著推动了自然语言处理（NLP）领域的发展。 凭借庞大的参数量和对大规模数据的访问，LLM 在各种任务上展现出显著准确性。 然而，当前最先进的 LLM 缺少专用记忆单元。 相反，它们被训练为基于上下文预测词语，把知识隐式编码在参数中，这不同于理想的记忆功能。

An ideal memory unit should possess certain characteristics. Firstly, it should allow for read and write operations, enabling the language model to interact with stored knowledge. Scalability is also crucial, as the memory unit should accommodate the consistently evolving nature of knowledge. Furthermore, the memory unit should not be limited to textual documents alone; it should be capable of acquiring knowledge from diverse sources such as database systems. Interpretabilty is desired, granting insight into the specific knowledge required by the LLM to solve a given task. Lastly, the information stored in the memory unit should be aggregatable, enabling the model to combine related information across multiple documents. For instance an LLM should be able to list all cities of a country mentioned in multiple documents.

理想的记忆单元应具有若干特征。 首先，它应允许读写操作，使语言模型能够与已存储知识交互。 可扩展性同样关键，因为记忆单元应适应知识持续演化的特性。 此外，记忆单元不应只限于文本文档；它应能够从数据库系统等多样来源中获取知识。 可解释性也是所期望的，因为它能让我们洞察 LLM 为解决给定任务所需的具体知识。 最后，记忆单元中存储的信息应当可聚合，使模型能够结合多个文档中的相关信息。 例如，LLM 应能够列出多个文档中提到的某个国家的所有城市。

图1：RET-LLM 的概览。用户可以用（A）信息性句子进行提示，我们的方法会把其中潜在信息存入记忆，或用（B）问题进行提示，此时应利用先前保存的信息来生成有效答案。

Previous attempts to incorporate memory into LLMs have fallen short in capturing the complete range of memory characteristics. For example, prior retrieval-based methods degrade the memory as the ability to retrieve relevant documents for a given query context, and adding them to the context when generating answers. Park et al. merely stores and retrieves previous observations and reflections of a generative agent in a simulated environment.

先前把记忆纳入 LLM 的尝试未能覆盖完整的记忆特征范围。 例如，先前基于检索的方法把记忆降格为针对给定查询上下文检索相关文档，并在生成答案时把这些文档加入上下文的能力。 Park 等人仅在模拟环境中存储并检索生成式智能体先前的观察和反思。

To address these limitations, we introduce RET-LLM, (Retentive LLM) a solution that endows LLMs with a scalable, updatable, interpretable, and aggregatable memory module. Our proposal involves equipping language models with a memory module, which allows them to extract knowledge from text and save it for future reference. When faced with a task, the LLM can query the memory module for additional information to support its response. The memory module supports updates and can incorporate information from non-textual sources such as SQL and no-SQL databases and spreadsheets. Furthermore, it enables aggregation of various pieces of information related to a particular concept scatterred in a huge document or within multiple documents.

为了解决这些限制，我们提出 RET-LLM（Retentive LLM），这是一种为 LLM 赋予可扩展、可更新、可解释和可聚合记忆模块的方案。 我们的方案是为语言模型配备记忆模块，使其能够从文本中抽取知识并保存以供未来引用。 面对任务时，LLM 可以查询记忆模块以获得额外信息来支持其回答。 记忆模块支持更新，并且可以纳入来自 SQL、NoSQL 数据库和电子表格等非文本来源的信息。 此外，它能够聚合与特定概念相关、散布在大型文档或多个文档中的各种信息片段。

Figure 1 shows the architecture of RET-LLM. It comprises three components: an LLM, a controller, and a memory unit. We employ Alpaca, a recently released instruction-tuned language model (LLM), and design a fine-tuning process to enable it to acquire the following abilities: information extraction, information lookup, and fact-based answer generation.

图1展示了 RET-LLM 的架构。 它由三个组件组成：一个 LLM、一个控制器和一个记忆单元。 我们采用 Alpaca，这是一个近期发布的指令微调语言模型（LLM），并设计微调过程，使其获得以下能力：信息抽取、信息查找和基于事实的答案生成。

Information extraction entails the identification and extraction of triplets in the form of <concept1, relationship, concept2> from informative sentences. The information lookup task involves querying the memory unit to acquire additional information concerning a given concept and its associated relationships when confronted with tasks necessitating further information. Lastly, fact-based answer generation involves generating a final answer based on the retrieved information. The triplet-based storage approach draws inspiration from the theoretical framework of Davidsonian semantics, which provides a foundation for representing concepts described in sentences using a triplet-like structure of <event, subject, object>.

信息抽取涉及从信息性句子中识别并抽取 <concept1, relationship, concept2> 形式的三元组。 信息查找任务是在面对需要更多信息的任务时，查询记忆单元以获取关于给定概念及其相关关系的额外信息。 最后，基于事实的答案生成涉及基于检索到的信息生成最终答案。 基于三元组的存储方法受到 Davidsonian 语义理论框架的启发，该框架为使用 <event, subject, object> 这类三元组结构表示句子中描述的概念提供了基础。

The memory module stores the triplets and their vector representations. During retrieval, it first searches for an exact match of the query text and resorts to a fuzzy search based on vector representations if no exact match is found. For efficient fuzzy search and retrieval, we employ LSH-based hashing of vector representations. The controller acts as an interface, automating interactions between users, the LLM, and the memory module, ensuring a seamless interaction experience with an intelligent chat system.

记忆模块存储三元组及其向量表示。 在检索时，它首先搜索查询文本的精确匹配；如果没有找到精确匹配，则转而基于向量表示进行模糊搜索。 为了实现高效的模糊搜索和检索，我们采用基于 LSH 的向量表示哈希。 控制器充当接口，自动化用户、LLM 和记忆模块之间的交互，从而确保与智能聊天系统的无缝交互体验。

Our proposed approach offers several advantages over previous methods. It enables LLMs to explicitly store and retrieve knowledge, which is crucial for real-world NLP applications. By incorporating explicit knowledge storage and retrieval, we gain better understanding of the workings of these models and the knowledge they rely on to solve tasks. The use of an external memory unit separate from the LLM ensures scalability and easy modification of stored information. The fuzzy search technique enables efficient retrieval of relevant information, even in the absence of exact matches. Storing information in triplets facilitates the generation of precise and comprehensive solutions, particularly when data aggregation is necessary. Lastly, the memory module allows for easy incorporation of information from diverse sources and accommodates changing facts over time.

我们提出的方法相较先前方法具有若干优势。 它使 LLM 能够显式存储和检索知识，这对真实世界 NLP 应用至关重要。 通过引入显式知识存储和检索，我们能更好理解这些模型的工作方式，以及它们为解决任务所依赖的知识。 使用与 LLM 分离的外部记忆单元可以确保可扩展性，并便于修改已存储信息。 即使不存在精确匹配，模糊搜索技术也能高效检索相关信息。 以三元组存储信息有助于生成精确且全面的解决方案，尤其是在需要数据聚合时。 最后，记忆模块允许轻松纳入来自多样来源的信息，并适应随时间变化的事实。

Over a qualitative evaluation using question answering examples, we demonstrate cases where a comparable LLM such as Alpaca-7B fails to return a correct answer. We show that this shortcoming occurs while the model has access to all the information required for generating a valid answer. However, in our proposed approach after storing the extractable knowledge from the context, the RET-LLM shows its capability in answering a question without the need of reinputting the context. We also demonstrate that RET-LLM could handle temporal based QA examples. Since it is equipped with a modifiable memory which could handle temporal facts.

通过使用问答示例进行定性评估，我们展示了 Alpaca-7B 等可比 LLM 无法返回正确答案的案例。 我们表明，即使模型能够访问生成有效答案所需的所有信息，这一缺陷仍会发生。 然而，在我们提出的方法中，当从上下文中存储可抽取知识后，RET-LLM 展现出无需重新输入上下文即可回答问题的能力。 我们还展示了 RET-LLM 能够处理基于时间的问答示例。 因为它配备了可修改的记忆，能够处理时间性事实。

2. Related Works

Prior works in the field have explored incorporating relevant context into large language models (LLMs) by retrieving and adding relevant documents to the task's context. Zhong et al. propose training LLMs with memory augmentation by introducing trainable memory units that are optimized during the training process. Wu et al. presents the Memorizing Transformer, which can attend to longer documents during inference. This approach stores (Key, Value) pairs, extracted from a transformer layer, in a memory and retrieves relevant pairs to add them to the current context during generation. Cheng et al. encode each documents, save them, and retrieve relevant documents based on the current context. In contrast to these approaches, our method offers improved scalability as we do not modify the architecture of the LLM. Instead, we suggest extracting and saving information from documents, allowing for the aggregation of extracted information from multiple sources. This enables us to provide more relevant and concise retrieved information that is closely aligned with the specific question being addressed.

该领域的先前工作已经探索了通过检索相关文档并将其加入任务上下文，把相关上下文纳入大语言模型（LLM）的方法。 Zhong 等人提出通过引入在训练过程中优化的可训练记忆单元，来训练带有记忆增强的 LLM。 Wu 等人提出 Memorizing Transformer，它能够在推理时关注更长文档。 该方法把从 Transformer 层抽取的（Key, Value）对存储在记忆中，并检索相关对，在生成期间将它们加入当前上下文。 Cheng 等人编码每个文档，保存它们，并基于当前上下文检索相关文档。 与这些方法相比，我们的方法具有更好的可扩展性，因为我们不修改 LLM 的架构。 相反，我们建议从文档中抽取并保存信息，从而允许聚合来自多个来源的抽取信息。 这使我们能够提供更相关、更简洁的检索信息，并与正在处理的具体问题紧密对齐。

Park et al. utilizes an LLM within a generative agent framework to facilitate the storage and dynamic retrieval of a comprehensive record of the agent's experiences using natural language. However, there exists a fundamental distinction between their architecture and ours. In Park's framework, the memory component is an inherent part of the agent itself, while the LLM serves as an external tool employed solely for planning the agent's behaviors. Consequently, the LLM lacks control over the specific content to be stored and retrieved within the agent's memory.

Park 等人在生成式智能体框架中使用 LLM，以自然语言促进对智能体经验完整记录的存储和动态检索。 然而，他们的架构与我们的架构之间存在根本区别。 在 Park 的框架中，记忆组件是智能体自身的内在部分，而 LLM 只是作为外部工具，仅用于规划智能体行为。 因此，LLM 无法控制要在智能体记忆中存储和检索的具体内容。

Dhingra et al. contribute to the field by curating a dataset specifically designed to differentiate between temporal and non-temporal facts. They propose training language models on temporally annotated data to enhance their temporal awareness. This work aligns with our research focus on addressing temporal information challenges. However, in our proposed solution, we address these challenges by introducing an updatable memory module.

Dhingra 等人通过构建一个专门用于区分时间性事实和非时间性事实的数据集，为该领域作出贡献。 他们提出在带时间标注的数据上训练语言模型，以增强模型的时间意识。 这项工作与我们处理时间信息挑战的研究重点一致。 然而，在我们提出的解决方案中，我们通过引入可更新记忆模块来应对这些挑战。

Schick et al. present a methodology that empowers LLMs to leverage external tools by generating API calls to access additional functionalities, such as using a calculator for task execution. Our work shares similarities with their approach in terms of teaching the LLM to utilize an external tool. However, it should be noted that our focus lies on incorporating a more intricate and influential tool, namely the memory module, which has the potential to significantly impact the LLM's output.

Schick 等人提出一种方法，使 LLM 能够通过生成 API 调用来利用外部工具，以访问额外功能，例如使用计算器执行任务。 在教会 LLM 使用外部工具这一点上，我们的工作与他们的方法相似。 然而，需要注意的是，我们的重点在于引入一个更复杂且更有影响力的工具，即记忆模块，它有潜力显著影响 LLM 的输出。

3. Approach

We aim to design a RET-LLM where the user can perform two actions: (1): Provide one or a series of informative statements where the RET-LLM should be able to memorize the containing information. Previous methods perform this task by either training/fine-tuning the LLM over the provided document or creating a vector representation for the document and storing the representation. (2): Asking related questions which the RET-LLM would answer based on the stored memory. All these actions should function in a seamless setting where the user should only interact in natural language.

我们的目标是设计一个 RET-LLM，使用户能够执行两种操作：（1）提供一个或一系列信息性陈述，RET-LLM 应能够记住其中包含的信息。 先前方法通过在提供的文档上训练/微调 LLM，或为文档创建向量表示并存储该表示来执行这一任务。 （2）提出相关问题，RET-LLM 将基于已存储记忆进行回答。 所有这些操作都应在无缝设置中运行，其中用户只需用自然语言交互。

Our RET-LLM is constituted by three main components: (1) Controller, (2): Fine-tuned LLM & (3): Memory. As shown in Figure 1, the controller moderates the flow of information between the user, the LLM and the memory. The LLM acts as a processing unit, where it receives the texts passed by the controller and figures where it needs to invoke a memory call or not. Since the LLM operates with text, inspired by Schick et al., we standardized the memory calls by implementing a text-based API schema. Therefore the LLM could generate memory API calls and the controller could apply the LLM API calls to the memory. In our setting, the memory stores data in triplets by using a three-columned table. This is based on the theoretical framework of Davidsonian semantics, where concepts described in sentences could be stored in a structure of <first argument, relation, second argument>.

我们的 RET-LLM 由三个主要组件组成：（1）控制器，（2）微调后的 LLM，和（3）记忆。 如图1所示，控制器调节用户、LLM 和记忆之间的信息流。 LLM 作为处理单元，接收控制器传递的文本，并判断是否需要调用记忆。 由于 LLM 以文本运行，受 Schick 等人的启发，我们通过实现基于文本的 API 模式来标准化记忆调用。 因此，LLM 可以生成记忆 API 调用，而控制器可以把 LLM 的 API 调用应用到记忆。 在我们的设置中，记忆使用三列表以三元组形式存储数据。 这基于 Davidsonian 语义理论框架，其中句子中描述的概念可以被存储为 <first argument, relation, second argument> 的结构。

In the following we describe RET-LLM in more detail. The memory-API, how we finetune the LLM to become capable of these calls and the memory structure.

下面我们更详细地描述 RET-LLM。 包括 memory-API、我们如何微调 LLM 使其具备这些调用能力，以及记忆结构。

(a) Memory-Write scenario.

(b) Memory-Read scenario.

图2：读写输入中流程的可视化。（a）Memory-Write 场景：控制器把输入传递给 LLM，LLM 生成适当的记忆写入调用，控制器把数据及其平均表示交给记忆存储。（b）Memory-Read 场景：控制器把问题传递给 LLM，LLM 生成适当的记忆读取调用，控制器用 LLM 给出的搜索词对记忆应用查询，记忆返回查询结果，结果再被转发给 LLM，LLM 使用查询结果生成问题答案并返回给用户。

3.1. Memory Structure

Each triplet defines a relationship between two arguments with the following format: $⟨t_1,t_2,t_3⟩$ where $t_1$ is the first argument, $t_2$ is the relation and $t_3$ is the second argument in the relationship. For instance in the sentence: ``Mark Zuckerberg is the CEO of Meta Inc.'' the informative triplet that could be extracted is: $(\text{Mark Zuckerberg},\text{CEO},\text{Meta Inc.})$.

每个三元组都定义两个论元之间的关系，其格式为 $⟨t_1,t_2,t_3⟩$，其中 $t_1$ 是第一个论元，$t_2$ 是关系，$t_3$ 是该关系中的第二个论元。 例如，在句子 “Mark Zuckerberg is the CEO of Meta Inc.” 中，可以抽取的信息性三元组是 $(\text{Mark Zuckerberg},\text{CEO},\text{Meta Inc.})$。

To store these triplets we use a three-columned table where each column is associated with each part of the triplet. Alongside saving the texts, we store the average representations so that the memory could also handle queries which have semantically similar words. If the memory module fails to find the exact text in the table, it checks for similar texts by comparing the vector representation of the query text with vector representations of text peices already stored in the dataset. Therefore for every $t_i$ the mean representation retrieved by the LLM (${\mathit{h}}_{AVG}(t_i)$) is stored in a Locality-Sensitive Hashing (LSH) table. The reason of utilizing LSH is to reduce the computation required for finding similar representations. Without a hash table for a given query representation, the distances to all of the stored representations should be computed which would be a computationally-expensive task.

为了存储这些三元组，我们使用三列表，其中每一列对应三元组的一个部分。 在保存文本的同时，我们存储平均表示，使记忆也能够处理包含语义相似词的查询。 如果记忆模块无法在表中找到精确文本，它会通过比较查询文本的向量表示和数据集中已存文本片段的向量表示来检查相似文本。 因此，对于每个 $t_i$，LLM 检索到的平均表示（${\mathit{h}}_{AVG}(t_i)$）都会存储在局部敏感哈希（LSH）表中。 使用 LSH 的原因是减少寻找相似表示所需的计算量。 如果没有针对给定查询表示的哈希表，就必须计算它到所有已存表示的距离，这将是一项计算代价高昂的任务。

Handling Memory Queries. In a memory query, one or two of the triplet parameters should be provided as input:

处理记忆查询。 在记忆查询中，应将一个或两个三元组参数作为输入提供：

$$\begin{array}{rl}\mathcal{Q}\in \{& ⟨q_1⟩,⟨q_2⟩,⟨q_3⟩,\\ & ⟨q_1,q_2⟩,⟨q_1,q_3⟩,⟨q_2,q_3⟩\}.\end{array}$$

Where $q_i$ is the search term for the $i$-th parameter in the stored tuples. Before retrieving the query results, each search term is checked For a given $\mathcal{Q}$, first the memory checks whether the search terms ($q_i$) have an exact match in the storage table. If $q_i$ does not exist in the stored terms, we use its average representation ${\mathit{h}}_{AVG}(q_i)$ and the LSH table for an alternative term (${\tilde{q}}_i$) that has an exact match in out memory table. Possibly, the LSH table may not find an alternative term for the given representation, therefore the query would not have a result: $\mathcal{Q}\to \mathrm{\varnothing }$. In any case (exact match or similar match), the query might have multiple matches in the data table ($q_i=t_i$). In this case all resulting triplets would be returned as the query output.

其中 $q_i$ 是已存元组中第 $i$ 个参数的搜索词。 在检索查询结果之前，每个搜索词都会被检查；对于给定的 $\mathcal{Q}$，记忆首先检查搜索词（$q_i$）在存储表中是否存在精确匹配。 如果 $q_i$ 不存在于已存词项中，我们使用其平均表示 ${\mathit{h}}_{AVG}(q_i)$ 和 LSH 表来寻找一个在记忆表中有精确匹配的替代词（${\tilde{q}}_i$）。 LSH 表可能找不到给定表示的替代词，因此查询将没有结果：$\mathcal{Q}\to \mathrm{\varnothing }$。 无论是精确匹配还是相似匹配，查询都可能在数据表中有多个匹配（$q_i=t_i$）。 在这种情况下，所有得到的三元组都会作为查询输出返回。

3.2. Memory-API & Dataflow

To enable communication between the memory and the LLM, we design an API schema for memory read and write functions. This API allows the controller to understand when the LLM is calling the memory and what parameters should be passed. Based on the triplets discussed in the previous section, the two memory calls are as the following:

为了实现记忆与 LLM 之间的通信，我们为记忆读写函数设计了 API 模式。 该 API 允许控制器理解 LLM 何时调用记忆，以及应传递哪些参数。 基于上一节讨论的三元组，两种记忆调用如下：

• [MEM_WRITE{$t_1$>>$t_2$>>$t_3$}]: This structure is for storing a triplet $⟨t_1,t_2,t_3⟩$.
• Depending on the prompt, multiple write calls could be sequentially generated by the LLM to store multiple triplets extracted from a text.
• [MEM_READ{>>>>_}:{$t_1$>>$t_2$>>$t_3$};...]: In a memory read, as shown in the API, there are three placeholders that based on $\mathcal{Q}$ atleast one of them should be filled with the search terms.
• Based on the query results from the memory, one or a list of triplets could be returned as shown in the highlighted segment.

• [MEM_WRITE{$t_1$>>$t_2$>>$t_3$}]：该结构用于存储三元组 $⟨t_1,t_2,t_3⟩$。
• 根据提示，LLM 可以连续生成多个写入调用，以存储从文本中抽取的多个三元组。
• [MEM_READ{>>>>_}:{$t_1$>>$t_2$>>$t_3$};...]：在记忆读取中，如 API 所示，有三个占位符；基于 $\mathcal{Q}$，其中至少一个应填入搜索词。
• 基于记忆中的查询结果，可以返回一个或一组三元组，如高亮片段所示。

Figure 2 demonstrates how RET-LLM operates using the memory-API. Depending on the input given by the user, RET-LLM either have to read or write information from or to the memory. If the user prompt an informative statement (or ideally a full document), it would be memory write scenario. On the other hand, by having a question in the input, we consider this to be a memory read case. In both cases the user input is the first input to RET-LLM that is passed on to the LLM.

图2展示了 RET-LLM 如何使用 memory-API 运行。 根据用户给出的输入，RET-LLM 需要从记忆中读取信息，或把信息写入记忆。 如果用户提示的是信息性陈述（理想情况下是完整文档），这就是记忆写入场景。 另一方面，如果输入中包含问题，我们将其视为记忆读取案例。 在两种情况下，用户输入都是传递给 LLM 的 RET-LLM 的第一个输入。

Based on the given input the LLM infers and generates the relevant API call. With a memory write case, after the API call is generated the controller detects it and invoke a memory storation function with the given parameters. The memory receives the data in a triplet format and stores it for future usage. If a memory read call is generated by the LLM, the controller also detects it and pauses the model's sequence generation for the memory retrieval. It uses the parameters given inside the read call as the query terms and passes them to the memory. The memory lists all stored triplets that feature the given search terms (or a semantically similar version of them according to Section 3.1) and return the results back to the controller. Using the API discussed in the beginning of this section, the read results are listed after the call so that the LLM could use them to produce a naturally sounded answer. After the answer is produced it is returned back to the user.

基于给定输入，LLM 推断并生成相关 API 调用。 在记忆写入案例中，API 调用生成后，控制器会检测到它，并使用给定参数调用记忆存储函数。 记忆以三元组格式接收数据，并将其存储以供未来使用。 如果 LLM 生成记忆读取调用，控制器也会检测到它，并暂停模型的序列生成以进行记忆检索。 它使用读取调用中给出的参数作为查询词，并把它们传递给记忆。 记忆列出所有包含给定搜索词（或根据第 3.1 节得到的语义相似版本）的已存三元组，并把结果返回给控制器。 使用本节开头讨论的 API，读取结果会列在调用之后，使 LLM 能够用它们生成自然的答案。 答案生成后会返回给用户。

As the controller is in between of the user and the LLM, it could hide the whole memory-API schema. This would make the user feel an end-to-end simple language modeling experience without knowing the memory functionality behind the scene.

由于控制器位于用户和 LLM 之间，它可以隐藏整个 memory-API 模式。 这会让用户感受到端到端的简单语言建模体验，而无需了解幕后记忆功能。

3.3. Finetuning the LLM

In this part we discuss how the LLM is finetuned to be capable of generating memory-API calls. In the end the LLM should be capable of detecting which type of memory call (read or write) it should provoke based on the input. As stated in Section 3.2, the LLM's input may have one of the two previously discussed structures depending on the memory function. Therefore the LLM should be able to generate and handle this API to store or read the relevant information. To this end, we develop a synthetic dataset to train the LLM. The synthetic task is to learn the relationships of the discussed people with the respective corporations. Based on the stored information, RET-LLM should be capable of answering any questions regarding the people, the corporations or the relationships.

在这一部分，我们讨论如何微调 LLM，使其能够生成 memory-API 调用。 最终，LLM 应能够基于输入检测应触发哪种记忆调用（读取或写入）。 如第 3.2 节所述，根据记忆功能，LLM 的输入可能具有前面讨论的两种结构之一。 因此，LLM 应能够生成并处理该 API，以存储或读取相关信息。 为此，我们开发了一个合成数据集来训练 LLM。 该合成任务是学习所讨论人物与相应公司之间的关系。 基于已存储信息，RET-LLM 应能够回答关于人物、公司或关系的任何问题。

表1：Memory read data examples for finetuning. 前三类问题基于单个三元组，因此 API response 只包含一个三元组；后三类问题可能有多个相关三元组存储在记忆中，因此答案应把三元组数据合并成一个句子。[per1, per2, ...] 是按自然方式顺序写出的姓名占位符。

Query Type	Question	API Query	API Response	Answer
$⟨per⟩$	Who is per?	{per>>>>}:	{per>>rel>>org}	per is rel to org.
$⟨per,org⟩$	How per is related to org?	{per>>>>org}:	{per>>rel>>org}	per is rel to org.
$⟨per,rel⟩$	per is rel which company?	{per>>rel>>}:	{per>>rel>>org}	per is rel to org.
$⟨org⟩$	Who are related to org?	{>>>>org}:	{per1>>rel1>>org}; {per2>>rel2>>org}; ...	[per1, per2, ...] is/are related to org.
$⟨rel⟩$	Who are the rel?	{>>rel>>}:	{per1>>rel>>org1}; {per2>>rel>>org2}; ...	[per1, per2, ...] is/are rel.
$⟨org,rel⟩$	Who are rel org?	{>>rel>>org}:	{per1>>rel>>org}; {per2>>rel>>org}; ...	[per1, per2, ...] is/are rel to org.

表2：Memory write data example structure for finetuning.

Triplet(s)	Statement	API Write Call(s)
$[⟨pe{r}_1,rel,org⟩,⟨pe{r}_2,rel,org⟩,\dots ]$	[per1, per2, ...] is/are rel to org.	[MEM_WRITE{per1>>rel1>>org}][MEM_WRITE{per2>>rel2>>org}]...

We use a set of firstname and lastnames to generate a synthetic population, called $\mathcal{P}$. Each person from this population $per\in \mathcal{P}$ could have only one relationship from the following list with an organization $org\in \mathcal{O}$:

我们使用一组名字和姓氏生成一个合成人群，称为 $\mathcal{P}$。 该人群中的每个人 $per\in \mathcal{P}$ 只能与某个组织 $org\in \mathcal{O}$ 存在以下列表中的一种关系：

$$\begin{array}{rl}rel\in \mathcal{R}=\{& \text{employment},\text{manager},\text{investor},\\ & \text{founder},\text{customer}\}.\end{array}$$

Where $\mathcal{O}$ is a set of corporation names. Hence, each triplet would be as: $(per,rel,org)$. For instance: $⟨\text{Dominick Alphonso},\text{employment},\text{BMW}⟩$. Even though the corporation names are real, the people names are entirely random generated. No identification with actual persons is intended or should be inferred.

其中 $\mathcal{O}$ 是公司名称集合。 因此，每个三元组可写作：$(per,rel,org)$。 例如：$⟨\text{Dominick Alphonso},\text{employment},\text{BMW}⟩$。 尽管公司名称是真实的，人物姓名完全是随机生成的。 不意图也不应推断其对应任何真实人物。

Based on this triplet we can build three triplet-specific questions:

基于该三元组，我们可以构造三类特定于三元组的问题：

• $\mathcal{Q}=⟨per⟩$, e.g. ``Who is Dominick Alphonso?''
• $\mathcal{Q}=⟨per,org⟩$, e.g. ``How Dominick Alphonso is related to BMW?''
• $\mathcal{Q}=⟨per,rel⟩$, e.g. ``Dominick Alphonso is employed by which company?''

• $\mathcal{Q}=⟨per⟩$，例如 “Who is Dominick Alphonso?”
• $\mathcal{Q}=⟨per,org⟩$，例如 “How Dominick Alphonso is related to BMW?”
• $\mathcal{Q}=⟨per,rel⟩$，例如 “Dominick Alphonso is employed by which company?”

and the answer to all above should be ``Dominick Alphonso is employed by BMW.''. Alongside these questions three other types of questions could be asked that could be relevant to multiple triplets:

上述所有问题的答案都应为 “Dominick Alphonso is employed by BMW.”。 除了这些问题之外，还可以提出另外三类可能与多个三元组相关的问题：

• $\mathcal{Q}=⟨rel⟩$, e.g. ``Who are the employees?''
• $\mathcal{Q}=⟨org⟩$, e.g. ``Who are related to BMW?''
• $\mathcal{Q}=⟨rel,org⟩$, e.g. ``Who are employed by BMW?''

• $\mathcal{Q}=⟨rel⟩$，例如 “Who are the employees?”
• $\mathcal{Q}=⟨org⟩$，例如 “Who are related to BMW?”
• $\mathcal{Q}=⟨rel,org⟩$，例如 “Who are employed by BMW?”

Unlike the first three, each of these questions could have multiple persons related to the answer. For each of these questions we expect the model answer the questions without any extra information (e.g. stating the corporation of employment when its not asked). To create a training data instance from these questions based on the memory-API, we use the templates stated in Table 1. During finetuning the Question, API query (with the MEM_READ command), API Response and the answer are concatenated as the data input for the LLM. However, the langauge modeling loss is only applied to the API query and Answer sections. Since these two segments are the text sequences that the LLM is expected to generate based on the other two segments (Question & API Response) that are provided by the controller.

与前三类不同，这些问题中的每一类都可能有多个人与答案相关。 对于这些问题中的每一个，我们期望模型在没有任何额外信息的情况下回答问题（例如在未被询问时不说明受雇公司）。 为了基于 memory-API 从这些问题创建训练数据实例，我们使用表1中给出的模板。 在微调期间，Question、API query（带 MEM_READ 命令）、API Response 和答案会被拼接为 LLM 的数据输入。 然而，语言建模损失只应用于 API query 和 Answer 部分。 因为这两个片段是 LLM 需要基于控制器提供的另外两个片段（Question 和 API Response）生成的文本序列。

As we also need informative examples where have MEM_WRITE calls, we use a similar strategy by using the population, organizations and relations that were previously defined ($\mathcal{P},\mathcal{Q},\mathcal{R}$). Based on the memory-API, in a memory write scenario the RET-LLM receives a sentence which here contains a relationship information and then the LLM should generate the corresponding memory write calls. In our dataset we opted to build examples where it states about multiple people whom have the same relationship with the same company: $(pe{r}_i,rel,org)$. The template for the memory write data examples are shown in Table 2. Similar to the question-based examples, the statement and the API call are concatenated to form the full input sequence. Also the loss function is applied only to the API segment, since the first part is provided by the controller.

由于我们还需要包含 MEM_WRITE 调用的信息性示例，我们使用先前定义的人群、组织和关系（$\mathcal{P},\mathcal{Q},\mathcal{R}$）采用类似策略。 基于 memory-API，在记忆写入场景中，RET-LLM 接收一个句子，这里该句子包含关系信息，然后 LLM 应生成对应的记忆写入调用。 在我们的数据集中，我们选择构造说明多个人与同一公司具有相同关系的示例：$(pe{r}_i,rel,org)$。 记忆写入数据示例的模板见表2。 与基于问题的示例类似，陈述和 API 调用会被拼接形成完整输入序列。 同样，损失函数只应用于 API 片段，因为第一部分由控制器提供。

We opted to use the Instruction-following Alpaca-7B model as a base model for our finetuning. To execute the training in a resource limited setup, we use low-rank adaptation (LoRA). The code for finetuning a llama-based model using LoRA is available in alpaca-lora. This parameter efficient measure allows us to finetune the base model on a single A6000 48GB GPU.

我们选择使用遵循指令的 Alpaca-7B 模型作为微调的基础模型。 为了在资源受限的设置中执行训练，我们使用低秩适配（LoRA）。 使用 LoRA 微调基于 LLaMA 的模型的代码可在 alpaca-lora 中获得。 这种参数高效措施使我们能够在单张 A6000 48GB GPU 上微调基础模型。

图3：一个在零样本设置中得到错误结果、而在我们的方法中得到正确结果的示例。注意，在零样本设置中，模型在输入中直接访问了回答问题所需的信息，但仍然得到错误答案。然而，在我们的方法中，每个用户提示都可以分别给到 RET-LLM。源码 caption 还提到 appendix 中的另一个示例；该 appendix 图未纳入本页。

图4：询问需要时间上下文的问题通常会导致过时答案，如这里的 Alpaca 所示。然而，在我们的 RET-LLM 中，借助可修改记忆，只需提供更新后的记忆条目即可回答这些问题。

4. Qualitative Results

In this part, we present the internal process and final output on multiple evaluation examples. These examples were generated with the same procedure stated in Section 3.3. First to demonstrate the importance of our approach, we provide the same example to our base model (Alpaca-7B) in a zero-shot setting. The input would be a short instruction for the task, the informative sentences from the example and in the end is the question. As shown in Figure 3, the zero-shot result from the instruction tuned model is clearly incorrect. While the model does have all the information in its context, its still produces an incorrect response.

在这一部分，我们展示多个评估示例的内部过程和最终输出。 这些示例使用第 3.3 节所述的相同流程生成。 首先，为了展示我们方法的重要性，我们在零样本设置中向基础模型（Alpaca-7B）提供同一示例。 输入将是任务的简短指令、示例中的信息性句子，最后是问题。 如图3所示，指令微调模型的零样本结果明显错误。 尽管模型在其上下文中拥有所有信息，它仍然生成了错误回答。

In thie same example, the RET-LLM first stores the extracted triplets from the examples into the memory. After storing the extracted relationships, the RET-LLM could respond to the same question even without having the information in the input. With the help of the memory-API and the memory itself, the relevant triplet is found. The LLM manages to answer correctly after appending the query result to the memory call.

在同一个示例中，RET-LLM 首先把从示例中抽取的三元组存入记忆。 在存储抽取出的关系后，即使输入中不包含该信息，RET-LLM 也能够回答同一问题。 借助 memory-API 和记忆本身，相关三元组被找到。 在把查询结果追加到记忆调用之后，LLM 成功给出正确答案。

One potential use cases of our approach is in answering questions that have a temporal context. For example, the presidency of the United States undergoes a change every 4 to 8 years. A normal PLM model answers the question about the presidency based on its own training data. While model retraining or parameter editing has its own challenges, our approach could provide an easy and interpretable solution for this issue (Figure 4).

我们方法的一个潜在用例是回答具有时间上下文的问题。 例如，美国总统职位每 4 到 8 年会发生变化。 普通 PLM 模型会基于自身训练数据回答关于总统职位的问题。 虽然模型重训练或参数编辑各有挑战，但我们的方法可以为这一问题提供一种简单且可解释的解决方案（图4）。

5. Conclusion & Future Work

In this work, we introduced a RET-LLM capable of storing information and retrieving it in further use. With a triplet based memory structure, information are stored in relationships between two arguments with a known relation. The memory could be utilized via a memory-API which is generated by a finetuned LLM. Using a controller, all components could communicate with each other and the user would interact with the controller being unbeknown of the behind process. We have shown that the LLM generates the proper API calls in some question answering examples without having the information in its input context. As this work is still under development, in our next revision we will add a more in-detail empirical evaluation, preferrably on a real dataset. We also seek to improve our finetuning method to a more generalized setting so that it could be capable of working with more types of informative relations.

在这项工作中，我们介绍了一个 RET-LLM，它能够存储信息并在后续使用中检索信息。 通过基于三元组的记忆结构，信息被存储为两个论元之间具有已知关系的关系。 记忆可以通过由微调后的 LLM 生成的 memory-API 来使用。 借助控制器，所有组件都可以彼此通信，而用户会与控制器交互，却不知道背后的过程。 我们已经展示，在一些问答示例中，即使输入上下文中没有信息，LLM 也能生成适当的 API 调用。 由于这项工作仍在开发中，在下一版中，我们将加入更详细的实证评估，最好是在真实数据集上进行。 我们还希望把微调方法改进为更通用的设置，使其能够处理更多类型的信息性关系。