MIRIX：面向基于 LLM 的智能体的多智能体记忆系统
Abstract
Although memory capabilities of AI agents are gaining increasing attention, existing solutions remain fundamentally limited. Most rely on flat, narrowly scoped memory components, constraining their ability to personalize, abstract, and reliably recall user-specific information over time. To this end, we introduce MIRIX, a modular, multi-agent memory system that redefines the future of AI memory by solving the field’s most critical challenge: enabling language models to truly remember. Unlike prior approaches, MIRIX transcends text to embrace rich visual and multimodal experiences, making memory genuinely useful in real-world scenarios. MIRIX consists of six distinct, carefully structured memory types: Core, Episodic, Semantic, Procedural, Resource Memory, and Knowledge Vault, coupled with a multi-agent framework that dynamically controls and coordinates updates and retrieval. This design enables agents to persist, reason over, and accurately retrieve diverse, long-term user data at scale.
尽管 AI 智能体的记忆能力日益受到关注，但现有方案仍存在根本性局限。 大多数方案依赖扁平、范围狭窄的记忆组件，限制了其随时间对用户专属信息进行个性化处理、抽象和可靠回忆的能力。 为此，我们提出 MIRIX，一种模块化多智能体记忆系统；它通过解决该领域最关键的挑战——让语言模型能够真正记住——重新定义 AI 记忆的未来。 与先前方法不同，MIRIX 超越文本，纳入丰富的视觉与多模态体验，使记忆在现实场景中真正有用。 MIRIX 由六种相互独立且经过精心组织的记忆类型组成：核心记忆、情景记忆、语义记忆、程序记忆、资源记忆和知识仓库，并配有一个动态控制和协调更新与检索的多智能体框架。 这一设计使智能体能够大规模持久保存、推理并准确检索多样化的长期用户数据。
We validate MIRIX in two demanding settings. First, on ScreenshotVQA, a challenging multimodal benchmark comprising nearly 20,000 high-resolution computer screenshots per sequence, requiring deep contextual understanding and where no existing memory systems can be applied, MIRIX achieves 35% higher accuracy than the RAG baseline while reducing storage requirements by 99.9%. Second, on LOCOMO, a long-form conversation benchmark with single-modal textual input, MIRIX attains state-of-the-art performance of 85.4%, far surpassing existing baselines. These results show that MIRIX sets a new performance standard for memory-augmented LLM Agents. To allow users to experience our memory system, we provide a packaged application powered by MIRIX. It monitors the screen in real time, builds a personalized memory base, and offers intuitive visualization and secure local storage to ensure privacy.
我们在两个高难度场景中验证 MIRIX。 首先，在 ScreenshotVQA 上，每个序列包含近 20,000 张高分辨率计算机屏幕截图，需要深入理解上下文，且现有记忆系统均无法应用；MIRIX 的准确率比 RAG 基线高 35%，同时将存储需求降低 99.9%。 其次，在采用单模态文本输入的长对话基准 LOCOMO 上，MIRIX 达到 85.4% 的最先进性能，远超现有基线。 这些结果表明，MIRIX 为记忆增强型 LLM 智能体确立了新的性能标准。 为了让用户体验我们的记忆系统，我们提供了一个由 MIRIX 驱动的打包应用程序。 它实时监控屏幕、构建个性化记忆库，并提供直观的可视化和安全的本地存储来保障隐私。
1. Introduction
Recent advancements in large language model (LLM) agents have focused primarily on improving their capabilities in complex task execution—ranging from code debugging and repository management to autonomous web browsing. While these functionalities are crucial, another foundational yet underexplored dimension is memory: the ability of agents to persist, retrieve, and utilize past user-specific information over time. Human cognition relies heavily on memory—recalling conversations, recognizing patterns, and adapting behavior based on prior experience. Analogously, memory mechanisms in LLM agents are essential for delivering consistent, personalized interactions, learning from feedback, and avoiding repetitive queries. However, most LLM-based personal assistants remain stateless beyond their current prompt window, retaining no lasting memory unless context is explicitly re-provided. This limitation hinders their long-term usability, especially in real-world settings where users expect assistants to evolve, recall, and personalize over time.
大语言模型（LLM）智能体的近期进展主要聚焦于提升复杂任务执行能力，涵盖代码调试、代码仓库管理和自主网页浏览等任务。 尽管这些功能至关重要，但记忆这一基础而研究不足的维度同样重要：智能体随时间持久保存、检索和利用过去用户专属信息的能力。 人类认知高度依赖记忆，包括回忆对话、识别模式以及根据以往经验调整行为。 同样，LLM 智能体中的记忆机制对于提供一致的个性化交互、从反馈中学习和避免重复询问至关重要。 然而，大多数基于 LLM 的个人助理一旦超出当前提示窗口便仍然是无状态的；除非明确重新提供上下文，否则不会保留持久记忆。 这一限制妨碍了它们的长期可用性，尤其是在用户期望助理能够随时间演化、回忆和个性化的现实环境中。
To address this, a range of memory-augmented systems have been proposed. One common approach is the use of knowledge graphs, as seen in systems like Zep and Cognee. These frameworks are well-suited for representing structured relationships between entities but struggle to model sequential events, emotional states, full-length documents, or multi-modal inputs such as images. Another approach involves flattened memory architectures that store and retrieve textual chunks using vector databases. Examples include Letta, Mem0, and ChatGPT’s memory system. Letta divides its memory into components—recall memory for conversation history, core memory for preferences, and archival memory for long documents—while ChatGPT focuses primarily on core and recall memories. Mem0 adopts a memory system that contains flattened facts distilled from the user inputs, which serves similar roles as Letta's archival memory while being more distilled.
为解决这一问题，研究者提出了一系列记忆增强系统。 一种常见方法是使用知识图谱，Zep 和 Cognee 等系统即采用这种方式。 这些框架非常适合表示实体之间的结构化关系，但难以建模连续事件、情绪状态、完整文档或图像等多模态输入。 另一种方法采用扁平化记忆架构，利用向量数据库存储和检索文本块。 相关例子包括 Letta、Mem0 和 ChatGPT 的记忆系统。 Letta 把记忆划分为用于对话历史的回忆记忆、用于偏好的核心记忆，以及用于长文档的归档记忆；ChatGPT 则主要关注核心记忆和回忆记忆。 Mem0 采用一种包含从用户输入中提炼出的扁平事实的记忆系统，它与 Letta 的归档记忆作用相似，但提炼程度更高。
While prevalent, these memory systems face several challenges: (1) Lack of compositional memory structure: Most approaches store all historical data in a single flat store without routing into specialized memory types (e.g., procedural, episodic, semantic), making retrieval inefficient and less accurate. (2) Poor multi-modal support: Text-centric memory mechanisms fail when the majority of the input is non-verbal (e.g., images, interface layouts, maps). (3) Scalability and abstraction: Storing raw inputs, especially images, leads to prohibitive memory requirements, with no effective abstraction layer to summarize and retain only salient information.
尽管这些记忆系统十分普遍，但它们面临多项挑战：（1）缺少组合式记忆结构：大多数方法把全部历史数据存储在单一扁平存储中，没有将其路由到程序记忆、情景记忆和语义记忆等专门记忆类型，导致检索效率和准确性较低。 （2）多模态支持不足：当大部分输入是图像、界面布局和地图等非语言内容时，以文本为中心的记忆机制便会失效。 （3）可扩展性与抽象：存储原始输入，尤其是图像，会带来难以承受的记忆需求，而且缺少有效的抽象层来总结并只保留显著信息。
To address the limitations of existing memory systems, we argue that effective Routing and Retrieving are the key capabilities a memory-augmented agent must possess. Most current systems focus primarily on Short-Term and Long-Term Memory, with some incorporating Mid-Term Memory. In contrast, we draw inspiration from works that explore more specialized memory types, including episodic memory, semantic memory, and procedural memory. Building on these foundations, we propose a more comprehensive architecture consisting of six memory components: Core Memory, Episodic Memory, Semantic Memory, Procedural Memory, Resource Memory, and the Knowledge Vault (As shown in Figure 1). Episodic Memory stores user-specific events and experiences; Semantic Memory captures concepts and named entities (e.g., the meaning of a new phrase or an understanding of a person); Procedural Memory records step-by-step instructions for performing tasks. Resource Memory is designed to store documents, files, and other media shared by the user. Knowledge Vault holds critical verbatim information that must be preserved exactly, such as addresses, phone numbers, email accounts, and other sensitive facts.
为解决现有记忆系统的局限，我们认为，有效的 路由 与 检索 是记忆增强型智能体必须具备的关键能力。 当前大多数系统主要关注短期记忆和长期记忆，部分系统还纳入中期记忆。 与之不同，我们从探索更专门记忆类型的工作中获得启发，包括情景记忆、语义记忆和程序记忆。 在这些基础上，我们提出一种更全面的架构，由六个记忆组件构成： 核心记忆、 情景记忆、 语义记忆、 程序记忆、 资源记忆 和 知识仓库，如 图1 所示。 情景记忆存储用户专属事件和经历；语义记忆捕获概念和命名实体，例如新短语的含义或对某个人的理解；程序记忆记录执行任务的分步指令。 资源记忆用于存储用户分享的文档、文件和其他媒体。 知识仓库保存必须逐字准确保留的关键信息，例如地址、电话号码、电子邮件账户和其他敏感事实。
Each memory component is internally organized using a hierarchical structure. For example, Episodic Memory includes fields such as summary and details, while Semantic Memory organizes information by name and description. Managing this structured and heterogeneous memory is challenging for a single agent. Therefore, we adopt a multi-agent architecture: six Memory Managers for six memory components, and a Meta Memory Manager responsible for task routing. While this memory system can be plugged and connected with other existing agents, we build an extra Chat Agent to demonstrate how we can interact with an agent that has access to our memories. This memory system, which we call MIRIX, is modular and designed as a comprehensive and full memory system for LLM-based agents. Moreover, when interacting with the Chat Agent, we propose an Active Retrieval mechanism, where the agent is required to generate a topic before answering the question or executing the next step, and the retrieved information is inputted into the model in the system prompt. Meanwhile, we design multiple retrieval tools so that the agent can choose appropriate ones in response to different situations.
每个记忆组件在内部均采用层次结构组织。 例如，情景记忆包含 summary 和 details 等字段，而语义记忆则按照 name 和 description 组织信息。 单个智能体很难管理这种结构化异构记忆。 因此，我们采用多智能体架构：由六个记忆管理器分别负责六个记忆组件，并由一个负责任务路由的 元记忆管理器 统筹。 尽管该记忆系统可以插接并连接到其他现有智能体，我们仍额外构建了一个 聊天智能体，用于展示如何与能够访问我们记忆的智能体交互。 我们把这一记忆系统称为 MIRIX；它采用模块化设计，旨在成为面向基于 LLM 的智能体的综合完整记忆系统。 此外，在与聊天智能体交互时，我们提出主动检索机制：要求智能体在回答问题或执行下一步之前生成一个主题，再把检索到的信息通过系统提示输入模型。 与此同时，我们设计了多种检索工具，使智能体能够针对不同情形选择适当工具。
We evaluate MIRIX in two experimental settings. First, we introduce a challenging benchmark that requires extracting information and building memory from multimodal input. To this end, we collect between 5,000 and 20,000 high-resolution screenshots spanning one month of computer usage from three PhD students and construct evaluation questions grounded in their visual activity history. This setting demands sophisticated memory modeling and exceeds the capacity of existing long-context models. For example, each screenshot ranges from 2K to 4K resolution depending on the user's monitor, which limits Gemini to processing no more than 500 images at full resolution or approximately 3,600 images when resized to 256×256 pixels. Second, following Mem0, we assess performance on the LOCOMO dataset, which comprises long-form, multi-turn conversations. While each conversation is relatively short (around 26,000 tokens on average), we constrain the Chat Agent to answer questions using only retrieved memories, without access to the original conversation transcripts. This setup enables us to evaluate whether our system can effectively distill and route essential information into memory.
我们在两个实验场景中评估 MIRIX。 首先，我们引入一个具有挑战性的基准，要求从多模态输入中抽取信息并构建记忆。 为此，我们从三名博士生长达一个月的计算机使用记录中收集 5,000 至 20,000 张高分辨率屏幕截图，并基于其视觉活动历史构建评测问题。 这一场景需要复杂的记忆建模，并超出现有长上下文模型的能力范围。 例如，依据用户显示器的不同，每张截图的分辨率介于 2K 到 4K，这使 Gemini 在完整分辨率下最多只能处理 500 张图像，或在把图像缩放至 256×256 像素后处理约 3,600 张。 其次，遵循 Mem0，我们在由长篇多轮对话组成的 LOCOMO 数据集上评估性能。 虽然每段对话相对较短，平均约 26,000 个 token，但我们限制聊天智能体只能使用检索到的记忆回答问题，不能访问原始对话记录。 这一设置使我们能够评估系统是否可以有效提炼关键信息并将其路由到记忆中。
In terms of results, because no existing memory systems can handle such a large volume of multimodal input, we compare against retrieval-augmented generation (RAG) baselines and long-context baselines. MIRIX achieves a 35% improvement over RAG baselines while reducing storage requirements by 99.9%, and a 410% improvement over long-context baselines with a 93.3% reduction in storage. On LOCOMO, MIRIX reaches state-of-the-art performance with an overall accuracy of 85.38%, outperforming the best existing method by 8.0% and approaching the upper bound set by long-context models. To make our memory system more accessible, we also developed a personal assistant powered by MIRIX. With the user’s permission to capture screen content, the assistant continuously builds a memory from screenshots and can answer any questions related to this accumulated information. This represents our effort to bring advanced memory capabilities to everyone, enabling users to experience the benefits of our system firsthand.
在实验结果方面，由于现有记忆系统均无法处理如此大规模的多模态输入，我们将其与检索增强生成（RAG）基线和长上下文基线进行比较。 MIRIX 相比 RAG 基线提升 35%，同时将存储需求降低 99.9%；相比长上下文基线提升 410%，并将存储量降低 93.3%。 在 LOCOMO 上，MIRIX 以 85.38% 的总体准确率达到最先进性能，比最佳现有方法高 8.0%，并接近长上下文模型设定的上界。 为了让我们的记忆系统更易使用，我们还开发了一个由 MIRIX 驱动的个人助理。 在获得用户截取屏幕内容的许可后，该助理会持续从截图中构建记忆，并能回答与这些累积信息有关的任何问题。 这是我们向所有人提供先进记忆能力的尝试，使用户能够亲身体验系统带来的益处。
Our contributions are summarized as follows:
- We analyze the limitations of existing memory architectures for LLM-based agents and propose a novel memory system composed of six specialized components and eight different agents.
- We introduce a new benchmark in which the agent must interpret a large collection of screenshots to build an understanding of user behavior—posing a significant challenge to the memory.
- Through experiments on our new benchmark and existing benchmark LOCOMO, we demonstrate that MIRIX significantly outperform existing memory systems.
- To make our approach accessible, we build and release a personal assistant application powered by MIRIX, enabling users to experience advanced memory capabilities in real-world scenarios.
我们的贡献总结如下：
- 我们分析了现有基于 LLM 的智能体记忆架构的局限，并提出一种由六个专门组件和八个不同智能体构成的新型记忆系统。
- 我们引入一个新基准，要求智能体解读大量屏幕截图来理解用户行为，从而对记忆提出重大挑战。
- 通过在新基准和现有 LOCOMO 基准上的实验，我们证明 MIRIX 显著优于现有记忆系统。
- 为使我们的方法更易使用，我们构建并发布了由 MIRIX 驱动的个人助理应用，使用户能够在现实场景中体验先进记忆能力。
2. Application & Use Cases
2.1 MIRIX Application
To demonstrate the full functionality of our memory system, we developed a cross-platform application using React-Electron for the frontend and Uvicorn as the backend server. Moreover, we release the obtained file for direct installation. Within the application, users can activate screen monitoring by selecting the ScreenShots tab. This enables the agent to observe the user’s screen activity and dynamically update its memory, gradually building a contextual understanding of the user over time. With the obtained memory, the agent can answer questions related to the memories.
为了展示记忆系统的完整功能，我们开发了一个跨平台应用，前端使用 React-Electron，后端服务器使用 Uvicorn。 此外，我们发布了生成的文件，供用户直接安装。 在应用中，用户可以选择 ScreenShots 标签页来启用屏幕监控。 这使智能体能够观察用户的屏幕活动并动态更新记忆，逐步形成对用户的上下文理解。 利用获得的记忆，智能体可以回答与这些记忆相关的问题。
Memory Updates. The application captures a screenshot every 1.5 seconds. To reduce redundancy, images that are visually similar to previously captured ones are discarded. Once 20 unique screenshots are collected, the memory update process is triggered—typically around every 60 seconds. During this process, relevant information is extracted and incorporated into the system's memory components. To reduce latency in processing user screenshots, we adopt a streaming upload strategy. Instead of batching and sending 20 images at once, we upload each screenshot immediately upon receiving it from the front-end. By leveraging the Gemini API—which supports loading images via Google Cloud URLs—we can efficiently transmit visual data without waiting for the full batch to accumulate. This approach significantly reduces end-to-end latency from approximately 50 seconds (as observed when using GPT-4 with direct image upload) to under 5 seconds using Gemini.
记忆更新。 应用每 1.5 秒截取一次屏幕。 为减少冗余，系统会丢弃与先前截图在视觉上相似的图像。 收集到 20 张不同截图后，系统便触发记忆更新流程，通常约每 60 秒一次。 在此过程中，相关信息会被抽取并整合到系统的记忆组件中。 为了降低处理用户截图的延迟，我们采用流式上传策略。 我们不会一次性批量发送 20 张图像，而是在从前端收到每张截图后立即上传。 借助支持通过 Google Cloud URL 加载图像的 Gemini API，我们无需等待完整批次积累完毕，即可高效传输视觉数据。 与使用 GPT-4 直接上传图像时约 50 秒的延迟相比，该方法使用 Gemini 后把端到端延迟显著降低至 5 秒以内。
Chat Interface. The chat interface allows users to interact with the agent, which has full access to its accumulated memories. As illustrated in Figure 2, users can query the assistant about past activities, and the agent can respond based on its memory content, enabling more informed and personalized interactions.
聊天界面。 聊天界面允许用户与能够完整访问其累积记忆的智能体交互。 如 图2 所示，用户可以向助理询问过去的活动，智能体则可根据记忆内容作答，从而实现信息更充分、更加个性化的交互。
Memory Visualization. After observing the screen for a sufficient period, the agent organizes its knowledge into structured memory components. An example of Semantic Memory (organized into a tree structure) is shown in Figure 3. We also provide a list view of the memories. An example of Procedural Memory is shown in Figure 4.
记忆可视化。 在观察屏幕足够长的时间后，智能体会把知识组织为结构化记忆组件。 图3 展示了采用树形结构组织的语义记忆示例。 我们还提供记忆的列表视图。 图4 展示了程序记忆示例。
2.2 Memory System for Wearable Devices
The wearable device market has seen rapid growth in recent years, driven by the increasing demand for intelligent, always-available personal assistants. Products like AI-powered glasses (e.g., Meta Ray-Ban, XREAL Air) and AI pins (e.g., Humane, Rabbit R1) aim to integrate seamless interaction into daily life through voice commands, visual capture, and real-time feedback. However, these devices often lack a long-term memory component that allows them to evolve with the user—retaining useful information over time, adapting to personal routines, and referencing past interactions in context-aware ways.
近年来，可穿戴设备市场快速增长，其推动力来自人们对智能、随时可用的个人助理日益增长的需求。 AI 眼镜（如 Meta Ray-Ban、XREAL Air）和 AI 徽章（如 Humane、Rabbit R1）等产品，旨在通过语音命令、视觉捕获和实时反馈，把无缝交互融入日常生活。 然而，这些设备往往缺少能使其随用户共同演化的长期记忆组件，无法随时间保留有用信息、适应个人日常习惯，并以感知上下文的方式引用过去的交互。
Our memory system is well-suited for integration into such wearable devices. By continuously collecting and processing data streams such as audio, visual scenes, and user queries, our system enables real-time memory formation. For instance, AI glasses equipped with our memory systems can automatically summarize meetings, remember frequently visited places, recognize recurring visual patterns, and recall previous conversations or tasks. With MIRIX, it can also evolve with the user and build a memory that is specifically made for the user.
我们的记忆系统非常适合集成到这类可穿戴设备中。 通过持续收集和处理音频、视觉场景和用户查询等数据流，系统可以实时形成记忆。 例如，配备我们记忆系统的 AI 眼镜可以自动总结会议、记住经常到访的地点、识别反复出现的视觉模式，并回忆先前的对话或任务。 借助 MIRIX，它还可以随用户共同演化，并构建专门面向该用户的记忆。
Moreover, the system’s modular memory architecture—including procedural, episodic, semantic, and resource memory—aligns naturally with the needs of lightweight, on-the-go devices. Procedural memory enables the assistant to learn user habits (e.g., daily routes, meeting structures), while semantic memory stores general knowledge about the user's preferences, environment, and routines. Episodic memory captures time-stamped, situational experiences, and can be queried for recalling specific events (e.g., “What did I see at the conference last week?”). Semantic Memory, on the other hand, can help organize the clients someone has seen over the past week and list them in a tree structure with details of their discussions.
此外，系统的模块化记忆架构——包括程序记忆、情景记忆、语义记忆和资源记忆——与轻量级移动设备的需求自然契合。 程序记忆使助理能够学习用户习惯，例如日常路线和会议结构；语义记忆则存储有关用户偏好、环境和日常习惯的一般知识。 情景记忆捕获带时间戳的情境化经历，并可用于查询和回忆特定事件，例如“我上周在会议上看到了什么？” 另一方面，语义记忆可以帮助组织某人在过去一周见过的客户，并以树形结构列出他们及其讨论细节。
Given the constraints of wearable hardware (limited compute and storage), our design also supports hybrid on-device/cloud memory management. Critical information in Knowledge Vault can be stored locally while large-scale memories such as Resource Memory can be offloaded and retrieved from the cloud on demand. In summary, our memory system serves as a cognitive backbone for wearable AI agents—enabling personalization, continuity, and intelligence at the edge. As the wearable market matures, embedding persistent, structured memory will be a key differentiator for next-generation AI assistants.
考虑到可穿戴硬件的计算和存储限制，我们的设计还支持设备端与云端混合记忆管理。 知识仓库中的关键信息可存储在本地，资源记忆等大规模记忆则可转移到云端，并按需检索。 总而言之，我们的记忆系统充当可穿戴 AI 智能体的认知骨干，在边缘侧实现个性化、连续性和智能。 随着可穿戴设备市场逐渐成熟，嵌入持久的结构化记忆将成为下一代 AI 助理的关键差异化因素。
2.3 Agent Memory Marketplace
We envision a future where personal memory—collected and structured through AI agents—becomes a new digital asset class. In the AI era, memory is no longer just a passive log of past events, but an active, evolving knowledge base that can be shared, personalized, and monetized. The Agent Memory Marketplace is our proposal for a decentralized ecosystem where memory is exchanged, reused, and built collaboratively through AI agents.
我们设想，在未来，由 AI 智能体收集和组织的个人记忆将成为一类新的数字资产。 在 AI 时代，记忆不再只是过去事件的被动日志，而是可以共享、个性化和变现的主动演化知识库。 智能体记忆市场是我们提出的去中心化生态系统，其中的记忆通过 AI 智能体进行交换、复用和协同构建。
We begin with a core belief: human memory will become the most valuable and irreplaceable asset in the age of AI. Unlike static data, memory encompasses lived experiences, subjective context, preferences, and interactions—making it deeply personal, yet highly reusable by intelligent systems. The marketplace is structured into three key layers:
1. AI Agents Infrastructure. Our technology provides infrastructure for lifetime intelligent, interactive agents. Examples include (1) Personal AI Assistants & Companions: Tailored agents that continuously learn and evolve with the user. (2) AI Wearables: AI agents embedded in decentralized physical infrastructure (e.g., smart glasses, AI pins) that extend memory capture to the real world. (3) Multi-Agent Systems: Collaborative agents with shared memory access, enabling coordination and collective intelligence.
2. Privacy-Preserving Memory Infrastructure. To support trust and adoption, we aim to build in a robust privacy layer: (1) Encryption Layer: All memories are stored using end-to-end encryption. (2) Privacy Control: Fine-grained permissions allow users to choose which parts of their memory to share, trade, or restrict. (3) Decentralized Storage: Memories are stored in a distributed, censorship-resistant infrastructure.
3. Memory Marketplace and Social Function. A peer-to-peer ecosystem for sharing, aggregating, and trading memories encoded in AI agents. Use cases include: (1) Memory Social/Trading: Tokenized exchange of memories (e.g., productivity hacks, niche workflows, or life advice). (2) Expert Communities: Collective memory-building for domain-specific expertise such as finance, education, or pet care. (3) Fan Economy and Dating Applications: Users can subscribe to AI personas based on influencer or celebrity memories, allowing fans to interact with memory-rich digital replicas of their favorite personalities. This technology also enables the creation of AI clones for accelerated dating and matching processes, where users can engage with AI agents in interactions before meeting in person.
我们的核心信念是： 人类记忆将成为 AI 时代最有价值且不可替代的资产。 与静态数据不同，记忆涵盖亲身经历、主观上下文、偏好和交互，因此既高度个人化，又可以被智能系统充分复用。 该市场分为三个关键层次：
1. AI 智能体基础设施。 我们的技术为终身智能交互式智能体提供基础设施。相关例子包括：（1）个人 AI 助理与伙伴：持续学习并随用户共同演化的定制智能体；（2）AI 可穿戴设备：嵌入智能眼镜、AI 徽章等去中心化物理基础设施的 AI 智能体，把记忆捕获扩展到现实世界；（3）多智能体系统：共享记忆访问能力的协作智能体，实现协调与集体智能。
2. 隐私保护型记忆基础设施。 为建立信任并推动采用，我们计划构建稳健的隐私层：（1）加密层：所有记忆均使用端到端加密存储；（2）隐私控制：细粒度权限允许用户选择要共享、交易或限制的记忆部分；（3）去中心化存储：记忆存储在分布式、抗审查的基础设施中。
3. 记忆市场与社交功能。 这是一个用于共享、聚合和交易编码在 AI 智能体中的记忆的点对点生态系统。相关用例包括：（1）记忆社交与交易：以 token 化方式交换生产力技巧、小众工作流或生活建议等记忆；（2）专家社区：面向金融、教育或宠物护理等领域专门知识的集体记忆构建；（3）粉丝经济与约会应用：用户可以订阅基于网红或名人记忆构建的 AI 人格，使粉丝与其喜爱人物的记忆丰富型数字复制体交互。该技术还支持创建 AI 克隆来加速约会和匹配流程，使用户可在真人见面前先与 AI 智能体交互。
In summary, we envision a future where personal memory transcends its traditional role to become an active, valuable digital asset. By combining lifelong AI agents, privacy-preserving infrastructure, and a decentralized marketplace, we aim to create an ecosystem where memories can be securely captured, meaningfully shared, and collectively advanced. This approach not only empowers individuals to benefit from their own experiences but also unlocks new possibilities for collaboration, personalization, and economic value in the AI era.
总而言之，我们设想个人记忆将超越传统角色，成为主动且有价值的数字资产。 通过结合终身 AI 智能体、隐私保护基础设施和去中心化市场，我们旨在创建一个能够安全捕获、有效共享并共同推进记忆的生态系统。 这种方法不仅使个人能够从自身经历中受益，也为 AI 时代的协作、个性化和经济价值开启新的可能性。
3. Methodology
3.1 Memory Components
We design a modular memory architecture consisting of six distinct components: Core Memory, Episodic Memory, Semantic Memory, Procedural Memory, Resource Memory, and the Knowledge Vault. Each component is structurally and functionally tailored to capture different aspects of user interaction and world knowledge, enabling the agent to retrieve, reason, and act effectively across time and tasks.
我们设计了一种模块化记忆架构，由六个不同组件组成： 核心记忆、 情景记忆、 语义记忆、 程序记忆、 资源记忆 和 知识仓库。 每个组件都在结构和功能上针对用户交互与世界知识的不同方面进行定制，使智能体能够跨时间和任务有效检索、推理与行动。
Core Memory. Core Memory stores high-priority, persistent information that should always remain visible to the agent when engaging with the user. Inspired by the design of MemGPT, this memory is divided into two primary blocks: persona and human. The persona block encodes the identity, tone, or behavior profile of the agent, while the human block stores enduring facts about the user such as their name, preferences, and self-identifying attributes (e.g., “User’s name is David”, “User enjoys Japanese cuisine”). When the memory size exceeds 90% of capacity, the system triggers a controlled rewrite process to maintain compactness without losing critical information.
核心记忆。 核心记忆存储高优先级的持久信息，智能体与用户交互时应始终能够看到这些信息。 受 MemGPT 设计启发，该记忆被划分为两个主要块：persona 和 human。 persona 块编码智能体的身份、语气或行为画像，human 块则存储有关用户的持久事实，例如姓名、偏好和自我身份属性，如“用户的名字是 David”“用户喜欢日本料理”。 当记忆大小超过容量的 90% 时，系统会触发受控重写流程，在不丢失关键信息的情况下保持紧凑。
Episodic Memory. Episodic Memory captures time-stamped events and temporally grounded interactions that reflect the user's behavior, experiences, or activities. It functions as a structured log or calendar, enabling the agent to reason about user routines, recency, and context-aware follow-ups. Each entry is defined by the following fields: event_type (e.g., user_message, inferred_result, system_notification), summary (a concise natural language description of the event), details (extended contextual information, including dialog excerpts or inferred states), actor (the origin of the event, either user or assistant), and timestamp (e.g., “2025-03-05 10:15”). This structure allows the agent to temporally index memory and track change over time, such as identifying ongoing tasks or following up on pending actions.
情景记忆。 情景记忆捕获带时间戳的事件和具有时间依据的交互，以反映用户的行为、经历或活动。 它充当结构化日志或日历，使智能体能够针对用户日常习惯、信息新近程度和感知上下文的后续事项进行推理。 每个条目由以下字段定义：event_type，例如 user_message、inferred_result、system_notification；summary，即事件的简明自然语言描述；details，即包括对话摘录或推断状态在内的扩展上下文信息；actor，即事件来源，可为 user 或 assistant；以及 timestamp，例如“2025-03-05 10:15”。 这一结构使智能体能够按时间索引记忆并追踪随时间发生的变化，例如识别正在进行的任务或跟进待处理操作。
Semantic Memory. Semantic Memory maintains abstract knowledge and factual information that is independent of specific times or events. This component serves as a knowledge base for general concepts, entities, and relationships — whether about the world or the user's social graph. For example, it may store entries like “Harry Potter is written by J.K. Rowling” or “John is a friend of the user who enjoys jogging and lives in San Francisco.” Each entry includes a name (the concept or entity identifier), summary (a concise definition or relationship statement), details (expanded background or contextual explanation), and source (e.g., user_provided, Wikipedia, or inferred from conversation). Unlike episodic memory, semantic entries are intended to persist unless conceptually overwritten and support reasoning over social, geographic, or commonsense knowledge.
语义记忆。 语义记忆维护与特定时间或事件无关的抽象知识和事实信息。 该组件充当一般概念、实体和关系的知识库，无论这些内容涉及世界还是用户的社交图谱。 例如，它可以存储“《哈利·波特》由 J.K. Rowling 创作”或“John 是用户的朋友，他喜欢慢跑并住在旧金山”等条目。 每个条目包括 name，即概念或实体标识符；summary，即简明定义或关系陈述；details，即扩展背景或上下文解释；以及 source，例如 user_provided、Wikipedia 或从对话中推断。 与情景记忆不同，除非在概念层面被覆盖，否则语义条目会持续存在，并支持针对社会、地理或常识知识的推理。
Procedural Memory. Procedural Memory stores structured, goal-directed processes such as how-to guides, operational workflows, and interactive scripts. These are neither time-sensitive (episodic) nor abstract facts (semantic) but instead represent actionable knowledge that can be invoked to assist the user with complex tasks. Typical examples include “how to file a travel reimbursement form,” “steps to set up a Zoom meeting,” or “how to book a restaurant via OpenTable.” Each entry includes an entry_type (workflow, guide, or script), a description of the goal or function, and steps expressed as a list of instructions (optionally in JSON or structured format). This memory component supports instructional planning, automation, and decomposition of user goals into sub-tasks.
程序记忆。 程序记忆存储结构化、目标导向的流程，例如操作指南、运行工作流和交互脚本。 这些内容既不是时间敏感的情景，也不是抽象的语义事实，而是能够被调用来帮助用户完成复杂任务的可行动知识。 典型例子包括“如何提交差旅报销表”“设置 Zoom 会议的步骤”或“如何通过 OpenTable 预订餐厅”。 每个条目包括 entry_type，可为 workflow、guide 或 script；目标或功能的 description；以及表示为指令列表的 steps，后者可选用 JSON 或结构化格式。 该记忆组件支持指令规划、自动化，以及把用户目标分解为子任务。
Resource Memory. Resource Memory handles full or partial documents, transcripts, or multi-modal files that the user is actively engaged with but do not fit into other memory categories. For instance, if the user is reading a friend's detailed picnic plan or a project proposal document, the agent can store and retrieve that information from Resource Memory. This component enables context continuity in long-running tasks. Each entry includes a title (resource name), summary (brief overview and context), resource_type (e.g., doc, markdown, pdf_text, image, voice_transcript), and the full or excerpted content. This design enables the agent to reference previously seen material, quote from documents, or search within them to aid the user’s workflow.
资源记忆。 资源记忆处理用户正在使用、但不属于其他记忆类别的完整或部分文档、转录文本和多模态文件。 例如，如果用户正在阅读朋友的详细野餐计划或项目提案文档，智能体可以在资源记忆中存储和检索这些信息。 该组件使长期运行任务能够保持上下文连续性。 每个条目包括 title，即资源名称；summary，即简要概述和上下文；resource_type，例如 doc、markdown、pdf_text、image、voice_transcript；以及完整或摘录的 content。 这一设计使智能体能够引用先前看过的材料、引述文档内容或在文档中搜索，从而辅助用户的工作流。
Knowledge Vault. The Knowledge Vault serves as a secure repository for verbatim and sensitive information such as credentials, addresses, contact information, and API keys. These entries are not typically relevant to conversation-level reasoning but are crucial for performing authenticated tasks or storing long-term identifiers. Each entry includes an entry_type (e.g., credential, bookmark, contact_info, api_key), a source (e.g., user_provided, github), a sensitivity level (low, medium, high), and the actual secret_value. Entries with high sensitivity are protected via access control and excluded from casual retrieval to prevent misuse or leakage.
知识仓库。 知识仓库充当逐字信息和敏感信息的安全存储库，例如凭据、地址、联系信息和 API 密钥。 这些条目通常与对话层面的推理无关，但对于执行需要身份验证的任务或存储长期标识符至关重要。 每个条目包括 entry_type，例如 credential、bookmark、contact_info、api_key；source，例如 user_provided、github；sensitivity 级别，即 low、medium 或 high；以及实际的 secret_value。 高敏感度条目通过访问控制得到保护，并从常规检索中排除，以防止滥用或泄露。
3.2 Active Retrieval and Retrieval Design
In many memory-augmented systems (e.g., Mem0, MemGPT), memory retrieval must be explicitly triggered. Otherwise, the language model often defaults to its parametric knowledge, which may be outdated or incorrect. For example, suppose the user previously said, “The CEO of Twitter is Linda Yaccarino,” and this information was saved in memory. A few days later, when this message is no longer present in the conversation history, the user might ask, “Who is the CEO of Twitter?” In this case, the language model may rely on outdated knowledge and incorrectly respond with “Elon Musk.” While explicitly instructing the model to “search your memory” can mitigate such errors, doing so repeatedly is impractical in natural conversations.
在许多记忆增强系统中，例如 Mem0 和 MemGPT，记忆检索必须被明确触发。 否则，语言模型通常会默认依赖其参数知识，而这些知识可能已经过时或并不正确。 例如，假设用户先前说过“Twitter 的首席执行官是 Linda Yaccarino”，并且该信息已保存到记忆中。 几天后，当这条消息已不在对话历史中时，用户可能会问：“Twitter 的首席执行官是谁？” 在这种情况下，语言模型可能依赖过时知识，错误回答“Elon Musk”。 尽管明确要求模型“搜索你的记忆”可以减少此类错误，但在自然对话中反复这样做并不现实。
To address this, we propose an Active Retrieval mechanism. As illustrated in Figure 5, the system operates in two stages: first, the agent generates a current topic based on the input context; second, this topic is used to retrieve relevant memories from each memory component. Retrieved results are then injected into the system prompt. For instance, given the query “Who is the CEO of Twitter?”, the agent may infer the topic “CEO of Twitter”, which is then used to retrieve the top-10 most relevant entries from each of the six memory components. Retrieved content is tagged according to its source, such as <episodic_memory>... </episodic_memory>, ensuring the model is aware of both the content and its origin. This automatic retrieval pipeline eliminates the need for explicit user prompts to trigger memory access and ensures the model can incorporate up-to-date, personalized, or contextual information during response generation.
为解决这一问题，我们提出 主动检索 机制。 如 图5 所示，系统分两个阶段运行：首先，智能体根据输入上下文生成一个当前主题；其次，系统使用该主题从每个记忆组件中检索相关记忆。 随后，检索结果被注入系统提示。 例如，对于查询“Twitter 的首席执行官是谁？”，智能体可能推断出“Twitter 首席执行官”这一主题，再用它从六个记忆组件的每一个中检索最相关的 top-10 条目。 检索内容会根据来源添加标签，例如 <episodic_memory>... </episodic_memory>，确保模型同时了解内容及其来源。 这一自动检索流水线无需用户通过显式提示触发记忆访问，并确保模型在生成响应时能够纳入最新、个性化或上下文相关的信息。
In addition to Active Retrieval, we support multiple retrieval functions, including embedding_match, bm25_match, and string_match. We are actively expanding this set with more diverse and specialized retrieval strategies, ensuring that each method is well-differentiated so the agent can choose the most appropriate one to invoke based on context.
除主动检索外，我们还支持多种检索函数，包括 embedding_match、bm25_match 和 string_match。 我们正积极使用更多样、更专门的检索策略扩展该集合，并确保不同方法之间具有明确差异，使智能体能够依据上下文选择最适合调用的方法。
3.3 Multi-Agent Workflow
To manage the dynamic and heterogeneous nature of user interactions, we adopt a modular multi-agent architecture. This system orchestrates input processing, memory updating, and information retrieval across six distinct memory components through a coordinated and efficient workflow. The overall system is governed by a central Meta Memory Manager and a set of specialized Memory Managers, each responsible for maintaining one memory type.
为了管理用户交互的动态异构特性，我们采用模块化多智能体架构。 该系统通过协调高效的工作流，在六个不同的记忆组件之间编排输入处理、记忆更新和信息检索。 整个系统由一个中央元记忆管理器以及一组专门的记忆管理器管理，每个记忆管理器负责维护一种记忆类型。
Memory Update Workflow. As illustrated in Figure 6, when new input is received from the user, the system first automatically performs a search over the memory base. The retrieved information, together with the user input, is passed to the Meta Memory Manager. The Meta Memory Manager then analyzes the content and determines which memory components are relevant, routing the input to the corresponding Memory Managers. These Memory Managers update their respective memories in parallel while ensuring that redundant information is avoided within each memory type. After completing the updates, they report back to the Meta Memory Manager, which finally sends an acknowledgment confirming that the memory update process is complete.
记忆更新工作流。 如 图6 所示，当系统收到用户的新输入时，首先自动搜索记忆库。 检索到的信息与用户输入一起传递给元记忆管理器。 随后，元记忆管理器分析内容并确定相关记忆组件，再把输入路由到对应的记忆管理器。 这些记忆管理器并行更新各自的记忆，同时确保每种记忆类型中不包含冗余信息。 更新完成后，它们向元记忆管理器报告，后者最终发送确认消息，表示记忆更新流程已经完成。
Conversational Retrieval Workflow. For interactive dialogues, the Chat Agent manages natural language communication with the user. To ground its responses in prior knowledge, it first performs an automatic search over the memory base upon receiving a user query. This initial search is a coarse retrieval spanning all six memory components and returns high-level summaries rather than detailed content. The Chat Agent then analyzes the query to determine which memory components warrant more targeted searches and selects appropriate retrieval methods accordingly. After obtaining the relevant results, it consolidates the information and synthesizes the final response. Moreover, if the user’s query involves updating memory—for example, providing new facts or corrections-the Chat Agent can interact directly with the corresponding Memory Managers to apply precise updates to specific memory components.
对话检索工作流。 在交互式对话中，聊天智能体负责与用户进行自然语言交流。 为使响应以先前知识为依据，它在收到用户查询时首先自动搜索记忆库。 该初始搜索是跨越全部六个记忆组件的粗粒度检索，返回高层摘要而非详细内容。 随后，聊天智能体分析查询，确定哪些记忆组件值得进行更有针对性的搜索，并据此选择适当的检索方法。 获得相关结果后，它会整合信息并合成最终响应。 此外，如果用户查询涉及更新记忆，例如提供新事实或更正，聊天智能体可以直接与相应的记忆管理器交互，对特定记忆组件进行精确更新。
4. Experiments
4.1 Experimental Setup
Datasets
ScreenshotVQA. We collect a new dataset that contains three PhD students' activities. We created a script that takes a screenshot every second. If the image taken this second is too similar to the last one (similarity
ScreenshotVQA。 我们收集了一个包含三名博士生活动的新数据集。 我们创建了一个每秒截取一次屏幕的脚本。 如果当前一秒截取的图像与上一张图像过于相似，相似度
LOCOMO. Following Mem0, we choose LOCOMO dataset for a vertical comparison between MIRIX and existing memory systems. LOCOMO has 10 conversations, where each conversation has 600 dialogues and 26000 tokens on average. There are averagely 200 questions for each conversation, which is suitable for the evaluation for memory systems where we are reuiqred to inject the conversation into the memory and then answer the 200 questions according to the obtained memory. The questions can be classified into multiple categories: single-hop, multi-hop, temporal and open-domain. The dataset also has another category called “adversarial”, which is used to test whether the system can identify unanswerable questions. Following Mem0, we exclude this category in our setting to provide fair comparisons with the earlier methods.
LOCOMO。 遵循 Mem0，我们选择 LOCOMO 数据集，对 MIRIX 与现有记忆系统进行纵向比较。 LOCOMO 包含 10 段对话，每段对话平均有 600 轮交流和 26,000 个 token。 每段对话平均有 200 个问题，适合评估需要把对话注入记忆、再依据获得的记忆回答这 200 个问题的记忆系统。 这些问题可分为单跳、多跳、时间和开放域等多个类别。 该数据集还有一个称为“对抗”的类别，用于测试系统能否识别不可回答的问题。 遵循 Mem0，为了与先前方法进行公平比较，我们在设置中排除该类别。
Evaluation Metrics and Implementation Details
Evaluation Metrics. In this paper, for both of the above datasets, we mainly consider the metric LLM-as-a-Judge. Specifically, we use GPT-4.1 as the judge to look at the question, answer, and response to predict whether the response addresses the question successfully.
评测指标。 对上述两个数据集，本文主要考虑 LLM-as-a-Judge 指标。 具体而言，我们使用 GPT-4.1 作为裁判，查看问题、答案和响应，预测响应是否成功回答问题。
Implementation Details for ScreenshotVQA. Across all experiments, we use the gemini-2.5-flash-preview-04-17 model as the backbone. We selected Gemini because it integrates seamlessly with Google Cloud, enabling asynchronous image uploads and retrieval. This significantly accelerates processing, as each step requires multiple function calls: one to the meta memory manager and between zero and six to the other memory managers.
ScreenshotVQA 实现细节。 在所有实验中，我们使用 gemini-2.5-flash-preview-04-17 模型作为骨干。 我们选择 Gemini，是因为它可以与 Google Cloud 无缝集成，支持异步上传和检索图像。 这显著加快了处理速度，因为每一步都需要多次函数调用：一次调用元记忆管理器，另有零到六次调用其他记忆管理器。
Implementation Details for LOCOMO. For MIRIX, since every agent needs to call many functions to successfully insert the information into the memory systems, it requires the language model to have strong abilities on function calling. To this end, we use gpt-4.1-mini as our backbone model as it shows a stronger ability than gpt-4o-mini in terms of function calling. This is also revealled in Berkeley Function Calling Benchmark where gpt-4o-mini has multi-turn overall acc as 22.12, vs the acc of gpt-4.1-mini being 29.75. For LOCOMO dataset, we run the baselines LangMem, RAG-500, Mem0 using the code provided in Mem0 with setting MODEL in the .env file as gpt-4.1-mini. Then for Zep, we use the code from their official repo and replace the backbone model with gpt-4.1-mini. For all the baselines, we run their code once, while for MIRIX and Full-Context method, we run three times to report the average scores. We provide the results of each single run in Appendix. The complete code for our evaluation and the predicted results from various baselines and MIRIX are provided in the public_evaluation branch in our official repository.
LOCOMO 实现细节。 对 MIRIX 而言，每个智能体都需要调用许多函数，才能成功把信息插入记忆系统，因此语言模型必须具备很强的函数调用能力。 为此，我们使用 gpt-4.1-mini 作为骨干模型，因为它在函数调用方面表现出比 gpt-4o-mini 更强的能力。 Berkeley Function Calling Benchmark 也反映了这一点：gpt-4o-mini 的多轮总体准确率为 22.12，而 gpt-4.1-mini 为 29.75。 对于 LOCOMO 数据集，我们使用 Mem0 提供的代码运行 LangMem、RAG-500 和 Mem0 基线，并把 .env 文件中的 MODEL 设置为 gpt-4.1-mini。 对于 Zep，我们使用其官方代码仓库中的代码，并把骨干模型替换为 gpt-4.1-mini。 对于所有基线，我们将其代码运行一次；对于 MIRIX 和完整上下文方法，我们运行三次并报告平均分。 各次独立运行的结果见附录。 我们在官方代码仓库的 public_evaluation 分支中提供完整评测代码，以及不同基线和 MIRIX 的预测结果。
4.2 Experimental Results on ScreenshotVQA
For ScreenshotVQA, since existing memory systems such as Letta, Mem0 still lack the ability to process multimodal input, we omit the comparisons with them. We consider the following baselines: Gemini. As a long-context baseline, Gemini directly ingests the full set of screenshots to answer the questions. Because the original high-resolution screenshots exceed the model’s context window, we resize them to 256×256 pixels, enabling approximately 3,600 images to fit into a single prompt. We then include the most recent 3,600 images in the prompt and query the model for an answer. Specifically, we use the gemini-2.5-flash-preview-04-17 model. SigLIP. SigLIP serves as a retrieval-augmented baseline. We first use SigLIP to identify the top 50 most relevant images for each query and then prompt Gemini to generate the answer based on these retrieved images. In this setup, we employ google/siglip-so400m-patch14-384 as the retriever and gemini-2.5-flash-preview-04-17 as the language model.
对于 ScreenshotVQA，由于 Letta、Mem0 等现有记忆系统仍缺少处理多模态输入的能力，我们不与它们进行比较。 我们考虑以下基线： Gemini。 作为长上下文基线，Gemini 直接摄入完整截图集合来回答问题。 由于原始高分辨率截图超出模型的上下文窗口，我们把它们缩放到 256×256 像素，使单个提示中能够容纳约 3,600 张图像。 随后，我们在提示中加入最近的 3,600 张图像，并向模型查询答案。 具体而言，我们使用 gemini-2.5-flash-preview-04-17 模型。 SigLIP。 SigLIP 充当检索增强基线。 我们首先使用 SigLIP 为每个查询识别最相关的 50 张图像，再提示 Gemini 根据这些检索图像生成答案。 在该设置中，我们采用 google/siglip-so400m-patch14-384 作为检索器，并采用 gemini-2.5-flash-preview-04-17 作为语言模型。
We evaluate all methods using two metrics: (1) Accuracy, measured via an LLM-as-a-Judge approach comparing the generated responses to ground-truth answers and reporting the average accuracy across the three participants; and (2) Storage, defined as follows—For Gemini, we report the total size of the images after resizing to 256×256. For SigLIP, we report the total size of the original retrieved images in their native 2K–4K resolution. For MIRIX, we use sqlite as the storage backend and report the size of the sqlite.db file containing all extracted information.
我们使用两个指标评估所有方法：（1）准确率，采用 LLM-as-a-Judge 方法，通过比较生成响应与标准答案来测量，并报告三名参与者的平均准确率；（2）存储量，定义如下：对于 Gemini，我们报告缩放至 256×256 后图像的总大小。 对于 SigLIP，我们报告以原始 2K–4K 分辨率保存的检索图像的总大小。 对于 MIRIX，我们使用 sqlite 作为存储后端，并报告包含所有抽取信息的 sqlite.db 文件大小。
The results are summarized in Table 1. As shown, MIRIX significantly outperforms all existing baselines while requiring substantially less storage. Specifically, because SigLIP retains the full set of high-resolution images, its storage footprint is very large, corresponding to the total size of all images in their original 2K–4K resolution. The long-context baseline (Gemini) reduces this by resizing images to 256×256 pixels, but still incurs considerable storage overhead. In contrast, MIRIX avoids storing raw images altogether and instead maintains only the compact sqlite database of extracted information, resulting in a much smaller storage size. Compared to retrieval-augmented generation (RAG) baselines, MIRIX achieves a 35% improvement in accuracy while reducing storage requirements by 99.9%. Relative to the long-context Gemini baseline, MIRIX yields a 410% improvement in accuracy with a 93.3% reduction in storage.
结果汇总于 表1。 如表所示，MIRIX 显著优于所有现有基线，同时所需存储量大幅减少。 具体而言，由于 SigLIP 保留完整的高分辨率图像集合，其存储占用非常大，对应所有图像以原始 2K–4K 分辨率保存时的总大小。 长上下文基线 Gemini 通过把图像缩放到 256×256 像素来减少存储量，但仍带来相当大的存储开销。 相比之下，MIRIX 完全避免存储原始图像，只维护包含抽取信息的紧凑 sqlite 数据库，因此存储量小得多。 相比检索增强生成（RAG）基线，MIRIX 的准确率提升 35%，同时存储需求降低 99.9%。 相比长上下文 Gemini 基线，MIRIX 的准确率提升 410%，存储量降低 93.3%。
| Method | Student 1 | Student 2 | Student 3 | Overall | ||||
|---|---|---|---|---|---|---|---|---|
| Acc ↑ | Storage ↓ | Acc ↑ | Storage ↓ | Acc ↑ | Storage ↓ | Acc ↑ | Storage ↓ | |
| Gemini | 0.0000 | 142.10MB | 0.0952 | 438.86MB | 0.2545 | 129.14MB | 0.1166 | 236.70MB |
| SigLIP@50 | 0.3636 | 22.55GB | 0.4138 | 19.88GB | 0.5455 | 2.82GB | 0.4410 | 15.07GB |
| MIRIX | 0.5455 | 20.57MB | 0.5667 | 19.83MB | 0.6727 | 7.28MB | 0.5950 | 15.89MB |
4.3 Experimental Results on LOCOMO
We compare MIRIX against the following baselines: A-Mem: A memory system that builds Zettelkasten-style knowledge graphs from user-agent interactions, dynamically linking notes using embedding similarity and LLM reasoning. LangMem: LangChain’s long-term memory module that extracts and stores salient facts from conversations for later retrieval through retrievers like FAISS or Chroma. Zep: A commercial memory API that constructs a temporal knowledge graph (Graphiti) over user conversations and metadata, designed for fast semantic querying. Mem0: An open-source memory system that incrementally compresses and stores memory facts using LLM-based summarization, with an optional graph memory extension. Memobase: A profile-based memory module that tracks persistent user attributes and preferences to enable long-term personalization.
我们把 MIRIX 与以下基线进行比较： A-Mem： 一种从用户与智能体交互中构建卡片盒式知识图谱的记忆系统，使用嵌入相似度和 LLM 推理动态链接笔记。 LangMem： LangChain 的长期记忆模块，从对话中抽取并存储显著事实，以便之后通过 FAISS 或 Chroma 等检索器进行检索。 Zep： 一种商业记忆 API，在用户对话和元数据上构建时间知识图谱 Graphiti，面向快速语义查询设计。 Mem0： 一种开源记忆系统，使用基于 LLM 的摘要逐步压缩和存储记忆事实，并提供可选的图记忆扩展。 Memobase： 一种基于画像的记忆模块，追踪持久的用户属性和偏好，以实现长期个性化。
All baselines are re-implemented using the same backbone model (gpt-4.1-mini). We also report the results shown in Mem0 where the backbone model is gpt-4o-mini. The results are shown in Table 2. From the table, we observe the following:
所有基线都使用相同的骨干模型 gpt-4.1-mini 重新实现。 我们还报告 Mem0 中使用 gpt-4o-mini 骨干模型的结果。 结果见 表2。 从表中可以得到以下观察：
| Method | Single Hop | Multi-Hop | Open Domain | Temporal | Overall |
|---|---|---|---|---|---|
| gpt-4o-mini | |||||
| A-Mem | 39.79 | 18.85 | 54.05 | 49.91 | 48.38 |
| LangMem | 62.23 | 47.92 | 71.12 | 23.43 | 58.10 |
| OpenAI | 63.79 | 42.92 | 62.29 | 21.71 | 52.90 |
| Mem0 | 67.13 | 51.15 | 72.93 | 55.51 | 66.88 |
| Mem0g | 65.71 | 47.19 | 75.71 | 58.13 | 68.44 |
| Memobase | 63.83 | 52.08 | 71.82 | 80.37 | 70.91 |
| Zep | 74.11 | 66.04 | 67.71 | 79.76 | 75.14 |
| gpt-4.1-mini | |||||
| LangMem | 74.47 | 61.06 | 67.71 | 86.92 | 78.05 |
| RAG-500 | 37.94 | 37.69 | 48.96 | 61.83 | 51.62 |
| Zep | 79.43 | 69.16 | 73.96 | 83.33 | 79.09 |
| Mem0 | 62.41 | 57.32 | 44.79 | 66.47 | 62.47 |
| MIRIX | 85.11 | 83.70 | 65.62 | 88.39 | 85.38 |
| Full-Context | 88.53 | 77.70 | 71.88 | 92.70 | 87.52 |
Overall: MIRIX achieves the highest average J score, outperforming all baselines by a significant margin. It improves upon the strongest open-source competitor, LangMem, by over 8 points. Single-Hop and Temporal: On fact lookup and temporal-ordering tasks, MIRIX shows significantly better performances than baselines, which validates the effectiveness of our hierarchical memory storage. We note that there is a minor gap between MIRIX and the Full-Context baseline on Single-Hop questions. Upon reviewing the results, we identified a major reason for this. In some cases, the questions are ambiguous.
总体： MIRIX 取得最高的平均 J 分数，以显著优势超过所有基线。 它比最强的开源竞争方法 LangMem 高出 8 分以上。 单跳与时间： 在事实查找和时间排序任务上，MIRIX 的表现显著优于基线，验证了层次记忆存储的有效性。 我们注意到，MIRIX 与完整上下文基线在单跳问题上存在小幅差距。 在检查结果后，我们发现了造成这一现象的一个主要原因。 在某些情况下，问题存在歧义。
For example, consider the question “When is Melanie planning on going camping?” In the conversation history, Melanie stated “We’re thinking about going camping next month” in May, which would suggest June as the answer. However, she later mentioned in October, “Absolutely! It really helps me reset and recharge. I love camping trips with my fam, ’cause nature brings such peace and serenity,” referring to a more recent trip. Because MIRIX saves the consolidated event “On 19 October 2023, Melanie and her family went camping after their road trip,” it tends to prioritize the confirmed occurrence over the earlier plan, leading to discrepancies when the question expects the planned date rather than the actual event. Other similar ambiguities contribute to the slightly lower performance of MIRIX on Single-Hop questions.
例如，考虑问题*“Melanie 计划什么时候去露营？”* 在对话历史中，Melanie 在五月说过*“我们正在考虑下个月去露营”，这意味着答案应为六月*。 然而，她后来在十月提到*“当然！这真的能帮助我恢复和充电。我喜欢和家人一起露营，因为大自然让人感到如此宁静祥和”*，所指的是一次更近的旅行。 由于 MIRIX 保存了合并事件*“2023 年 10 月 19 日，Melanie 和家人在公路旅行后去露营”*，它倾向于优先考虑已经确认发生的事件，而不是先前计划；当问题期望的是计划日期而非实际事件时，这会造成答案差异。 其他类似歧义也导致 MIRIX 在单跳问题上的性能略低。
Multi-Hop: MIRIX demonstrates the largest gains in this category, outperforming all baselines by over 24 points. For example, in questions such as “Where did Caroline move from 4 years ago?”, where the correct answer is Sweden, the supporting evidence is dispersed across multiple parts of the conversation. One part may state “Caroline moved from her hometown 4 years ago”, while an earlier statement establishes that “Caroline’s hometown is Sweden”. For multi-hop questions like these, MIRIX achieves better performance because it explicitly stores the consolidated event “Caroline moved from her hometown, Sweden, 4 years ago”, removing the need to stitch together partial information at query time. In contrast, full-context methods must first retrieve the partial answer “hometown” and then figure out separately that “hometown” refers to “Swedon”. These multi-hop questions might be easy for reasoning models like OpenAI-O3, but for non-reasoning models such as gpt-4.1-mini, this additional reasoning step might fail, leading to slightly inferior performance compared with MIRIX.
多跳： MIRIX 在该类别中取得最大增益，比所有基线高出 24 分以上。 例如，对于*“Caroline 四年前从哪里搬来？”*这类问题，正确答案是瑞典，但支持证据分散在对话的多个部分。 其中一处可能写着*“Caroline 四年前离开家乡”，更早的一条陈述则说明“Caroline 的家乡是瑞典”*。 对于这类多跳问题，MIRIX 表现更好，因为它明确存储了合并事件*“Caroline 四年前从家乡瑞典搬来”*，无需在查询时拼接零散信息。 相比之下，完整上下文方法必须先检索到部分答案*“家乡”，再另外弄清楚“家乡”指的是“Swedon”*。 这些多跳问题对 OpenAI-O3 等推理模型而言可能很容易，但对于 gpt-4.1-mini 等非推理模型，这一额外推理步骤可能失败，导致性能略逊于 MIRIX。
Open-Domain: While MIRIX performs well, the margin between ours and the baselines is narrower. This category of questions usually asks the agent “what if” questions, requiring the agent to infer across longer terms. The gap between MIRIX and Full-Context method show the inherent limitation of RAG methods, which is the lack of global understanding. While MIRIX is no longer simple RAG, we still rely on RAG to retrieve important information in the memory, which might lead to the bottleneck of our agent in this category. In summary, these results demonstrate that MIRIX delivers state-of-the-art performance on LOCOMO while remaining highly efficient and modular. Its component-specific memory management and intelligent routing are particularly effective for long-range multi-hop reasoning.
开放域： 尽管 MIRIX 表现良好，但我们与基线之间的差距较小。 该类别的问题通常向智能体提出“如果……会怎样”的问题，要求智能体在更长跨度上进行推断。 MIRIX 与完整上下文方法之间的差距表明 RAG 方法存在缺乏全局理解这一固有局限。 尽管 MIRIX 已不再是简单 RAG，我们仍依赖 RAG 检索记忆中的重要信息，这可能构成智能体在该类别上的瓶颈。 总而言之，这些结果证明 MIRIX 在 LOCOMO 上达到最先进性能，同时保持高效率和模块化。 其面向特定组件的记忆管理和智能路由对长程多跳推理尤为有效。
5. Related Work
Memory-Augmented Large Language Models. A growing body of work focuses on building latent-space memory systems, as characterized in M+, where transformer architectures are modified to support memory augmentation. These memory components can reside in various latent forms, including model parameters, external memory matrices, hidden states, soft prompts, and key-value caches. While these approaches demonstrate promising results and continue to advance the frontier of latent-space memory, most require retraining the model, making them incompatible with powerful closed-source models like GPT-4 or DeepSeek-R1. Additionally, methods based on key-value caching rely on preserving past keys and values, functioning more as long-context methods rather than true memory systems that support abstraction, consolidation, and reasoning over stored experiences.
记忆增强大语言模型。 越来越多的工作聚焦于构建潜在空间记忆系统，M+ 是其代表；这类方法会修改 Transformer 架构来支持记忆增强。 这些记忆组件可以采用多种潜在形式，包括模型参数、外部记忆矩阵、隐藏状态、软提示和键值缓存。 尽管这些方法展现出良好结果，并持续推进潜在空间记忆的前沿，但大多数方法需要重新训练模型，因此无法兼容 GPT-4 或 DeepSeek-R1 等强大的闭源模型。 此外，基于键值缓存的方法依赖保留过去的键和值，其作用更像长上下文方法，而非支持对存储经历进行抽象、巩固和推理的真正记忆系统。
Memory-Augmented LLM Agents. Token-level memory remains the dominant approach in current LLM agents, where past conversational content is stored in raw text form within external databases. Notable examples include commercial systems such as Zep, Mem0, and MemGPT. These agents perform well on long-term conversational benchmarks and document-based retrieval tasks. However, they often fall short in real-world applications due to simplistic memory architectures. Most notably, the absence of modular memory components hinders effective memory routing and leads to inefficiencies in retrieval and usage.
记忆增强 LLM 智能体。 token 级记忆仍是当前 LLM 智能体的主导方法，其中过去的对话内容以原始文本形式存储在外部数据库中。 典型例子包括 Zep、Mem0 和 MemGPT 等商业系统。 这些智能体在长期对话基准和基于文档的检索任务上表现良好。 然而，由于记忆架构过于简单，它们在现实应用中往往表现不足。 尤其值得注意的是，缺少模块化记忆组件会妨碍有效的记忆路由，并导致检索和使用效率低下。
Various Memory Types. Cognitive science broadly categorizes memory into short-term (working) memory and long-term memory. In the context of LLMs, short-term memory is often mapped to the input context window, while long-term memory becomes a catch-all category for any information outside the context. To address this limitation, recent works have proposed finer-grained memory architectures. For example, a recent study emphasizes the importance of episodic memory in LLM agents. Other systems incorporate both episodic and semantic memory. Semantic memory has also been highlighted as critical for real-world reasoning and abstraction. In addition, procedural memory, responsible for learned skills and routine tasks, has been identified as another crucial component. Despite these advancements, existing methods stop at identifying individual memory types, and they are not formed into a comprehensive memory system.
多种记忆类型。 认知科学通常把记忆分为短期记忆（工作记忆）和长期记忆。 在 LLM 语境中，短期记忆通常映射到输入上下文窗口，而长期记忆则成为上下文以外任何信息的统称。 为解决这一局限，近期工作提出了粒度更细的记忆架构。 例如，一项近期研究强调了情景记忆在 LLM 智能体中的重要性。 其他系统则同时纳入情景记忆和语义记忆。 语义记忆也被认为对现实世界推理和抽象至关重要。 此外，负责已学习技能和日常任务的程序记忆也被确定为另一关键组件。 尽管取得了这些进展，现有方法仍止步于识别单独的记忆类型，尚未将它们组成综合记忆系统。
Multi-Agent Systems. Rather than relying on a monolithic agent, recent advances explore multi-agent frameworks where specialized agents coordinate to accomplish complex tasks. Early systems like AutoGPT and BabyAGI adopt an autonomous planning-execution loop while maintaining a shared memory log. More recent designs introduce role specialization: MetaGPT mimics a software development team structure, and AgentVerse assigns agents to specific roles such as planning or evaluation. Cognitive theories also support modularity in memory, particularly distinctions between episodic and semantic types, as emphasized in a recent study. MIRIX builds on these ideas by deploying eight specialized agents, each managing a distinct memory type (e.g., episodic, semantic, procedural), and coordinating to process multi-modal inputs effectively.
多智能体系统。 近期进展不再依赖单体智能体，而是探索由专门智能体相互协调来完成复杂任务的多智能体框架。 AutoGPT 和 BabyAGI 等早期系统采用自主规划与执行循环，同时维护共享记忆日志。 较新的设计引入角色专业化：MetaGPT 模仿软件开发团队结构，AgentVerse 则为智能体分配规划或评估等特定角色。 认知理论同样支持记忆的模块化，尤其强调情景类型与语义类型之间的区别。 MIRIX 在这些思想的基础上部署八个专门智能体，每个智能体管理一种不同的记忆类型，例如情景记忆、语义记忆和程序记忆，并相互协调以有效处理多模态输入。
6. Conclusion and Future Work
In this work, we introduce MIRIX, a novel memory architecture designed to enhance the long-term reasoning and personalization capabilities of LLM-based agents. Unlike existing memory systems that primarily rely on flat storage or limited memory types, MIRIX leverages a structured and compositional approach, incorporating six specialized memory components—Core, Episodic, Semantic, Procedural, Resource, and Knowledge Vault—managed by dedicated Memory Managers under the coordination of a Meta Memory Manager. To rigorously evaluate our system, we introduce a challenging multimodal benchmark based on high-resolution screenshots of real user activity, demonstrating that MIRIX achieves substantial gains in accuracy and storage efficiency compared to both retrieval-augmented generation and long-context baselines. Experiments on the LOCOMO benchmark confirm that MIRIX delivers state-of-the-art performance in long-form conversational settings. Finally, to make these capabilities accessible to a broader audience, we build and release a personal assistant application powered by MIRIX, allowing users to experience consistent, memory-enhanced interactions in everyday scenarios. We hope this work paves the way for more robust, scalable, and human-like memory systems for LLM-based agents. In the future, we aim to build more challenging real-world benchmarks to comprehensively evaluate our system and constantly improve MIRIX and the associated personal assistant application to deliver better experiences to the users.
在这项工作中，我们提出 MIRIX，一种旨在增强基于 LLM 的智能体长期推理和个性化能力的新型记忆架构。 与主要依赖扁平存储或有限记忆类型的现有记忆系统不同，MIRIX 采用结构化组合式方法，纳入核心、情景、语义、程序、资源和知识仓库六个专门记忆组件；各组件由专门记忆管理器负责，并由元记忆管理器协调。 为严格评估系统，我们引入一个基于真实用户活动高分辨率截图的高难度多模态基准，并证明 MIRIX 相比检索增强生成和长上下文基线，在准确率和存储效率方面均取得显著增益。 在 LOCOMO 基准上的实验证实，MIRIX 在长篇对话场景中达到最先进性能。 最后，为了让更广泛的用户使用这些能力，我们构建并发布了由 MIRIX 驱动的个人助理应用，使用户能够在日常场景中体验一致的记忆增强交互。 我们希望这项工作为构建更稳健、可扩展且更接近人类的基于 LLM 的智能体记忆系统铺平道路。 未来，我们计划构建更具挑战性的现实世界基准来全面评估系统，并持续改进 MIRIX 及其个人助理应用，为用户提供更好的体验。