---

Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)

LLM EvaluationDiversityAlignmentAI SafetyNeurIPS 2025 OralCCF-A华盛顿大学卡内基梅隆大学艾伦人工智能研究所斯坦福大学

Jiang L, Chai Y, Li M, et al. Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)[C]. Advances in Neural Information Processing Systems, 2025.

https://github.com/liweijiang/artificial-hivemind

https://huggingface.co/collections/liweijiang/artificial-hivemind-6826e108da3260c02a1a2ec0

---

人工蜂群思维：语言模型的开放式同质化（及其他）

Abstract

Large language models (LMs) often struggle to generate diverse, human-like creative content, raising concerns about the long-term homogenization of human thought through repeated exposure to similar outputs. Yet scalable methods for evaluating LM output diversity remain limited, especially beyond narrow tasks such as random number or name generation, or beyond repeated sampling from a single model. To address this gap, we introduce Infinity-Chat, a large-scale dataset of 26K diverse, real-world, open-ended user queries that admit a wide range of plausible answers with no single ground truth. We introduce the first comprehensive taxonomy for characterizing the full spectrum of open-ended prompts posed to LMs, comprising 6 top-level categories (e.g., creative content generation, brainstorm & ideation) that further breaks down to 17 subcategories. Using Infinity-Chat, we present a large-scale study of mode collapse in LMs, revealing a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs. Infinity-Chat also includes 31,250 human annotations, across absolute ratings and pairwise preferences, with 25 independent human annotations per example. This enables studying collective and individual-specific human preferences in response to open-ended queries. Our findings show that state-of-the-art LMs, reward models, and LM judges are less well calibrated to human ratings on model generations that elicit differing idiosyncratic annotator preferences, despite maintaining comparable overall quality. Overall, Infinity-Chat presents the first large-scale resource for systematically studying real-world open-ended queries to LMs, revealing critical insights to guide future research for mitigating long-term AI safety risks posed by the Artificial Hivemind.

大语言模型（LM）往往难以生成多样且具有人类特征的创造性内容，这引发了对人类反复接触相似输出后，其思想会长期趋于同质化的担忧。 然而，可扩展的语言模型输出多样性评估方法仍然有限，尤其是超出随机数字或姓名生成等狭窄任务，或超出对单一模型进行重复采样之后。 为弥补这一空白，我们提出 Infinity-Chat，这是一个包含 2.6 万条多样、真实世界用户查询的大规模数据集；这些开放式用户查询允许存在广泛的合理答案，而没有唯一的标准答案。 我们提出首个用于刻画用户向语言模型提出的全部开放式提示范围的综合分类体系，其中包含 6 个顶层类别（例如创造性内容生成、头脑风暴与构思），并进一步细分为 17 个子类别。 利用 Infinity-Chat，我们对语言模型中的模式坍缩开展大规模研究，揭示出语言模型开放式生成中显著的人工蜂群思维效应，其特征包括：(1) 模型内重复，即单个模型持续生成相似响应；更值得关注的是 (2) 模型间同质化，即不同模型产生惊人相似的输出。 Infinity-Chat 还包含 31,250 条人类标注，涵盖绝对评分和成对偏好，每个样例均有 25 名独立的人类标注者参与。 这使我们能够研究人类针对开放式查询响应的群体偏好与个体特有偏好。 我们的发现表明，尽管模型生成内容的整体质量相当，但当这些内容引发标注者之间不同的个体化偏好时，最先进的语言模型、奖励模型和语言模型评判器与人类评分之间的校准程度会降低。 总体而言，Infinity-Chat 是首个用于系统研究语言模型所面对的真实世界开放式查询的大规模资源，它揭示了关键见解，可指导未来研究缓解人工蜂群思维带来的长期人工智能安全风险。

1. Introduction

图1：Responses to the query “Write a metaphor about time” clustered by applying PCA to reduce sentence embeddings to two dimensions. Each of the 25 models generates 50 responses using top-p sampling (p = 0.9) and temperature = 1.0. Despite the diversity of model families and sizes, the responses form just two primary clusters: a dominant cluster on the left centered on the metaphor “time is a river,” and a smaller cluster on the right revolving around variations of “time is a weaver.”

Large language models (LMs) are the core backbone of modern AI systems but often fail to produce the diverse, human-like creativity expected in open-ended tasks that do not have a ground truth answer. This shortfall has sparked growing concerns about the long-term homogenization of human thought, as users are repeatedly exposed to similar outputs. While ensemble methods or model “swarms” have been proposed to enhance diversity, their scalable evaluations of diversity in real-world settings are still lacking. Existing benchmarks often target stylized tasks such as persona generation, keyword-driven storytelling, or random number generation, and often rely on narrowly defined tests centered on poetry or figurative language. Yet, these settings fail to capture the open-endedness and pluralism of real-world user interactions.

大语言模型（LM）是现代人工智能系统的核心骨干，但在没有标准答案的开放式任务中，它们往往无法展现人们所期待的多样、具有人类特征的创造力。 由于用户反复接触相似输出，这一缺陷日益引发人们对人类思想长期同质化的担忧。 尽管已有研究提出使用集成方法或模型“群体”来增强多样性，但对其在真实世界环境中多样性的可扩展评估仍然缺失。 现有基准通常针对角色生成、关键词驱动的故事创作或随机数字生成等风格化任务，并且往往依赖以诗歌或比喻语言为中心、定义狭窄的测试。 然而，这些设置无法捕捉真实世界用户交互中的开放性和多元性。

We introduce Infinity-Chat, a large-scale dataset of 26K real-world open-ended queries spanning diverse, naturally occurring prompts mined from WildChat. These queries admit a wide range of plausible answers with no single correct response. We further develop the first comprehensive taxonomy of open-ended LM queries, encompassing 6 top-level categories (e.g., Brainstorm & Ideation, and less explored types such as Speculative & Hypothetical Scenarios, and Skill Development) and 17 subcategories grounded in natural chatbot-user interactions.

我们提出 Infinity-Chat，这是一个包含 2.6 万条真实世界开放式查询的大规模数据集，覆盖从 WildChat 中挖掘出的多样且自然产生的提示。 这些查询允许存在广泛的合理答案，而没有唯一的正确响应。

我们还进一步构建了首个开放式语言模型查询综合分类体系，其中包含 6 个顶层类别（例如“头脑风暴与构思”，以及较少探索的“推测性与假设性场景”和“技能发展”等类型）和 17 个植根于自然聊天机器人与用户交互的子类别。

Using Infinity-Chat, we systematically study intra- and inter-model mode collapse across 70+ open and closed source LMs (25 detailed in the main paper). We uncover a pronounced Artificial Hivemind effect: (1) intra-model repetition, where a single model repeatedly generates similar outputs, and, more critically, (2) inter-model homogeneity, where different models independently converge on similar ideas with minor variations in phrasing. The latter warns that model ensembles may not yield true diversity when their constituents share overlapping alignment and training priors.

利用 Infinity-Chat，我们系统研究了 70 多个开源和闭源语言模型（其中 25 个在主文中详细介绍）的模型内和模型间模式坍缩。 我们揭示出显著的人工蜂群思维效应：(1) 模型内重复，即单个模型反复生成相似输出；更关键的是 (2) 模型间同质化，即不同模型独立收敛到相似观点，仅在措辞上略有变化。 后者警示我们，当模型集成中的各组成模型具有重叠的对齐方式和训练先验时，集成未必能产生真正的多样性。

Beyond generative behaviors, we also examine whether LMs are calibrated to assess alternative responses of comparable quality to open-ended queries. To enable this study, we collect 31,250 human annotations on distinct model responses in Infinity-Chat, encompassing both absolute quality ratings and pairwise preferences, with dense annotations from 25 independent annotators per query–response pair. Our results show that LMs, reward models, and LM-based judges are often miscalibrated with respect to human ratings on responses that elicit divergent, idiosyncratic preferences among annotators despite comparable overall quality. This exposes key limitations in current modeling pipelines, which tend to assume a single, consensus notion of quality and thus overlook or fail to reward the diverse, pluralistic preferences that arise in open-ended responses.

除生成行为外，我们还考察了语言模型在评估开放式查询的质量相当的替代响应时是否经过良好校准。 为支持这项研究，我们收集了 31,250 条针对 Infinity-Chat 中不同模型响应的人类标注，其中既包括绝对质量评分，也包括成对偏好；每个查询—响应对都有来自 25 名独立标注者的密集标注。 我们的结果表明，对于整体质量相当、但会在标注者之间引发不同个体化偏好的响应，语言模型、奖励模型和基于语言模型的评判器相对于人类评分往往存在校准偏差。 这暴露了当前建模流水线的关键局限：它们往往假设质量存在单一共识，因此忽略了开放式响应中出现的多样、多元偏好，或未能对这些偏好给予奖励。

Altogether, our work introduces a comprehensive framework for evaluating realistic open-endedness, diversity, and pluralistic alignment in LMs, both within and across LMs. By integrating real-world queries, a taxonomy of query types, and dense human annotations, Infinity-Chat provides a useful resource for diagnosing the Artificial Hivemind effect and for guiding the development of safer, more expressive, and more resourceful LMs that better empower human creativity.

总而言之，我们的工作提出了一个综合框架，用于评估语言模型内部及不同语言模型之间真实的开放性、多样性和多元对齐。 通过整合真实世界查询、查询类型分类体系和密集人类标注，Infinity-Chat 为诊断人工蜂群思维效应提供了有用资源，并可指导开发更安全、更富表现力、更足智多谋的语言模型，从而更好地赋能人类创造力。

2. Infinity-Chat: Real-World Open-Ended Queries with Diverse Responses

图2：A taxonomy of real-world open-ended queries that invite diverse model responses that are mined from in-the-wild user-chatbot interactions, categorized into 6 top-level and 17 fine-grained subcategories, along with their occurrence percentages.

Most existing LM alignment datasets prioritize response correctness over diversity, and rarely include multiple distinctive responses to the same prompt. This overlooks the inherent variability of open-ended queries, which often admit several equally valid answers. This gap motivates our first central research question: What types of open-ended queries do users actually pose to language models?

大多数现有语言模型对齐数据集将响应正确性置于多样性之上，并且很少为同一提示包含多个不同响应。 这忽略了开放式查询固有的可变性，因为此类查询通常允许多个同样有效的答案。 这一空白引出了我们的第一个核心研究问题：用户实际会向语言模型提出哪些类型的开放式查询？

Mining in-the-wild open-ended user queries. We construct Infinity-Chat, a dataset of real-world open-ended queries to language models, by filtering and refining user inputs from WildChat. From 37,426 high-quality, single-turn GPT-4 queries (English, non-toxic, 15–200 characters), GPT-4o classifies each by whether it seeks meaningful information, is a greeting or model inquiry, and allows single or multiple valid responses.

Ambiguous queries are revised for clarity. The result is an extensive collection of 26,070 open-ended and 8,817 closed-ended queries, which elicit diverse, high-quality LM responses. Full details of the query mining process are provided in the Appendix.

挖掘真实环境中的开放式用户查询。 我们通过筛选和完善 WildChat 中的用户输入，构建了 Infinity-Chat，这是一个面向语言模型的真实世界开放式查询数据集。 对于 37,426 条高质量单轮 GPT-4 查询（英文、无毒、长度为 15–200 个字符），GPT-4o 会判断每条查询是否寻求有意义的信息、是否属于问候或模型询问，以及是否允许一个或多个有效响应。 我们会修改含义模糊的查询，使其更加清晰。 最终得到一个包含 26,070 条开放式查询和 8,817 条封闭式查询的大规模集合，这些查询能够引出多样、高质量的语言模型响应。 查询挖掘过程的完整细节见附录。

图3：A word cloud visualizing new open-ended categories mined from in-the-wild queries.

Categorizing the diverse landscape of open-ended queries. To understand the types of open-ended queries users pose to LMs, we develop a taxonomy of fine-grained categories. We adopt a semi-automatic process to construct the taxonomy. Starting with $\sim$ 100 mined queries, we manually assign tentative labels, then iteratively refine and group them into a hierarchical structure. This results in 6 high-level categories and 17 fine-grained sub-categories, as shown in 图2. Next, we scale the annotation process to the full set of open-ended queries using GPT-4o. We instruct GPT-4o to label each user query with one or more of the existing open-ended categories, and to detect novel types beyond the seed categories. Full details of the taxonomy construction process are provided in the Appendix.

对开放式查询的多样版图进行分类。 为理解用户向语言模型提出的开放式查询类型，我们构建了一个细粒度类别分类体系。 我们采用半自动流程来构建该分类体系。 我们从挖掘出的 $\sim$ 100 条查询开始，手动分配暂定标签，然后反复完善标签并将其组织成层次结构。 由此得到 6 个高层类别和 17 个细粒度子类别，如图2所示。 随后，我们使用 GPT-4o 将标注流程扩展到完整的开放式查询集合。 我们要求 GPT-4o 使用一个或多个现有开放式类别标注每条用户查询，并检测种子类别之外的新类型。 分类体系构建过程的完整细节见附录。

As shown in 图2, while Creative Content Generation dominates (58.0%), we identify several underexplored yet popular types, such as Alternative Writing Genres (38.5%), Concept Explanation (23.6%), Skill Development (23.5%), Analytical & Interpretive Questions (22.6%), and Hypothetical Scenarios (22.2%). Notably, 15.2% of queries involve Brainstorming & Ideation, underscoring users’ reliance on LMs for direct ideas and inspirations, and raising concerns about the long-term risk of homogenized thinking driven by overly uniform AI outputs.

如图2所示，尽管“创造性内容生成”占主导地位（58.0%），我们仍识别出若干尚未得到充分研究但很受欢迎的类型，例如“替代写作体裁”（38.5%）、“概念解释”（23.6%）、“技能发展”（23.5%）、“分析与解释性问题”（22.6%）和“假设场景”（22.2%）。 值得注意的是，15.2% 的查询涉及“头脑风暴与构思”，这凸显了用户对语言模型直接提供想法和灵感的依赖，也引发了人们对过度统一的人工智能输出会造成思想长期同质化风险的担忧。

In addition to our pre-defined categories, we identify 314 novel ones. 图3 visualizes a word cloud of the most prominent keywords, such as “Cultural,” “Analysis,” “Ethical,” “Historical,” “Media,” and “Humor,” highlighting previously underexplored dimensions of open-ended query categories.

除预定义类别外，我们还识别出 314 个新类别。 图3通过词云展示了其中最突出的关键词，例如“文化”“分析”“伦理”“历史”“媒体”和“幽默”，从而凸显此前尚未得到充分探索的开放式查询类别维度。

With Infinity-Chat, we introduce the first comprehensive taxonomy of real-world open-ended queries that invite diverse responses. This dataset serves as a rich resource for studying LMs' capacity to generate varied appropriate outputs, and for advancing pluralistic alignment of LMs.

借助 Infinity-Chat，我们提出首个能够引出多样响应的真实世界开放式查询综合分类体系。 该数据集为研究语言模型生成多种适当输出的能力，以及推动语言模型的多元对齐提供了丰富资源。

3. Artificial Hivemind: Intra- and Inter-Model Homogeneity in LMs

图4：The heatmap shows degree of repetition in responses to open-ended queries generated by the same LMs. For each model, we generate 50 responses per query across 100 open-ended queries from Infinity-Chat100. We then compute the average pairwise sentence embedding similarities for each query's response pool and measure the percentage of queries falling into the similarity ranges indicated on the y-axis. Under the sampling parameters (top-p = 0.9, temperature = 1.0), the average pairwise similarity among responses to the same prompt typically exceeds 0.8. As a baseline, randomly paired responses from the global pool 100% fall within the 0.1–0.2 range.

Using a subset of 100 representative open-ended queries from Infinity-Chat (denoted Infinity-Chat100, human verified to be open-ended as detailed in the Appendix), we systematically examine the “Artificial Hivemind” of LMs. We focus on two aspects: (1) intra-model repetition, where the same LM fails to generate diverse outputs, and (2) inter-model homogeneity, where different models produce similar outputs. Prior studies have explored intra-model repetition at small scales or with synthetic tasks (e.g., random number/name generation). In contrast, we conduct a large-scale study on real-world open-ended questions, spanning 70+ LMs (25 detailed in the main paper, representing the strongest or largest models from major model families), providing the first systematic analysis of cross-model output convergence. Full experimental setup, complete model results, and examples are provided in the Appendix.

我们使用 Infinity-Chat 中 100 条具有代表性的开放式查询构成的子集（记为 Infinity-Chat100，附录详述了对其开放性的人工验证），系统考察语言模型的“人工蜂群思维”。 我们关注两个方面：(1) 模型内重复，即同一个语言模型无法生成多样化输出；(2) 模型间同质化，即不同模型产生相似输出。 此前研究曾以小规模或合成任务（例如随机数字或姓名生成）探索模型内重复。 相比之下，我们在真实世界开放式问题上开展大规模研究，覆盖 70 多个语言模型（其中 25 个在主文中详细介绍，代表主要模型家族中最强或规模最大的模型），首次系统分析跨模型输出收敛现象。 完整的实验设置、模型结果和示例见附录。

图5：The heatmap shows degree of repetition in responses to open-ended queries generated by the same LMs. Using min-p sampling with parameters (top-p = 1.0, min-p = 0.1, temperature = 2.0), the average pairwise similarity among responses to the same prompt typically exceeds 0.8.

Intra-model repetition. For each model, we sample 50 responses per query from Infinity-Chat100, compute the average pairwise embeddings similarity within each response pool (sentence embeddings from OpenAI’s text-embedding-3-small API are used), and report the percentage of queries falling into different similarity ranges. Despite using high-stochasticity decoding parameters (top-p $=0.9$, t $=1.0$), responses from the same model remain highly repetitive, as shown in 图4: in 79% of cases, the average similarity exceeds 0.8. Since higher temperatures tend to produce incoherent text, these results show that even under maximally aggressive sampling, LMs still fail to generate diverse responses to open-ended queries.

模型内重复。 对于每个模型，我们从 Infinity-Chat100 中为每条查询采样 50 个响应，计算每个响应池内的平均成对嵌入相似度（使用 OpenAI text-embedding-3-small API 生成句子嵌入），并报告落入不同相似度区间的查询百分比。 尽管采用高随机性解码参数（top-p $=0.9$，t $=1.0$），同一个模型的响应仍然高度重复，如图4所示：79% 的情况下，平均相似度超过 0.8。 由于更高的温度往往会产生不连贯文本，这些结果表明，即便采用尽可能激进的采样方式，语言模型仍无法为开放式查询生成多样化响应。

Recent work introduces min-p decoding, a dynamic strategy for enhancing generation diversity that adjusts the sampling threshold based on model confidence. We evaluate min-p decoding with the same setup and compute pairwise sentence embedding similarities. As shown in 图5, while min-p reduces extreme repetition (fewer pairs above 0.9), $81\mathrm{\%}$ of response pairs still exceed 0.7 similarity and $61.2\mathrm{\%}$ exceed 0.8, revealing mode collapse even under diversity-oriented decoding.

近期工作提出了 min-p 解码，这是一种根据模型置信度调整采样阈值、以增强生成多样性的动态策略。 我们在相同设置下评估 min-p 解码，并计算成对句子嵌入相似度。 如图5所示，虽然 min-p 减少了极端重复（相似度高于 0.9 的响应对更少），但仍有 $81\mathrm{\%}$ 的响应对相似度超过 0.7，$61.2\mathrm{\%}$ 超过 0.8，这表明即便采用面向多样性的解码，模式坍缩依然存在。

Despite its promise, min-p is not widely adopted, as it is better suited for creative tasks and less effective for close-ended ones. Further, addressing LM repetitiveness through decoding alone places the burden on users to choose the right strategies.

Thus, more generalizable solutions are needed at the model training level to robustly preserve output diversity without requiring user intervention. For the complete breakdown of results of all models, see the Appendix.

尽管 min-p 很有潜力，但它尚未得到广泛采用，因为它更适合创造性任务，而在封闭式任务上的效果较差。 此外，仅通过解码来解决语言模型的重复问题，会把选择正确策略的负担转嫁给用户。 因此，需要在模型训练层面提出更具普适性的解决方案，在无需用户干预的情况下稳健保留输出多样性。 所有模型结果的完整细分见附录。

图6：Average pairwise sentence embedding similarities between responses from different models reveal substantial semantic overlap across model outputs. Qualitative examples further illustrate that different models often produce strikingly similar responses to fully open-ended queries, including extended verbatim spans, underscoring the extent of repetition across models in open-ended generation tasks. All responses are generated using top-p = 0.9 and temperature = 1.0.

Inter-model homogeneity. Not only do individual models repeatedly generate similar content, but different model sizes and families also produce highly repetitive outputs, sometimes sharing substantial phrase overlaps. As shown in 图6, the average pairwise similarity between responses from different models ranges from $71\mathrm{\%}$ to $82\mathrm{\%}$, with some pairs notably higher. For example, DeepSeek-V3 and qwen-max-2025-01-25 share a similarity of $0.82$, while DeepSeek-V3 and gpt-4o-2024-11-20 reach $0.81$. Interestingly, OpenAI's GPT models and Qwen's API models tend to have higher similarities even with models outside their own families. Although the exact causes remain unclear due to proprietary training details, possible explanations include shared data pipelines across regions or contamination from synthetic data. We highlight the need for future work to rigorously investigate the sources of such cross-model repetition.

模型间同质化。 不仅单个模型会反复生成相似内容，不同规模和不同家族的模型也会产生高度重复的输出，有时还共享大量重叠短语。 如图6所示，不同模型响应之间的平均成对相似度介于 $71\mathrm{\%}$ 到 $82\mathrm{\%}$，部分模型对的相似度明显更高。 例如，DeepSeek-V3 与 qwen-max-2025-01-25 的相似度为 $0.82$，而 DeepSeek-V3 与 gpt-4o-2024-11-20 的相似度达到 $0.81$。 有趣的是，OpenAI 的 GPT 模型和 Qwen 的 API 模型即使与自身家族之外的模型比较，也往往具有更高的相似度。 由于专有训练细节不可知，确切原因仍不明确；可能的解释包括不同地区共享数据流水线，或合成数据污染。 我们强调，未来工作需要严格调查这种跨模型重复的来源。

Beyond general trends, we further analyze how repetition emerges at the instance level. As in prior work, we observe verbatim phrase overlaps within responses from the same model. Surprisingly, such overlaps are also prevalent across different models, even for fully open-ended queries with large output spaces. For example, 图6 shows that DeepSeek-V3 and gpt-4o-2024-11-20 generate overlapping phrases like “Elevate your iPhone with our,” “sleek, without compromising,” and “with bold, eye-catching” in answer to the query “Create a description with 2-3 sentences for an iPhone case collection that is a slim-fitted case with bold designs.” In some cases, models output identical responses: for “Generate a motto for a social media page focused on successes, wealth, and self-help,” both qwen-max-2025-01-25 and qwen-plus-2025-01-25 generate “Empower Your Journey: Unlock Success, Build Wealth, Transform Yourself.” These instance-level verbatim overlaps illustrate the severity of the “Artificial Hivemind” effect across models. Paraphrases of the same open-ended queries also lead to verbatim overlaps, as illustrated in the Appendix.

除总体趋势外，我们还进一步分析重复如何在样例层面出现。 与此前工作一致，我们观察到同一个模型的响应中存在逐字短语重叠。 令人惊讶的是，即使对于输出空间很大的完全开放式查询，这种重叠在不同模型之间也很普遍。 例如，图6显示，对于查询“为一个采用大胆设计的修身 iPhone 手机壳系列撰写 2–3 句话的描述”，DeepSeek-V3 和 gpt-4o-2024-11-20 生成了诸如“Elevate your iPhone with our”“sleek, without compromising”和“with bold, eye-catching”等重叠短语。 在某些情况下，模型会输出完全相同的响应：对于“为一个关注成功、财富和自我提升的社交媒体页面生成一句格言”，qwen-max-2025-01-25 和 qwen-plus-2025-01-25 都生成了“Empower Your Journey: Unlock Success, Build Wealth, Transform Yourself.” 这些样例层面的逐字重叠说明了跨模型“人工蜂群思维”效应的严重程度。 对同一开放式查询进行释义也会导致逐字重叠，相关示例见附录。

Beyond surface-level overlap, repetition also manifests semantically: models convey the same core ideas using different phrasing. As shown in 图1 (with more examples in the Appendix), for the query “Write a metaphor about time,” 50 responses from each of 25 models form just two clusters: a dominant one centered on “time is a river” and a secondary one on “time is a weaver.” This convergence of abstract concepts reveals the depth of the “Artificial Hivemind” exposed in more subtle forms.

除表层重叠外，重复也体现在语义层面：模型使用不同措辞表达相同的核心观点。 如图1所示（附录提供了更多示例），对于查询“写一个关于时间的比喻”，25 个模型各自生成的 50 个响应只形成了两个聚类：一个以“时间是一条河”为中心的主导聚类，另一个以“时间是一位织工”为中心的次要聚类。 这种抽象概念的收敛揭示了以更隐蔽形式出现的“人工蜂群思维”有多么深入。

图7：The left histogram shows the distribution of Shannon entropy across the 25 human annotations for each (Query, Response 1, Response 2) triplet, where annotators judge which response is better. Given open-ended queries, multiple high-quality responses are possible, often leading to disagreement among annotators and, on average, high entropy. With 25 annotations, label distributions can vary widely across triplets. The middle panel presents example triplets from different entropy regions, and the right bar plots show their corresponding label distributions.

图8：The average number of unique source models among the top-N most similar responses to each open-ended query across 25 LMs.

To quantify response uniformity across models, we examine the extent to which outputs from different models become indistinguishable from one another. Given 25 unique models, each generating 50 outputs to queries from Infinity-Chat, we ideally expect greater diversity across different models than from within a single model. To measure this, we identify the top $N$ most similar outputs for each query and count the unique models contributing to that set. A higher count suggests stronger cross-model similarity. As shown in 图8, the most similar responses often originate from multiple models. For instance, with $N=50$, perfectly disjoint responses would yield all 50 from a single model. Yet, we find an average of $\sim$ 8 unique models per top-50 cluster, with some queries exceeding 10, indicating distinct models frequently generate highly similar content, sometimes resulting in higher inter- than intra-model similarity. See full inter-model homogeneity results in the Appendix.

为量化跨模型响应的一致性，我们考察不同模型的输出在多大程度上会变得彼此难以区分。 对于 25 个不同模型，每个模型都针对 Infinity-Chat 查询生成 50 个输出；理想情况下，我们期望不同模型之间的多样性高于单个模型内部的多样性。 为此，我们为每条查询找出最相似的前 $N$ 个输出，并统计为该集合贡献响应的不同模型数量。 数量越高，说明跨模型相似性越强。 如图8所示，最相似的响应往往来自多个模型。 例如，当 $N=50$ 时，如果不同模型的响应完全不重叠，那么这 50 个响应应全部来自同一个模型。 然而，我们发现每个前 50 响应聚类平均包含 $\sim$ 8 个不同模型，部分查询甚至超过 10 个，这说明不同模型经常生成高度相似的内容，有时导致模型间相似度高于模型内相似度。 完整的模型间同质化结果见附录。

To summarize, our work provides further evidence of high syntactic repetition across different models. While a full causal analysis is beyond the scope of this study, our findings motivate future research to investigate whether such repetition arises from pretraining data, alignment processes, memorization, contamination, or generalization.

总而言之，我们的工作进一步证明，不同模型之间存在高度句法重复。 尽管完整的因果分析超出了本研究范围，但我们的发现促使未来研究考察这种重复究竟源自预训练数据、对齐过程、记忆、污染还是泛化。

图9：The left histogram shows the distribution of Shannon Entropy computed over the 25 human ratings for each (Query, Response) pair. Given the open-ended nature of the queries, multiple responses can be valid, leading to diverse preferences across annotators. As a result, the annotation label distributions vary significantly across examples. The middle panel presents representative (Query, Response) pairs from different entropy regions, and the right bar plots display their corresponding label distributions.

4. How Do LMs, Reward Models, and LM Judges Handle Alternative Responses to Open-Ended Queries?

Having established the generative homogeneity of LMs, in this section, we examine whether the ratings of LMs, reward models, and LM judges are calibrated to match human scores given different responses to open-ended queries from Infinity-Chat.

在确认语言模型的生成同质化之后，本节考察面对 Infinity-Chat 开放式查询的不同响应时，语言模型、奖励模型和语言模型评判器的评分是否经过校准，能够与人类分数相匹配。

4.1 Gathering Distributional Annotations Across Many Humans

Humans may have divergent preferences over similar-quality alternative responses to open-ended queries. To study how models handle such diversity, we need densely annotated data that captures distributional human preferences. Existing alignment datasets, like HelpSteer3, typically contain only sparse labels (e.g., 3 annotators per item). To address this, we collect both absolute ratings (1–5 scale for response quality) and pairwise preference ratings (strong/weak preference between two responses to the same query), each with extensive annotations. For absolute ratings, we sample 15 responses for each of 50 prompts from Infinity-Chat100 and collect 25 ratings per (Query, Response), yielding $25\times 15\times 50=\mathrm{18,750}$ labels. For pairwise preference rating, we sample 10 response pairs per prompt and gather 25 annotations per (Query, Response 1, Response 2), totaling $25\times 10\times 50=\mathrm{12,500}$ labels. This is the first large-scale human-annotated dataset with dense human ratings on alternative responses to the same open-ended queries, providing both absolute and pairwise preference labels for fine-grained analysis of human idiosyncratic and collective preference distributions. Full details of the human annotation process are provided in the Appendix.

对于开放式查询中质量相近的替代响应，人类可能持有不同偏好。 为了研究模型如何处理这种多样性，我们需要能够捕捉人类偏好分布的密集标注数据。 HelpSteer3 等现有对齐数据集通常只包含稀疏标签（例如每个项目由 3 名标注者标注）。 为此，我们同时收集绝对评分（按 1–5 分评价响应质量）和成对偏好评分（对同一查询的两个响应表达强偏好或弱偏好），两类数据都包含大量标注。 对于绝对评分，我们为 Infinity-Chat100 中的 50 条提示各采样 15 个响应，并为每个（查询，响应）收集 25 个评分，共得到 $25\times 15\times 50=\mathrm{18,750}$ 个标签。 对于成对偏好评分，我们为每条提示采样 10 个响应对，并为每个（查询，响应 1，响应 2）收集 25 条标注，共计 $25\times 10\times 50=\mathrm{12,500}$ 个标签。 这是首个针对相同开放式查询的替代响应提供密集人类评分的大规模人类标注数据集，同时包含绝对偏好和成对偏好标签，可用于细粒度分析人类的个体化偏好分布和群体偏好分布。 人类标注过程的完整细节见附录。

图7 shows the distribution of Shannon entropy over human preference annotations for (Query, Response 1, Response 2) triplets. Annotators often disagree on which response is better, resulting in entropy skewed toward the higher end. As shown by the bar charts on the right, label distributions vary widely across examples: some response pairs show near-uniform support across all options, indicating substantial annotator disagreement for alternative responses of open-ended queries. 图9 shows a similar trend in the entropy of human annotations for absolute ratings of (Query, Response) pairs.

图7展示了（查询，响应 1，响应 2）三元组的人类偏好标注所对应的香农熵分布。 标注者往往对哪个响应更好存在分歧，导致熵分布偏向较高一端。 如右侧条形图所示，不同样例的标签分布差异很大：部分响应对在所有选项上获得近乎均匀的支持，表明标注者对开放式查询的替代响应存在显著分歧。 图9显示，（查询，响应）对的绝对评分人类标注熵也呈现类似趋势。

4.2 Gathering LMs, Reward Models, and LM Judges Ratings

We aim to assess how LMs, reward models, and LM judges align with human ratings when evaluating alternative responses to open-ended queries. Specifically, we compare 3 types of model-generated ratings against human annotations. LM scores are derived from response perplexity given the query. Reward model scores are based on standardized scalar reward outputs.

LM judge ratings follow standard prompting protocols using two rubrics: an overall quality score and the HHH rubric (Helpfulness, Harmlessness, Honesty). See the Appendix for the full list of 56 state-of-the-art LMs, 6 top-ranked reward models (per RewardBench), 4 LM judges (including GPT-4o and Prometheus variants), and details on the rating procedures.

我们的目标是评估语言模型、奖励模型和语言模型评判器在评价开放式查询的替代响应时，与人类评分的对齐程度。 具体而言，我们将 3 类模型生成评分与人类标注进行比较。 语言模型分数来自给定查询时响应的困惑度。 奖励模型分数基于标准化的标量奖励输出。 语言模型评判器评分遵循标准提示协议，使用两套评分准则：整体质量分数，以及 HHH 准则（有帮助、无危害、诚实）。 56 个最先进语言模型、6 个按 RewardBench 排名靠前的奖励模型、4 个语言模型评判器（包括 GPT-4o 和 Prometheus 的不同版本）的完整列表及评分流程细节见附录。

图10：We compute Spearman's correlation coefficients between human-annotated and model-generated absolute rating scores, including LM perplexity scores, reward model scalar outputs, and LM judge scalar ratings. Correlations are calculated for the full set, as well as two groups of subsets: (a) responses with similar human-rated quality, and (b) responses with high human disagreement. The results show that correlations are notably lower in these two subsets, indicating weaker alignment between model scores and human judgments in cases of subtle or contested quality differences.

4.3 Comparing Model Ratings to Human Scores for Responses to Open-Ended Queries

We examine how model ratings align with human judgments on (1) similar-quality alternative responses to the same open-ended queries and (2) responses with high annotator disagreement.

我们考察模型评分如何与以下两类响应的人类判断对齐：(1) 同一开放式查询下质量相近的替代响应；(2) 标注者分歧很大的响应。

Motivation for comparing model scores to average human ratings. Our motivation stems from how reward models (or LM judges) are used in training to evaluate responses to open-ended queries without a single ground truth. Different annotators may prefer different answers, yet their average ratings are often similar, implying multiple responses can be equally high-quality. Current reward models, however, fail to capture this equivalence, assigning diverging scores and causing downstream models to overvalue one response despite comparable human approval. To address this, we collect 25 human ratings per example to capture diverse preferences, using the average score to reflect shared human judgment. We then test whether LMs, reward models, and LM judges correlate less reliably with responses that humans broadly consider comparably good, hence our choice to compute human correlation using average human ratings.

将模型分数与人类平均评分比较的动机。 我们的动机源于训练中使用奖励模型（或语言模型评判器）评估没有唯一标准答案的开放式查询响应的方式。 不同标注者可能偏好不同答案，但其平均评分往往相近，这意味着多个响应可能具有同样高的质量。 然而，当前奖励模型无法捕捉这种等价性，会给出差异很大的分数，导致下游模型在获得相近人类认可的响应中高估某一个响应。 为解决这一问题，我们为每个样例收集 25 个人类评分以捕捉多样偏好，并使用平均分反映人类的共同判断。 随后，我们检验对于人类普遍认为质量相当的响应，语言模型、奖励模型和语言模型评判器与人类评分的相关性是否更不可靠；这也是我们选择使用人类平均评分计算相关性的原因。

Models show weaker alignment with human ratings for alternative responses of similar quality. We hypothesize that models are less aligned with human judgments on similar-quality examples, as models are typically trained with more clearly differentiated responses.

对于质量相近的替代响应，模型与人类评分的对齐更弱。 我们假设，模型在质量相近的样例上与人类判断的对齐程度较低，因为模型通常使用差异更明确的响应进行训练。

For the absolute rating setup, we identify similar-quality (Query, Response) pairs by filtering out outliers using Tukey’s fences. This method defines outliers as points beyond $Q_1-k\cdot \mathrm{IQR}$ or $Q_3+k\cdot \mathrm{IQR}$, where $Q_1$ and $Q_3$ are the 25th and 75th percentiles and $\mathrm{IQR}=Q_3-Q_1$. We vary the constant $k$ from 0.5 (aggressive filtering) to 3.0 (conservative filtering) in increments of 0.5 to generate subsets of increasing similarity. We then compute Pearson correlations between model and human absolute ratings on the full set and these filtered subsets, as shown in 图10 (a). For the pairwise preference rating setup, we identify similar-quality (Query, Response 1, Response 2) triplets by ranking examples by how many annotators rate the two responses as similar in quality. We select the top 60%–95% most similar examples to form subsets, with the 60% subset containing examples of higher similarity. We then compute Pearson correlations between the score differences of model ratings and those of human ratings across the full set and each subset, as shown in fig:hivemind-pairwise-correlation (a).

对于绝对评分设置，我们使用 Tukey 围栏过滤离群值，以识别质量相近的（查询，响应）对。 该方法将超出 $Q_1-k\cdot \mathrm{IQR}$ 或 $Q_3+k\cdot \mathrm{IQR}$ 的点定义为离群值，其中 $Q_1$ 和 $Q_3$ 分别是第 25 和第 75 百分位数，$\mathrm{IQR}=Q_3-Q_1$。 我们以 0.5 为步长，将常数 $k$ 从 0.5（激进过滤）调整到 3.0（保守过滤），生成相似度逐渐提高的子集。 随后，我们计算完整集合和这些过滤后子集上模型绝对评分与人类绝对评分之间的 Pearson 相关性，如图10 (a) 所示。 对于成对偏好评分设置，我们依据把两个响应评为质量相近的标注者数量对样例排序，从而识别质量相近的（查询，响应 1，响应 2）三元组。 我们选取相似度最高的前 60%–95% 样例构成子集，其中 60% 子集包含相似度更高的样例。 随后，我们计算完整集合和每个子集上模型评分差值与人类评分差值之间的 Pearson 相关性，如fig:hivemind-pairwise-correlation (a) 所示。

图11：We compute Spearman's correlation coefficients between human-annotated and model-generated pairwise preference rating scores, comparing the full set to two groups of subsets: (a) responses with similar human-rated quality, and (b) responses with high human disagreement.

Our results show that correlations between human ratings and those of LMs, reward models, and LM judges drop significantly on similar-quality subsets, for both absolute and pairwise preference rating setups. Since there is no single gold-standard approach for selecting subsets of responses with similar quality given our data structure, we additionally report results using alternative subset selection methods in the Appendix. Our findings remain consistent across methods, highlighting the need for better modeling of fine-grained distinctions among equally high-quality responses to open-ended queries. For full results, including alternative grouping methods and model-level breakdowns, see the Appendix.

我们的结果表明，无论在绝对评分还是成对偏好评分设置中，人类评分与语言模型、奖励模型及语言模型评判器评分之间的相关性都会在质量相近的子集上显著下降。 由于在我们的数据结构下，不存在选择质量相近响应子集的唯一黄金标准方法，因此我们还在附录中报告了使用其他子集选择方法得到的结果。 不同方法得到的发现保持一致，凸显出对开放式查询中同样高质量响应之间的细粒度差异进行更好建模的必要性。 包括其他分组方法和模型级细分在内的完整结果见附录。

Model judgments are less aligned where annotators disagree. We hypothesize that model ratings are less calibrated to human judgments on examples with high annotator disagreement, as models are primarily trained on examples with higher human agreement.

标注者存在分歧时，模型判断的对齐程度更低。 我们假设，模型评分在标注者分歧很大的样例上与人类判断的校准程度更低，因为模型主要在具有更高人类共识的样例上训练。

For the absolute rating setup, we identify disagreement by ranking (Query, Response) pairs by Shannon entropy across 25 human labels. We then select the top $2,4,6,8,10,12,14,16\mathrm{\%}$ highest-entropy examples as disagreed subsets. 图10 (b) shows the Pearson correlations between model and human ratings across the full set and these subsets. For the pairwise preference rating setup, we quantify disagreement for each (Query, Response 1, Response 2) triplet using percentage disagreement:

对于绝对评分设置，我们按照 25 个人类标签的香农熵对（查询，响应）对进行排序，以识别分歧。 随后，我们选择熵最高的前 $2,4,6,8,10,12,14,16\mathrm{\%}$ 样例作为分歧子集。 图10 (b) 展示了完整集合和这些子集上模型评分与人类评分之间的 Pearson 相关性。 对于成对偏好评分设置，我们使用分歧百分比量化每个（查询，响应 1，响应 2）三元组的分歧：

$$P_{\text{disagree}}=1-\frac{max(C_{\text{prefer 1}},C_{\text{prefer 2}})+0.5\cdot C_{\text{tie}}}{C_{\text{total}}}.$$

where $C$ denotes the count of annotations per preference type. We retain the top 60% to 95% most disagreed examples, with the 60% subset representing stronger disagreement.

Pearson correlations between model and human score differences across the full set and each subset are shown in 图11 (b).

其中，$C$ 表示每种偏好类型的标注数量。 我们保留分歧程度最高的前 60% 到 95% 样例，其中 60% 子集代表更强的分歧。 完整集合和每个子集上模型评分差值与人类评分差值之间的 Pearson 相关性见图11 (b)。

Our results show that correlations with human ratings across models drop substantially for examples with high annotator disagreement, in both absolute and pairwise rating setups. We also report results using alternative subset selection methods in the Appendix. The findings remain consistent across methods, highlighting the need for more nuanced modeling of idiosyncratic human disagreement to better capture the broad spectrum of open-ended possibilities. For complete results, including alternative grouping methods and model-level breakdowns, see the Appendix.

我们的结果表明，在绝对评分和成对评分设置中，对于标注者分歧很大的样例，各类模型与人类评分的相关性都会大幅下降。 我们还在附录中报告了使用其他子集选择方法得到的结果。 不同方法得到的发现保持一致，这凸显出需要以更细致的方式建模人类个体化分歧，从而更好地捕捉开放式可能性的广阔范围。 包括其他分组方法和模型级细分在内的完整结果见附录。

5. Related Work

The diversity collapse problem of LMs. Diversity collapse, characterized by the inability of LMs to generate diverse outputs, presents a significant challenge to pluralistic alignment research. Prior studies identify some key factors contributing to diversity collapse, including training on synthetic data, LM alignment, and insufficient diversity in training data. Potential consequences of diversity collapse include reduced creativity, loss of minority perspectives, spread of bias, and overall decline in model utility and trustworthiness. In response, a range of mitigation strategies are proposed, such as training corpora diversification, training algorithm modifications, alternative decoding and prompting strategies.

语言模型的多样性坍缩问题。 多样性坍缩是指语言模型无法生成多样化输出，它给多元对齐研究带来了重大挑战。 此前研究识别出造成多样性坍缩的一些关键因素，包括使用合成数据训练、语言模型对齐，以及训练数据的多样性不足。 多样性坍缩的潜在后果包括创造力下降、少数群体观点丢失、偏见扩散，以及模型效用和可信度整体下降。 为此，研究者提出了一系列缓解策略，例如增加训练语料多样性、修改训练算法，以及采用其他解码和提示策略。

Measuring the creativity and divergent thinking of language models. Recent efforts to measure the creativity and divergent thinking of LMs often adapt established psychometric tests. For example, Chen and Ding utilize a divergent association task (DAT) by asking models to generate semantically distant or unrelated words. Similarly, tasks such as the Alternate Uses Test (AUT), the Torrance Tests of Creative Thinking (TTCT), Human Evaluation, and LLM-as-a-judge are employed to assess dimensions like fluency, originality, complexity, and effective semantic diversity of LM responses. Despite these demonstrated capabilities, LLM-generated creative content tends towards homogeneity, even when individual outputs achieve high creativity scores. To address these evaluation complexities, some benchmarks focus on specific creative abilities like scientific idea and code generation. Other works propose new metrics. While LMs show promise in creative tasks, comprehensively evaluating their creativity remains an active and challenging research area. We conduct a large-scale systematic study of real-world open-ended user queries and provide a comprehensive taxonomy, query dataset, and dense human annotations to improve evaluation and model training for reducing mode collapse in language models.

衡量语言模型的创造力和发散思维。 近期衡量语言模型创造力和发散思维的工作通常会改造已有的心理测量测试。 例如，Chen 和 Ding 使用发散联想任务（DAT），要求模型生成语义距离较远或彼此无关的词语。 类似地，替代用途测试（AUT）、托兰斯创造性思维测试（TTCT）、人工评估和“语言模型作为评判器”等任务被用于评估语言模型响应的流畅性、原创性、复杂性和有效语义多样性等维度。 尽管语言模型已经展现出这些能力，但即使单个输出获得很高的创造力分数，语言模型生成的创造性内容仍然趋于同质化。 为应对这些评估难点，一些基准聚焦于科学构思和代码生成等特定创造能力。 另一些工作则提出了新指标。 虽然语言模型在创造性任务上展现出潜力，但全面评估其创造力仍是一个活跃且充满挑战的研究领域。 我们对真实世界开放式用户查询开展大规模系统研究，并提供综合分类体系、查询数据集和密集人类标注，以改进评估和模型训练，减少语言模型中的模式坍缩。

Disagreement and pluralistic alignment of language models. Advances in AI value alignment research have substantially improved LM utility and safety, through enhanced training processes, and the use of both synthetic data and human datasets. Yet, a significant challenge remains: the potential for monolithic value representation. In contrast, the emerging focus on pluralistic alignment emphasizes the need for AI to serve the varied demands of a wide population. This shift is driving innovation in methods, benchmarks, and data collection strategies to support this vision of diversity. Additionally, approaches leveraging multiple LMs interacting through system messages are also being explored to boost variety. Parallel efforts are dedicated to quantifying and improving the cultural diversity exhibited by LLMs. Yet, a common characteristic of many existing pluralistic alignment work is its reliance on predefined diversity dimensions, like demographics, personality style, and cultural background. To enable models that genuinely cater to individuality without relying on stereotypes, individual-level alignment is needed.

语言模型中的分歧与多元对齐。 人工智能价值对齐研究的进展通过改进训练过程，并结合使用合成数据与人类数据集，显著提升了语言模型的效用和安全性。 然而，一个重大挑战依然存在：价值表示可能趋于单一化。 相比之下，新兴的多元对齐研究强调，人工智能需要服务于广大人群的多样化需求。 这一转变正在推动方法、基准和数据收集策略创新，以支持这种多样性愿景。 此外，研究者也在探索让多个语言模型通过系统消息交互的方法，以提升多样性。 与此同时，另一些工作致力于量化并改善大语言模型表现出的文化多样性。 然而，许多现有多元对齐工作的共同特征，是依赖人口统计、人格风格和文化背景等预定义的多样性维度。 为了让模型在不依赖刻板印象的情况下真正满足个体需求，我们需要个体层面的对齐。

6. Conclusion

Our work introduces Infinity-Chat, a large-scale resource designed to evaluate LMs' diversity in naturally occurring, open-ended settings. Through comprehensive analysis, we uncover the “Artificial Hivemind” effect, highlighting both intra-model repetition and inter-model homogeneity of current LMs. By coupling a diverse taxonomy of prompts with dense human preference annotations, Infinity-Chat provides a new foundation for diagnosing, benchmarking, and ultimately mitigating mode collapse in generative AI. We hope this resource catalyzes future efforts to foster genuine diversity in model outputs and guard against the homogenization of human expression.

我们的工作提出 Infinity-Chat，这是一个用于评估语言模型在自然产生的开放式环境中多样性的大规模资源。 通过综合分析，我们揭示了“人工蜂群思维”效应，凸显当前语言模型中同时存在的模型内重复和模型间同质化。 Infinity-Chat 将多样的提示分类体系与密集人类偏好标注相结合，为诊断、基准评测并最终缓解生成式人工智能中的模式坍缩奠定了新基础。 我们希望这一资源能够推动未来工作，在模型输出中培育真正的多样性，并防止人类表达趋于同质化。