评分理由: 论文核心研究指令调优语言模型在用户压力下对上下文证据的忠实度问题，因此与"Large Language Models"、“Instruction Tuning”、“Post-training”、“Hallucination Mitigation"和"In-context Learning"高度相关（10分）。论文评估了0.27B到32B参数的模型，包含较小模型，与"Small Language Models"有一定关联（5分）。研究涉及模型行为分析，与"Explainable AI"有一定关联（5分）。使用气候评估数据，与"AI for Science"有一定关联（5分）。其他关键词如MoE、Scaling Laws、RLHF、RAG等未在论文中涉及，得0分。

在存在争议的领域中，指令微调语言模型必须在适应用户需求的压力与忠实于上下文证据之间取得平衡。为评估这种张力，我们基于《美国国家气候评估》构建了一个受控认知冲突框架。我们对19个参数量从0.27B到32B不等的指令微调模型进行了细粒度消融实验，涵盖证据构成和不确定性线索两个维度。在中性提示下，更丰富的证据通常能提升证据一致性准确率与序数评分性能。然而在受控固定证据设置中，当面临用户压力时，证据并不能可靠地防止模型出现迎合用户倾向的立场逆转。我们报告了三种主要失效模式：首先，我们发现了负面部分证据交互现象——增加认知细微差别（特别是研究空白）会加剧Llama-3和Gemma-3等模型系列对迎合倾向的敏感性；其次，鲁棒性呈现非单调缩放规律：在某些模型系列中，部分低至中等规模的模型对对抗性用户压力尤为敏感；第三，模型在冲突下的分布集中度存在差异：部分指令微调模型在压力下能保持尖锐的峰值序数分布，而其他模型的分布则显著更分散；在规模匹配的Qwen模型对比中，经过推理蒸馏的变体（DeepSeek-R1-Qwen）始终比其指令微调版本表现出更高的分布离散度。这些发现表明，在受控固定证据设置中，若缺乏针对认知完整性的显式训练，仅提供更丰富的上下文证据并不能保证模型抵御用户压力。

In contested domains, instruction-tuned language models must balance user-alignment pressures against faithfulness to the in-context evidence. To evaluate this tension, we introduce a controlled epistemic-conflict framework grounded in the U.S. National Climate Assessment. We conduct fine-grained ablations over evidence composition and uncertainty cues across 19 instruction-tuned models spanning 0.27B to 32B parameters. Across neutral prompts, richer evidence generally improves evidence-consistent accuracy and ordinal scoring performance. Under user pressure, however, evidence does not reliably prevent user-aligned reversals in this controlled fixed-evidence setting. We report three primary failure modes. First, we identify a negative partial-evidence interaction, where adding epistemic nuance, specifically research gaps, is associated with increased susceptibility to sycophancy in families like Llama-3 and Gemma-3. Second, robustness scales non-monotonically: within some families, certain low-to-mid scale models are especially sensitive to adversarial user pressure. Third, models differ in distributional concentration under conflict: some instruction-tuned models maintain sharply peaked ordinal distributions under pressure, while others are substantially more dispersed; in scale-matched Qwen comparisons, reasoning-distilled variants (DeepSeek-R1-Qwen) exhibit consistently higher dispersion than their instruction-tuned counterparts. These findings suggest that, in a controlled fixed-evidence setting, providing richer in-context evidence alone offers no guarantee against user pressure without explicit training for epistemic integrity.

!!! info

基于美国国家气候评估（NCA4和NCA5）提取770个原子主张，利用其四层结构（主张、证据基础、研究缺口、信心描述）构建受控实验条件。采用交叉设计，结合4种证据层级和4种用户压力类型（中性、直接信念、怀疑挑战、权威诉求），共16种实验条件。选取19个指令微调模型（0.27B至32B参数），包括Qwen, Gemma, Llama, Mistral等家族及DeepSeek-R1蒸馏模型。使用证据一致性准确率和序数评分规则进行分析，并考察模型在冲突下的置信度分布（集中度与分散度）。

• 新颖的评估框架：利用NCA的分层结构，创新性地将证据层级与用户压力类型结合，能够精确控制变量。
• 细粒度的分析：不仅关注准确性，还深入分析了置信度分布和不确定性表达，使用了序数评分规则，符合科学决策的评估标准。
• 广泛的模型覆盖：测试了19个不同规模和架构的模型，包括最新的推理蒸馏模型，结论具有较好的普适性。
• 发现反直觉现象：指出了增加证据细节（如不确定性）可能带来的负面效果，对未来的模型对齐和训练具有重要启示。

• 领域限制：研究主要集中在气候科学领域，虽然具有代表性，但结论在其他科学领域或通用领域的泛化能力有待进一步验证。
• 低置信度样本不足：数据集中“低置信度”的样本数量较少（仅1.7%），限制了针对该类别的统计功效。
• 固定证据设置：实验使用固定的上下文证据，未考虑动态检索或多轮对话中证据变化的情况，可能与真实的RAG应用场景存在差距。

评分理由: 论文核心研究2D基础图像模型是否具备3D世界模型能力，并提出了一个多智能体架构来实现3D世界合成。与"World Models"高度相关（10分），因为直接研究3D世界模型能力；与"LLM Agents"和"Multi-agent Systems"高度相关（10分），因为使用了VLM-based director、generator和verifier的多智能体架构；与"Foundation Models"相关（8分），因为评估了多种基础图像模型和VLMs；与"Pre-training"有一定关联（5分），因为基础模型的能力来自预训练；其他关键词如MoE、SLMs、Scaling Laws、SFT、Alignment、RAG等均未涉及，得0分。

鉴于二维基础图像模型生成高保真输出的卓越能力，我们探究了一个根本性问题：二维基础图像模型是否内在地具备三维世界模型能力？为回答此问题，我们系统性地评估了多种先进图像生成模型和视觉语言模型在三维世界合成任务上的表现。为利用并基准测试其潜在的隐式三维能力，我们提出一种智能体框架以促进三维世界生成。我们的方法采用多智能体架构：一个基于视觉语言模型的导演智能体负责制定提示以引导图像合成，一个生成器负责合成新的图像视角，以及一个由视觉语言模型支持的两步验证器，从二维图像和三维重建空间两个维度评估并筛选生成的帧。关键的是，我们证明了我们的智能体方法能够实现连贯且稳健的三维重建，生成的输出场景可通过渲染新视角进行探索。通过对多种基础模型的大量实验，我们证明二维模型确实内蕴了对三维世界的理解。通过利用这种理解，我们的方法成功合成了广阔、真实且三维一致的世界。

Given the remarkable ability of 2D foundation image models to generate high-fidelity outputs, we investigate a fundamental question: do 2D foundation image models inherently possess 3D world model capabilities? To answer this, we systematically evaluate multiple state-of-the-art image generation models and Vision-Language Models (VLMs) on the task of 3D world synthesis. To harness and benchmark their potential implicit 3D capability, we propose an agentic framing to facilitate 3D world generation. Our approach employs a multi-agent architecture: a VLM-based director that formulates prompts to guide image synthesis, a generator that synthesizes new image views, and a VLM-backed two-step verifier that evaluates and selectively curates generated frames from both 2D image and 3D reconstruction space. Crucially, we demonstrate that our agentic approach provides coherent and robust 3D reconstruction, producing output scenes that can be explored by rendering novel views. Through extensive experiments across various foundation models, we demonstrate that 2D models do indeed encapsulate a grasp of 3D worlds. By exploiting this understanding, our method successfully synthesizes expansive, realistic, and 3D-consistent worlds.

评分理由: 论文PowerLens的核心是应用大语言模型（LLMs）的推理能力构建多智能体系统（multi-agent architecture）用于移动设备电源管理，因此与"Large Language Models"和"LLM Agents"及"Multi-agent Systems"高度相关（10分）。系统利用LLMs的常识推理进行上下文感知策略生成，涉及多步推理过程，与"Chain of Thought"和"System 2 Thinking"有一定关联（5分）。论文未涉及其他关键词如MoE、模型压缩、训练技术、科学AI应用等具体内容，这些评分为0分。

电池续航能力仍是移动设备面临的关键挑战，而现有的电源管理机制依赖静态规则或粗粒度启发式策略，忽略了用户活动与个人偏好。本文提出PowerLens系统，该系统利用大语言模型（LLMs）的推理能力，在Android设备上实现安全且个性化的移动电源管理。其核心思想在于：大语言模型的常识推理能够弥合用户活动与系统参数之间的语义鸿沟，从而通过隐式反馈生成零样本、上下文感知的策略，以适应个体偏好。PowerLens采用多智能体架构，通过界面语义识别用户上下文，并针对18项设备参数生成全局性电源策略。系统通过基于PDL的约束框架在执行前验证所有操作，同时采用双层记忆系统，通过基于置信度蒸馏的方法从用户隐式覆盖行为中学习个体偏好，无需显式配置，并在3至5天内实现偏好收敛。在已获取root权限的Android设备上进行的大量实验表明，PowerLens相比原生Android系统实现了81.7%的操作准确率和38.8%的节能效果，其性能优于基于规则和基于大语言模型的基线方法，同时具备高用户满意度、快速偏好收敛和强安全性保障，且系统自身仅消耗每日电池容量的0.5%。

Battery life remains a critical challenge for mobile devices, yet existing power management mechanisms rely on static rules or coarse-grained heuristics that ignore user activities and personal preferences. We present PowerLens, a system that tames the reasoning power of Large Language Models (LLMs) for safe and personalized mobile power management on Android devices. The key idea is that LLMs’ commonsense reasoning can bridge the semantic gap between user activities and system parameters, enabling zero-shot, context-aware policy generation that adapts to individual preferences through implicit feedback. PowerLens employs a multi-agent architecture that recognizes user context from UI semantics and generates holistic power policies across 18 device parameters. A PDL-based constraint framework verifies every action before execution, while a two-tier memory system learns individualized preferences from implicit user overrides through confidence-based distillation, requiring no explicit configuration and converging within 3–5 days. Extensive experiments on a rooted Android device show that PowerLens achieves 81.7% action accuracy and 38.8% energy saving over stock Android, outperforming rule-based and LLM-based baselines, with high user satisfaction, fast preference convergence, and strong safety guarantees, with the system itself consuming only 0.5% of daily battery capacity.

!!! info

论文采用多智能体架构，将问题分解为上下文识别和策略生成。利用LLM的常识推理能力弥合用户活动与系统参数间的语义鸿沟。通过状态差异检测机制捕捉用户的手动覆盖行为作为隐式反馈，并使用置信度蒸馏将其转化为稳定的偏好规则。为了确保安全，设计了基于命题动态逻辑（PDL）的验证器，对每个生成的动作进行约束检查。实验在已root的Android设备上进行，构建了涵盖7种应用类别的PowerLensBench基准进行评估。

• 创新性强：首次将LLM智能体引入移动电源管理领域，突破了传统静态规则和粗粒度启发式的局限。
• 个性化与自动化：通过隐式反馈学习用户习惯，实现了无需显式配置的个性化优化，提升了用户体验。
• 安全性高：引入PDL形式化验证框架，有效解决了LLM可能产生的幻觉和无效操作问题，确保系统稳定性。
• 实效显著：在真实设备上展示了显著的节能效果和较高的操作准确率，且系统开销极低。

• 依赖Root权限：系统实现需要Android设备获取Root权限，限制了其在普通消费级设备上的直接部署。
• 隐私考量：虽然论文未详细展开，但分析UI语义和用户行为可能涉及用户隐私数据的处理。
• 泛化性：虽然实验涵盖了多种场景，但在极度边缘或新出现的应用类型上的表现可能仍需验证。
• LLM开销：尽管系统消耗低，但调用LLM本身可能带来延迟或网络依赖（如果使用云端模型），推理延迟可能影响实时性。

评分理由: 论文专注于大型多模态模型（LMMs）在建筑结构病理学推理中的应用，属于大模型在工程科学领域的创新应用。核心相关关键词包括：1）“Large Language Models” OR “LLMs” OR “Foundation Models”（10分）- 论文明确研究LMMs，属于基础模型范畴；2）“LLM Agents” OR “Autonomous Agents” OR “Agentic Workflow”（10分）- 论文旨在推进自主AI智能体在土木工程中的应用；3）“AI for Science” OR “Bioinformatics” OR “Cheminformatics”（10分）- 属于AI for Science在工程领域的应用；4）“Chain of Thought” OR “CoT Reasoning” OR “Multi-step Reasoning"和"System 2 Thinking” OR “Slow Thinking” OR “In-depth Reasoning”（各5分）- 论文评估LMMs的认知维度包括空间定位和生成几何分割，涉及多步推理和深度推理能力。其他关键词如MoE、量化、对齐等未在论文中涉及，故得0分。

自动化建筑立面检测是城市韧性与智慧城市运维的关键组成部分。传统上，该领域依赖于专门的判别式模型（如YOLO、Mask R-CNN），这些模型擅长像素级定位，但局限于被动感知，且因缺乏对结构拓扑的视觉理解而泛化能力不足。大型多模态模型（Large Multimodal Models, LMMs）有望推动向主动推理的范式转变，然而其在此类高风险工程领域的应用尚缺乏严格的评估标准。为弥补这一空白，我们引入了一种人机协同的半自动化标注框架，利用专家提议验证机制，将12个分散的数据集统一为标准化、层次化的本体。在此基础上，我们提出了首个超越基础语义识别的多维度基准测试集——\textit{DefectBench}，旨在系统检验LMMs的综合能力。\textit{DefectBench} 在三个递进的认知维度——语义感知、空间定位与生成式几何分割——上评估了18个前沿的LMMs。大量实验表明，尽管当前LMMs展现出卓越的拓扑意识与语义理解能力（能有效诊断“是什么”和“如何发生”），但在度量定位精度（“在何处”）方面存在显著不足。然而，关键的是，我们验证了零样本生成式分割的可行性，证明通用基础模型无需领域特定训练即可媲美专门的监督网络。本研究不仅提供了严格的基准测试标准与高质量开源数据库，也为自主人工智能代理在土木工程领域的进步确立了新的基线。

Automated building facade inspection is a critical component of urban resilience and smart city maintenance. Traditionally, this field has relied on specialized discriminative models (e.g., YOLO, Mask R-CNN) that excel at pixel-level localization but are constrained to passive perception and worse generization without the visual understandng to interpret structural topology. Large Multimodal Models (LMMs) promise a paradigm shift toward active reasoning, yet their application in such high-stakes engineering domains lacks rigorous evaluation standards. To bridge this gap, we introduce a human-in-the-loop semi-automated annotation framework, leveraging expert-proposal verification to unify 12 fragmented datasets into a standardized, hierarchical ontology. Building on this foundation, we present \textit{DefectBench}, the first multi-dimensional benchmark designed to interrogate LMMs beyond basic semantic recognition. \textit{DefectBench} evaluates 18 state-of-the-art (SOTA) LMMs across three escalating cognitive dimensions: Semantic Perception, Spatial Localization, and Generative Geometry Segmentation. Extensive experiments reveal that while current LMMs demonstrate exceptional topological awareness and semantic understanding (effectively diagnosing “what” and “how”), they exhibit significant deficiencies in metric localization precision (“where”). Crucially, however, we validate the viability of zero-shot generative segmentation, showing that general-purpose foundation models can rival specialized supervised networks without domain-specific training. This work provides both a rigorous benchmarking standard and a high-quality open-source database, establishing a new baseline for the advancement of autonomous AI agents in civil engineering.

评分理由: 论文核心研究LLM在低资源语言翻译中的应用，直接使用LLMs（10分）。提出SAGE框架，核心是使用强化学习（RL）代理（Agent）来筛选数据，与’LLM Agents’高度相关（10分）。框架采用LoRA进行高效微调，是参数高效微调的核心技术（10分）。研究强调’right data over big data’，关注数据质量对模型性能的影响，与’Scaling Laws AND Data Quality’有一定关联（8分）。论文未涉及其他关键词如MoE、SLMs、预训练、对齐、RAG、推理加速等具体技术，因此这些关键词评分为0。

构建包容性万维网的愿景正遭受严峻语言鸿沟的阻碍，这在东南亚资源匮乏地区尤为突出。尽管大语言模型（LLMs）为翻译提供了潜在解决方案，但其在数据稀缺环境中的部署面临双重挑战：高质量、文化适配数据的匮乏，以及在海量嘈杂网络语料上进行训练所产生的高昂能源成本。为化解数字包容性与环境可持续性之间的矛盾，我们提出了可持续智能体引导专家调优框架（Sustainable Agent-Guided Expert-tuning, SAGE）。该框架开创了一种能源感知新范式，强调“优质数据”优先于“海量数据”。SAGE摒弃对未过滤数据集进行高碳排放训练的传统方式，转而采用通过群体相对策略优化（Group Relative Policy Optimization, GRPO）训练的强化学习（RL）智能体，自主构建精炼训练集。该智能体利用从少量专家构建的社区对话集中提取的语义奖励信号，有效过滤噪声及文化失配内容。随后，我们采用低秩自适应技术（Low-Rank Adaptation, LoRA）在此精选数据上高效微调开源大语言模型。我们将SAGE应用于英语与七种东南亚低资源语言（low-resource languages, LRLs）的翻译任务。该方法在BLEU-4和COMET-22指标上取得了最先进的性能表现，能有效捕捉本地语言细微特征。至关重要的是，SAGE在超越基于完整数据集训练的基线模型的同时，将数据使用量降低了97.1%，训练能耗减少了95.2%。通过以最小环境代价实现高性能模型，SAGE为弥合全球南方数字鸿沟提供了一条可扩展且负责任的技术路径。

The vision of an inclusive World Wide Web is impeded by a severe linguistic divide, particularly for communities in low-resource regions of Southeast Asia. While large language models (LLMs) offer a potential solution for translation, their deployment in data-poor contexts faces a dual challenge: the scarcity of high-quality, culturally relevant data and the prohibitive energy costs of training on massive, noisy web corpora. To resolve the tension between digital inclusion and environmental sustainability, we introduce Sustainable Agent-Guided Expert-tuning (SAGE). This framework pioneers an energy-aware paradigm that prioritizes the “right data” over “big data”. Instead of carbon-intensive training on unfiltered datasets, SAGE employs a reinforcement learning (RL) agent, optimized via Group Relative Policy Optimization (GRPO), to autonomously curate a compact training set. The agent utilizes a semantic reward signal derived from a small, expert-constructed set of community dialogues to filter out noise and cultural misalignment. We then efficiently fine-tune open-source LLMs on this curated data using Low-Rank Adaptation (LoRA). We applied SAGE to translation tasks between English and seven low-resource languages (LRLs) in Southeast Asia. Our approach establishes new state-of-the-art performance on BLEU-4 and COMET-22 metrics, effectively capturing local linguistic nuances. Crucially, SAGE surpasses baselines trained on full datasets while reducing data usage by 97.1% and training energy consumption by 95.2%. By delivering high-performance models with a minimal environmental footprint, SAGE offers a scalable and responsible pathway to bridge the digital divide in the Global South.

评分理由: 论文提出All-Mem框架，用于构建具有终身记忆的交互式智能体（LLM Agents），核心是通过动态拓扑演化管理记忆库，以改进检索和问答性能。这与"LLM Agents"高度相关（10分），因为论文直接研究智能体系统；与"Retrieval-Augmented Generation"高度相关（10分），因为框架涉及记忆检索以增强生成；与"Large Language Models"高度相关（10分），因为摘要提到使用LLM作为诊断器；与"Context Window Extension"有一定关联（5分），因为框架旨在处理长期记忆，间接涉及上下文管理；其他关键词如MoE、SFT、RLHF等未在论文中涉及，给0分。

终身交互智能体需在数月或数年内持续协助用户，这要求其在固定的上下文与延迟预算下，持续写入长期记忆并为每个新查询检索正确的证据。现有记忆系统常随历史增长而性能下降，检索出的上下文易出现冗余、过时或噪声问题。我们提出All-Mem——一种在线/离线终身记忆框架，通过显式的非破坏性整合来维护拓扑结构化的记忆库，避免了基于摘要的压缩方法中典型的信息不可逆损失。在线运行时，该系统将检索锚定在有限的可见表层，以保持粗粒度搜索成本可控。在定期离线阶段，大型语言模型诊断器会提出带有置信度评分的拓扑编辑建议，通过SPLIT、MERGE和UPDATE三种算子进行门控执行，同时保留不可变的证据以确保可追溯性。在查询时，类型化链接支持在必要时从活跃锚点向归档证据进行跳数受限、预算可控的扩展。在LOCOMO和LONGMEMEVAL数据集上的实验表明，本方法在检索与问答任务上优于代表性基线模型。

Lifelong interactive agents are expected to assist users over months or years, which requires continually writing long term memories while retrieving the right evidence for each new query under fixed context and latency budgets. Existing memory systems often degrade as histories grow, yielding redundant, outdated, or noisy retrieved contexts. We present All-Mem, an online/offline lifelong memory framework that maintains a topology structured memory bank via explicit, non destructive consolidation, avoiding the irreversible information loss typical of summarization based compression. In online operation, it anchors retrieval on a bounded visible surface to keep coarse search cost bounded. Periodically offline, an LLM diagnoser proposes confidence scored topology edits executed with gating using three operators: SPLIT, MERGE, and UPDATE, while preserving immutable evidence for traceability. At query time, typed links enable hop bounded, budgeted expansion from active anchors to archived evidence when needed. Experiments on LOCOMO and LONGMEMEVAL show improved retrieval and QA over representative baselines.

评分理由: 论文核心研究LLMs在文本分类任务中应用推理策略（如CoT、ToT等）的有效性和效率，与"Large Language Models"和"Chain of Thought"高度相关（10分），与"System 2 Thinking"相关（8分），因为涉及深思熟虑的推理过程。论文提到在小型模型上的测试，与"Small Language Models"有一定关联（5分）。其他关键词如MoE、Scaling Laws、Alignment等未在摘要中涉及，评分为0分。

从大型语言模型中引出显式、逐步的推理轨迹已成为增强模型能力的主导范式。尽管此类推理策略最初是为需要显式多步推理的问题设计的，但它们已越来越多地应用于广泛的自然语言处理任务中。这种扩展隐含地假设审慎推理能一致地有益于异构任务。然而，此类推理机制是否真正有益于分类任务在很大程度上仍未得到充分探索，特别是考虑到其巨大的令牌和时间成本。为填补这一空白，我们引入了TextReasoningBench，这是一个旨在系统评估大型语言模型在文本分类任务中推理策略有效性与效率的基准。我们在五个文本分类数据集上，针对十种大型语言模型，比较了七种推理策略，即IO、思维链、自洽思维链、思维树、思维图、思维链束以及长思维链。除了准确率和宏观F1分数等传统指标外，我们引入了两个成本感知评估指标，用于量化每个推理令牌带来的性能增益，以及相对于令牌成本增长的性能提升效率。实验结果揭示了三个值得注意的发现：（1）推理并非普遍提升分类性能：虽然中等复杂度的策略如思维链和自洽思维链能带来一致但有限的增益（通常在大模型上为+1%至+3%），但更复杂的方法（例如思维树和思维图）往往无法超越更简单的基线，甚至可能降低性能，尤其是在小模型上；（2）推理通常是低效的：许多推理策略将令牌消耗增加了10倍至100倍（例如自洽思维链和思维树），却仅带来微小的性能提升。

Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.

!!! info

研究采用了基准测试的方法。选取了10个不同规模的LLM（包括4个小模型和5个大模型），在5个文本分类数据集（涵盖客观任务如AGNews和主观任务如SST-2）上进行实验。对比了7种推理策略：IO, CoT, SC-CoT, ToT, GoT, BoC, 和 long-CoT。所有实验在零样本设置下进行，重复五次以处理随机性。评估指标包括准确率、Macro-F1，以及新提出的绝对效率（F1 per token）和边际效率。

• 挑战了现有的“推理万能论”假设，具有批判性思维。
• 引入了成本感知指标，不仅关注性能，还关注计算效率，符合实际应用需求。
• 实验设计全面，涵盖了多种模型规模、推理策略和不同类型的分类任务（客观与主观）。

• 虽然涵盖了10个模型，但具体的模型架构细节和更多最新模型的测试可能仍有扩展空间。
• 主要关注文本分类任务，结论是否适用于其他类型的NLP任务（如生成任务）尚需进一步验证。
• 零样本设置可能无法完全代表经过特定指令微调后的模型表现。

评分理由: 论文核心研究多模态大语言模型（MLLMs）的监督数据选择问题，与"Large Language Models"高度相关（10分），因为MLLMs是LLMs的扩展。与"Post-training"高度相关（10分），因为研究重点是训练后性能增益的预测。与"Scaling Laws” AND “Data Quality"有一定关联（5分），因为涉及数据质量对性能的影响。与"Pre-training"有一定关联（5分），因为研究监督数据选择，属于预训练或适应阶段。其他关键词如MoE、SLMs、RLHF、RAG等与论文内容无直接关系，得0分。

为多模态大语言模型（MLLMs）选择监督数据的传统思路是优先选择与目标基准看似相似的数据集，例如文本密集型或视觉中心型任务。然而，这种直观的相似性能否可靠地预测下游性能提升尚不明确。在本研究中，我们首次尝试回答一个实际问题：能否在训练开始前就预估某个训练数据集对目标基准的影响？为探究此问题，我们对涵盖7种不同任务的14个视觉-语言数据集间的迁移效应进行了深入分析。结果表明，直观的任务相似性并非迁移能力的可靠预测指标，泛化性能更多地取决于具体数据集而非其宽泛的任务类别。基于此发现，我们提出了DATAPROPHET——一种简单有效的免训练度量方法，它融合了多模态困惑度、相似性和数据多样性。实验表明，DATAPROPHET生成的监督数据排序与实际训练后性能增益的排序高度相关，肯德尔tau系数达到86.0%。此外，DATAPROPHET能实现更优的监督数据选择：相比均匀选择提升最高达6.9%，优于当前最先进的基于训练的基线方法1.4%，甚至比基于实验性能的预言机选择高出0.2%。我们的代码与数据将公开释放。

Conventional wisdom for selecting supervision data for multimodal large language models (MLLMs) is to prioritize datasets that appear similar to the target benchmark, such as text-intensive or vision-centric tasks. However, it remains unclear whether such intuitive similarity reliably predicts downstream performance gains. In this work, we take a first step toward answering a practical question: can we estimate the influence of a training dataset on a target benchmark before any training is performed? To investigate this question, we conduct an in-depth analysis of transfer across 14 vision-language datasets spanning 7 diverse tasks. Our results show that intuitive task similarity is an unreliable predictor of transferability, and that generalization depends more on the specific dataset than on its broad task category. Motivated by this finding, we propose DATAPROPHET, a simple and effective training-free metric that combines multimodal perplexity, similarity, and data diversity. Experiments show that DATAPROPHET produces supervision-data rankings that strongly correlate with rankings based on actual post-training performance gains, achieving a Kendall’s tau of 86.0%. Moreover, DATAPROPHET enables better supervision-data selection, yielding up to 6.9% improvement over uniform selection, 1.4% over a state-of-the-art training-based baseline, and 0.2% above oracle selection based on experimental performance. Our code and data will be released.

评分理由: 论文核心研究指令调优语言模型在用户压力下对上下文证据的忠实度问题，因此与"Large Language Models”、“Instruction Tuning”、“Post-training”、“Hallucination Mitigation"和"In-context Learning"高度相关（10分）。论文评估了0.27B到32B参数的模型，包含较小模型，与"Small Language Models"有一定关联（5分）。研究涉及模型行为分析，与"Explainable AI"有一定关联（5分）。使用气候评估数据，与"AI for Science"有一定关联（5分）。其他关键词如MoE、Scaling Laws、RLHF、RAG等未在论文中涉及，得0分。

在存在争议的领域中，指令微调语言模型必须在适应用户需求的压力与忠实于上下文证据之间取得平衡。为评估这种张力，我们基于《美国国家气候评估》构建了一个受控认知冲突框架。我们对19个参数量从0.27B到32B不等的指令微调模型进行了细粒度消融实验，涵盖证据构成和不确定性线索两个维度。在中性提示下，更丰富的证据通常能提升证据一致性准确率与序数评分性能。然而在受控固定证据设置中，当面临用户压力时，证据并不能可靠地防止模型出现迎合用户倾向的立场逆转。我们报告了三种主要失效模式：首先，我们发现了负面部分证据交互现象——增加认知细微差别（特别是研究空白）会加剧Llama-3和Gemma-3等模型系列对迎合倾向的敏感性；其次，鲁棒性呈现非单调缩放规律：在某些模型系列中，部分低至中等规模的模型对对抗性用户压力尤为敏感；第三，模型在冲突下的分布集中度存在差异：部分指令微调模型在压力下能保持尖锐的峰值序数分布，而其他模型的分布则显著更分散；在规模匹配的Qwen模型对比中，经过推理蒸馏的变体（DeepSeek-R1-Qwen）始终比其指令微调版本表现出更高的分布离散度。这些发现表明，在受控固定证据设置中，若缺乏针对认知完整性的显式训练，仅提供更丰富的上下文证据并不能保证模型抵御用户压力。

In contested domains, instruction-tuned language models must balance user-alignment pressures against faithfulness to the in-context evidence. To evaluate this tension, we introduce a controlled epistemic-conflict framework grounded in the U.S. National Climate Assessment. We conduct fine-grained ablations over evidence composition and uncertainty cues across 19 instruction-tuned models spanning 0.27B to 32B parameters. Across neutral prompts, richer evidence generally improves evidence-consistent accuracy and ordinal scoring performance. Under user pressure, however, evidence does not reliably prevent user-aligned reversals in this controlled fixed-evidence setting. We report three primary failure modes. First, we identify a negative partial-evidence interaction, where adding epistemic nuance, specifically research gaps, is associated with increased susceptibility to sycophancy in families like Llama-3 and Gemma-3. Second, robustness scales non-monotonically: within some families, certain low-to-mid scale models are especially sensitive to adversarial user pressure. Third, models differ in distributional concentration under conflict: some instruction-tuned models maintain sharply peaked ordinal distributions under pressure, while others are substantially more dispersed; in scale-matched Qwen comparisons, reasoning-distilled variants (DeepSeek-R1-Qwen) exhibit consistently higher dispersion than their instruction-tuned counterparts. These findings suggest that, in a controlled fixed-evidence setting, providing richer in-context evidence alone offers no guarantee against user pressure without explicit training for epistemic integrity.

评分理由: 论文核心研究2D基础图像模型是否具备3D世界模型能力，并提出了一个多智能体架构来实现3D世界合成。与"World Models"高度相关（10分），因为直接研究3D世界模型能力；与"LLM Agents"和"Multi-agent Systems"高度相关（10分），因为使用了VLM-based director、generator和verifier的多智能体架构；与"Foundation Models"相关（8分），因为评估了多种基础图像模型和VLMs；与"Pre-training"有一定关联（5分），因为基础模型的能力来自预训练；其他关键词如MoE、SLMs、Scaling Laws、SFT、Alignment、RAG等均未涉及，得0分。

鉴于二维基础图像模型生成高保真输出的卓越能力，我们探究了一个根本性问题：二维基础图像模型是否内在地具备三维世界模型能力？为回答此问题，我们系统性地评估了多种先进图像生成模型和视觉语言模型在三维世界合成任务上的表现。为利用并基准测试其潜在的隐式三维能力，我们提出一种智能体框架以促进三维世界生成。我们的方法采用多智能体架构：一个基于视觉语言模型的导演智能体负责制定提示以引导图像合成，一个生成器负责合成新的图像视角，以及一个由视觉语言模型支持的两步验证器，从二维图像和三维重建空间两个维度评估并筛选生成的帧。关键的是，我们证明了我们的智能体方法能够实现连贯且稳健的三维重建，生成的输出场景可通过渲染新视角进行探索。通过对多种基础模型的大量实验，我们证明二维模型确实内蕴了对三维世界的理解。通过利用这种理解，我们的方法成功合成了广阔、真实且三维一致的世界。

Given the remarkable ability of 2D foundation image models to generate high-fidelity outputs, we investigate a fundamental question: do 2D foundation image models inherently possess 3D world model capabilities? To answer this, we systematically evaluate multiple state-of-the-art image generation models and Vision-Language Models (VLMs) on the task of 3D world synthesis. To harness and benchmark their potential implicit 3D capability, we propose an agentic framing to facilitate 3D world generation. Our approach employs a multi-agent architecture: a VLM-based director that formulates prompts to guide image synthesis, a generator that synthesizes new image views, and a VLM-backed two-step verifier that evaluates and selectively curates generated frames from both 2D image and 3D reconstruction space. Crucially, we demonstrate that our agentic approach provides coherent and robust 3D reconstruction, producing output scenes that can be explored by rendering novel views. Through extensive experiments across various foundation models, we demonstrate that 2D models do indeed encapsulate a grasp of 3D worlds. By exploiting this understanding, our method successfully synthesizes expansive, realistic, and 3D-consistent worlds.

评分理由: 论文PowerLens的核心是应用大语言模型（LLMs）的推理能力构建多智能体系统（multi-agent architecture）用于移动设备电源管理，因此与"Large Language Models"和"LLM Agents"及"Multi-agent Systems"高度相关（10分）。系统利用LLMs的常识推理进行上下文感知策略生成，涉及多步推理过程，与"Chain of Thought"和"System 2 Thinking"有一定关联（5分）。论文未涉及其他关键词如MoE、模型压缩、训练技术、科学AI应用等具体内容，这些评分为0分。

电池续航能力仍是移动设备面临的关键挑战，而现有的电源管理机制依赖静态规则或粗粒度启发式策略，忽略了用户活动与个人偏好。本文提出PowerLens系统，该系统利用大语言模型（LLMs）的推理能力，在Android设备上实现安全且个性化的移动电源管理。其核心思想在于：大语言模型的常识推理能够弥合用户活动与系统参数之间的语义鸿沟，从而通过隐式反馈生成零样本、上下文感知的策略，以适应个体偏好。PowerLens采用多智能体架构，通过界面语义识别用户上下文，并针对18项设备参数生成全局性电源策略。系统通过基于PDL的约束框架在执行前验证所有操作，同时采用双层记忆系统，通过基于置信度蒸馏的方法从用户隐式覆盖行为中学习个体偏好，无需显式配置，并在3至5天内实现偏好收敛。在已获取root权限的Android设备上进行的大量实验表明，PowerLens相比原生Android系统实现了81.7%的操作准确率和38.8%的节能效果，其性能优于基于规则和基于大语言模型的基线方法，同时具备高用户满意度、快速偏好收敛和强安全性保障，且系统自身仅消耗每日电池容量的0.5%。

Battery life remains a critical challenge for mobile devices, yet existing power management mechanisms rely on static rules or coarse-grained heuristics that ignore user activities and personal preferences. We present PowerLens, a system that tames the reasoning power of Large Language Models (LLMs) for safe and personalized mobile power management on Android devices. The key idea is that LLMs’ commonsense reasoning can bridge the semantic gap between user activities and system parameters, enabling zero-shot, context-aware policy generation that adapts to individual preferences through implicit feedback. PowerLens employs a multi-agent architecture that recognizes user context from UI semantics and generates holistic power policies across 18 device parameters. A PDL-based constraint framework verifies every action before execution, while a two-tier memory system learns individualized preferences from implicit user overrides through confidence-based distillation, requiring no explicit configuration and converging within 3–5 days. Extensive experiments on a rooted Android device show that PowerLens achieves 81.7% action accuracy and 38.8% energy saving over stock Android, outperforming rule-based and LLM-based baselines, with high user satisfaction, fast preference convergence, and strong safety guarantees, with the system itself consuming only 0.5% of daily battery capacity.

评分理由: 论文专注于大型多模态模型（LMMs）在建筑结构病理学推理中的应用，属于大模型在工程科学领域的创新应用。核心相关关键词包括：1）“Large Language Models” OR “LLMs” OR “Foundation Models”（10分）- 论文明确研究LMMs，属于基础模型范畴；2）“LLM Agents” OR “Autonomous Agents” OR “Agentic Workflow”（10分）- 论文旨在推进自主AI智能体在土木工程中的应用；3）“AI for Science” OR “Bioinformatics” OR “Cheminformatics”（10分）- 属于AI for Science在工程领域的应用；4）“Chain of Thought” OR “CoT Reasoning” OR “Multi-step Reasoning"和"System 2 Thinking” OR “Slow Thinking” OR “In-depth Reasoning”（各5分）- 论文评估LMMs的认知维度包括空间定位和生成几何分割，涉及多步推理和深度推理能力。其他关键词如MoE、量化、对齐等未在论文中涉及，故得0分。

自动化建筑立面检测是城市韧性与智慧城市运维的关键组成部分。传统上，该领域依赖于专门的判别式模型（如YOLO、Mask R-CNN），这些模型擅长像素级定位，但局限于被动感知，且因缺乏对结构拓扑的视觉理解而泛化能力不足。大型多模态模型（Large Multimodal Models, LMMs）有望推动向主动推理的范式转变，然而其在此类高风险工程领域的应用尚缺乏严格的评估标准。为弥补这一空白，我们引入了一种人机协同的半自动化标注框架，利用专家提议验证机制，将12个分散的数据集统一为标准化、层次化的本体。在此基础上，我们提出了首个超越基础语义识别的多维度基准测试集——\textit{DefectBench}，旨在系统检验LMMs的综合能力。\textit{DefectBench} 在三个递进的认知维度——语义感知、空间定位与生成式几何分割——上评估了18个前沿的LMMs。大量实验表明，尽管当前LMMs展现出卓越的拓扑意识与语义理解能力（能有效诊断“是什么”和“如何发生”），但在度量定位精度（“在何处”）方面存在显著不足。然而，关键的是，我们验证了零样本生成式分割的可行性，证明通用基础模型无需领域特定训练即可媲美专门的监督网络。本研究不仅提供了严格的基准测试标准与高质量开源数据库，也为自主人工智能代理在土木工程领域的进步确立了新的基线。

Automated building facade inspection is a critical component of urban resilience and smart city maintenance. Traditionally, this field has relied on specialized discriminative models (e.g., YOLO, Mask R-CNN) that excel at pixel-level localization but are constrained to passive perception and worse generization without the visual understandng to interpret structural topology. Large Multimodal Models (LMMs) promise a paradigm shift toward active reasoning, yet their application in such high-stakes engineering domains lacks rigorous evaluation standards. To bridge this gap, we introduce a human-in-the-loop semi-automated annotation framework, leveraging expert-proposal verification to unify 12 fragmented datasets into a standardized, hierarchical ontology. Building on this foundation, we present \textit{DefectBench}, the first multi-dimensional benchmark designed to interrogate LMMs beyond basic semantic recognition. \textit{DefectBench} evaluates 18 state-of-the-art (SOTA) LMMs across three escalating cognitive dimensions: Semantic Perception, Spatial Localization, and Generative Geometry Segmentation. Extensive experiments reveal that while current LMMs demonstrate exceptional topological awareness and semantic understanding (effectively diagnosing “what” and “how”), they exhibit significant deficiencies in metric localization precision (“where”). Crucially, however, we validate the viability of zero-shot generative segmentation, showing that general-purpose foundation models can rival specialized supervised networks without domain-specific training. This work provides both a rigorous benchmarking standard and a high-quality open-source database, establishing a new baseline for the advancement of autonomous AI agents in civil engineering.

评分理由: 论文核心研究LLM在低资源语言翻译中的应用，直接使用LLMs（10分）。提出SAGE框架，核心是使用强化学习（RL）代理（Agent）来筛选数据，与’LLM Agents’高度相关（10分）。框架采用LoRA进行高效微调，是参数高效微调的核心技术（10分）。研究强调’right data over big data’，关注数据质量对模型性能的影响，与’Scaling Laws AND Data Quality’有一定关联（8分）。论文未涉及其他关键词如MoE、SLMs、预训练、对齐、RAG、推理加速等具体技术，因此这些关键词评分为0。

构建包容性万维网的愿景正遭受严峻语言鸿沟的阻碍，这在东南亚资源匮乏地区尤为突出。尽管大语言模型（LLMs）为翻译提供了潜在解决方案，但其在数据稀缺环境中的部署面临双重挑战：高质量、文化适配数据的匮乏，以及在海量嘈杂网络语料上进行训练所产生的高昂能源成本。为化解数字包容性与环境可持续性之间的矛盾，我们提出了可持续智能体引导专家调优框架（Sustainable Agent-Guided Expert-tuning, SAGE）。该框架开创了一种能源感知新范式，强调“优质数据”优先于“海量数据”。SAGE摒弃对未过滤数据集进行高碳排放训练的传统方式，转而采用通过群体相对策略优化（Group Relative Policy Optimization, GRPO）训练的强化学习（RL）智能体，自主构建精炼训练集。该智能体利用从少量专家构建的社区对话集中提取的语义奖励信号，有效过滤噪声及文化失配内容。随后，我们采用低秩自适应技术（Low-Rank Adaptation, LoRA）在此精选数据上高效微调开源大语言模型。我们将SAGE应用于英语与七种东南亚低资源语言（low-resource languages, LRLs）的翻译任务。该方法在BLEU-4和COMET-22指标上取得了最先进的性能表现，能有效捕捉本地语言细微特征。至关重要的是，SAGE在超越基于完整数据集训练的基线模型的同时，将数据使用量降低了97.1%，训练能耗减少了95.2%。通过以最小环境代价实现高性能模型，SAGE为弥合全球南方数字鸿沟提供了一条可扩展且负责任的技术路径。

The vision of an inclusive World Wide Web is impeded by a severe linguistic divide, particularly for communities in low-resource regions of Southeast Asia. While large language models (LLMs) offer a potential solution for translation, their deployment in data-poor contexts faces a dual challenge: the scarcity of high-quality, culturally relevant data and the prohibitive energy costs of training on massive, noisy web corpora. To resolve the tension between digital inclusion and environmental sustainability, we introduce Sustainable Agent-Guided Expert-tuning (SAGE). This framework pioneers an energy-aware paradigm that prioritizes the “right data” over “big data”. Instead of carbon-intensive training on unfiltered datasets, SAGE employs a reinforcement learning (RL) agent, optimized via Group Relative Policy Optimization (GRPO), to autonomously curate a compact training set. The agent utilizes a semantic reward signal derived from a small, expert-constructed set of community dialogues to filter out noise and cultural misalignment. We then efficiently fine-tune open-source LLMs on this curated data using Low-Rank Adaptation (LoRA). We applied SAGE to translation tasks between English and seven low-resource languages (LRLs) in Southeast Asia. Our approach establishes new state-of-the-art performance on BLEU-4 and COMET-22 metrics, effectively capturing local linguistic nuances. Crucially, SAGE surpasses baselines trained on full datasets while reducing data usage by 97.1% and training energy consumption by 95.2%. By delivering high-performance models with a minimal environmental footprint, SAGE offers a scalable and responsible pathway to bridge the digital divide in the Global South.

评分理由: 论文提出All-Mem框架，用于构建具有终身记忆的交互式智能体（LLM Agents），核心是通过动态拓扑演化管理记忆库，以改进检索和问答性能。这与"LLM Agents"高度相关（10分），因为论文直接研究智能体系统；与"Retrieval-Augmented Generation"高度相关（10分），因为框架涉及记忆检索以增强生成；与"Large Language Models"高度相关（10分），因为摘要提到使用LLM作为诊断器；与"Context Window Extension"有一定关联（5分），因为框架旨在处理长期记忆，间接涉及上下文管理；其他关键词如MoE、SFT、RLHF等未在论文中涉及，给0分。

终身交互智能体需在数月或数年内持续协助用户，这要求其在固定的上下文与延迟预算下，持续写入长期记忆并为每个新查询检索正确的证据。现有记忆系统常随历史增长而性能下降，检索出的上下文易出现冗余、过时或噪声问题。我们提出All-Mem——一种在线/离线终身记忆框架，通过显式的非破坏性整合来维护拓扑结构化的记忆库，避免了基于摘要的压缩方法中典型的信息不可逆损失。在线运行时，该系统将检索锚定在有限的可见表层，以保持粗粒度搜索成本可控。在定期离线阶段，大型语言模型诊断器会提出带有置信度评分的拓扑编辑建议，通过SPLIT、MERGE和UPDATE三种算子进行门控执行，同时保留不可变的证据以确保可追溯性。在查询时，类型化链接支持在必要时从活跃锚点向归档证据进行跳数受限、预算可控的扩展。在LOCOMO和LONGMEMEVAL数据集上的实验表明，本方法在检索与问答任务上优于代表性基线模型。

Lifelong interactive agents are expected to assist users over months or years, which requires continually writing long term memories while retrieving the right evidence for each new query under fixed context and latency budgets. Existing memory systems often degrade as histories grow, yielding redundant, outdated, or noisy retrieved contexts. We present All-Mem, an online/offline lifelong memory framework that maintains a topology structured memory bank via explicit, non destructive consolidation, avoiding the irreversible information loss typical of summarization based compression. In online operation, it anchors retrieval on a bounded visible surface to keep coarse search cost bounded. Periodically offline, an LLM diagnoser proposes confidence scored topology edits executed with gating using three operators: SPLIT, MERGE, and UPDATE, while preserving immutable evidence for traceability. At query time, typed links enable hop bounded, budgeted expansion from active anchors to archived evidence when needed. Experiments on LOCOMO and LONGMEMEVAL show improved retrieval and QA over representative baselines.

评分理由: 论文核心研究LLMs在文本分类任务中应用推理策略（如CoT、ToT等）的有效性和效率，与"Large Language Models"和"Chain of Thought"高度相关（10分），与"System 2 Thinking"相关（8分），因为涉及深思熟虑的推理过程。论文提到在小型模型上的测试，与"Small Language Models"有一定关联（5分）。其他关键词如MoE、Scaling Laws、Alignment等未在摘要中涉及，评分为0分。

从大型语言模型中引出显式、逐步的推理轨迹已成为增强模型能力的主导范式。尽管此类推理策略最初是为需要显式多步推理的问题设计的，但它们已越来越多地应用于广泛的自然语言处理任务中。这种扩展隐含地假设审慎推理能一致地有益于异构任务。然而，此类推理机制是否真正有益于分类任务在很大程度上仍未得到充分探索，特别是考虑到其巨大的令牌和时间成本。为填补这一空白，我们引入了TextReasoningBench，这是一个旨在系统评估大型语言模型在文本分类任务中推理策略有效性与效率的基准。我们在五个文本分类数据集上，针对十种大型语言模型，比较了七种推理策略，即IO、思维链、自洽思维链、思维树、思维图、思维链束以及长思维链。除了准确率和宏观F1分数等传统指标外，我们引入了两个成本感知评估指标，用于量化每个推理令牌带来的性能增益，以及相对于令牌成本增长的性能提升效率。实验结果揭示了三个值得注意的发现：（1）推理并非普遍提升分类性能：虽然中等复杂度的策略如思维链和自洽思维链能带来一致但有限的增益（通常在大模型上为+1%至+3%），但更复杂的方法（例如思维树和思维图）往往无法超越更简单的基线，甚至可能降低性能，尤其是在小模型上；（2）推理通常是低效的：许多推理策略将令牌消耗增加了10倍至100倍（例如自洽思维链和思维树），却仅带来微小的性能提升。

Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.

评分理由: 论文核心研究多模态大语言模型（MLLMs）的监督数据选择问题，与"Large Language Models"高度相关（10分），因为MLLMs是LLMs的扩展。与"Post-training"高度相关（10分），因为研究重点是训练后性能增益的预测。与"Scaling Laws” AND “Data Quality"有一定关联（5分），因为涉及数据质量对性能的影响。与"Pre-training"有一定关联（5分），因为研究监督数据选择，属于预训练或适应阶段。其他关键词如MoE、SLMs、RLHF、RAG等与论文内容无直接关系，得0分。

为多模态大语言模型（MLLMs）选择监督数据的传统思路是优先选择与目标基准看似相似的数据集，例如文本密集型或视觉中心型任务。然而，这种直观的相似性能否可靠地预测下游性能提升尚不明确。在本研究中，我们首次尝试回答一个实际问题：能否在训练开始前就预估某个训练数据集对目标基准的影响？为探究此问题，我们对涵盖7种不同任务的14个视觉-语言数据集间的迁移效应进行了深入分析。结果表明，直观的任务相似性并非迁移能力的可靠预测指标，泛化性能更多地取决于具体数据集而非其宽泛的任务类别。基于此发现，我们提出了DATAPROPHET——一种简单有效的免训练度量方法，它融合了多模态困惑度、相似性和数据多样性。实验表明，DATAPROPHET生成的监督数据排序与实际训练后性能增益的排序高度相关，肯德尔tau系数达到86.0%。此外，DATAPROPHET能实现更优的监督数据选择：相比均匀选择提升最高达6.9%，优于当前最先进的基于训练的基线方法1.4%，甚至比基于实验性能的预言机选择高出0.2%。我们的代码与数据将公开释放。

Conventional wisdom for selecting supervision data for multimodal large language models (MLLMs) is to prioritize datasets that appear similar to the target benchmark, such as text-intensive or vision-centric tasks. However, it remains unclear whether such intuitive similarity reliably predicts downstream performance gains. In this work, we take a first step toward answering a practical question: can we estimate the influence of a training dataset on a target benchmark before any training is performed? To investigate this question, we conduct an in-depth analysis of transfer across 14 vision-language datasets spanning 7 diverse tasks. Our results show that intuitive task similarity is an unreliable predictor of transferability, and that generalization depends more on the specific dataset than on its broad task category. Motivated by this finding, we propose DATAPROPHET, a simple and effective training-free metric that combines multimodal perplexity, similarity, and data diversity. Experiments show that DATAPROPHET produces supervision-data rankings that strongly correlate with rankings based on actual post-training performance gains, achieving a Kendall’s tau of 86.0%. Moreover, DATAPROPHET enables better supervision-data selection, yielding up to 6.9% improvement over uniform selection, 1.4% over a state-of-the-art training-based baseline, and 0.2% above oracle selection based on experimental performance. Our code and data will be released.

评分理由: 论文核心研究LLM的上下文压缩技术，以降低推理成本，因此与"Large Language Models"高度相关（10分）。研究涉及缩短上下文长度，与"Context Window Extension"相关（8分），因为压缩上下文可视为扩展上下文窗口的补充技术。研究目标是降低推理成本，与"Speculative Decoding"相关（8分），因为两者都关注推理效率。其他关键词如MoE、SLMs、训练方法、对齐、代理等均未在论文中涉及，给0分。

尽管上下文压缩能够通过缩短上下文来缓解大型语言模型日益增长的计算成本，但现有指定目标压缩率或长度的方法存在性能下降不可预测的问题，阻碍了其可靠部署。我们引入了一种面向性能的上下文压缩范式转变，即开发者指定可接受的性能下限而非压缩率。该范式采用轻量级性能预测器，在引导现成压缩器之前自动寻找满足此约束的最激进压缩率。我们设计并比较了两种预测器变体：一种是简单的上下文无关预测器，另一种是更复杂的上下文感知预测器，后者会考虑输入内容固有的可压缩性。在问答和摘要生成基准测试中，上下文感知预测器始终比上下文无关预测器实现更低的性能预测误差，而由此产生的上下文感知面向性能压缩方案获得了更优的整体性能。我们的工作为大型语言模型上下文压缩实现更可靠、高效且性能感知的部署铺平了道路。

While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio or length suffer from unpredictable performance degradation, hindering their reliable deployment. We introduce a paradigm shift to Performance-oriented Context Compression (PoC), where developers specify an acceptable performance floor instead of a compression ratio. PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio that satisfies this constraint before steering an off-the-shelf compressor. We design and compare two predictor variants: a simple context-agnostic predictor and a more sophisticated context-aware one that considers the input’s inherent compressibility. On both question-answering and summarization benchmarks, the context-aware predictor consistently achieves lower performance prediction error than the context-agnostic predictor, while the resulting context-aware PoC attains a superior overall performance. Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for LLMs.

评分理由: 论文核心是LLMs在航空航天领域电子元件资格数据集成与检索中的应用，属于大模型在不同领域的研究应用。摘要明确提到使用LLMs增强检索和减少数据清洗的人工工作，并与RAG方法进行了比较，因此"Large Language Models"和"Retrieval-Augmented Generation"高度相关（10分）。论文涉及航空航天领域的科学应用，与"AI for Science"有一定关联（5分）。其他关键词如MoE、SFT、RLHF、量化等均未在摘要中提及或暗示，属于完全无关（0分）。

大型制造企业因各部门维护的数据孤岛而面临信息检索挑战，导致数据库间存在不一致与错位问题。本文介绍了在卫星板卡设计中整合与检索电子元器件资质数据的实践经验。由于数据孤岛的存在，设计人员无法即时确定单个元器件的资质状态。然而，该流程在规划阶段至关重要——即生产前发布装配图纸时——以优化新资质认证并避免重复工作。为此，我们提出一种技术流程：利用虚拟知识图谱（Virtual Knowledge Graphs）实现异构数据源的统一视图，并采用大语言模型（LLMs）增强检索能力、减少数据清洗的人工投入。资质检索通过两种机制实现：基于本体论的数据访问（Ontology-based Data Access）方法用于结构化查询，以及向量搜索机制用于根据相似文本属性检索资质记录。我们进行了成本效益对比分析，证明所提出的流程在长期效率上也优于单纯依赖大语言模型的方法（如检索增强生成技术RAG）。

Large manufacturing companies face challenges in information retrieval due to data silos maintained by different departments, leading to inconsistencies and misalignment across databases. This paper presents an experience in integrating and retrieving qualification data for electronic components used in satellite board design. Due to data silos, designers cannot immediately determine the qualification status of individual components. However, this process is critical during the planning phase, when assembly drawings are issued before production, to optimize new qualifications and avoid redundant efforts. To address this, we propose a pipeline that uses Virtual Knowledge Graphs for a unified view over heterogeneous data sources and LLMs to enhance retrieval and reduce manual effort in data cleansing. The retrieval of qualifications is then performed through an Ontology-based Data Access approach for structured queries and a vector search mechanism for retrieving qualifications based on similar textual properties. We perform a comparative cost-benefit analysis, demonstrating that the proposed pipeline also outperforms approaches relying solely on LLMs, such as Retrieval-Augmented Generation (RAG), in terms of long-term efficiency.

评分理由: 论文提出了一种名为SeGroS的微调框架，专门用于解决统一多模态模型（UMMs）中的粒度不匹配和监督冗余问题，核心贡献是增强跨模态对齐。因此，与"Post-training” OR “Supervised Fine-tuning” OR “SFT"高度相关（10分），因为SeGroS是一个微调框架；与"Instruction Tuning” OR “Alignment” OR “Value Alignment"高度相关（10分），因为论文核心目标是增强对齐（alignment）。与"Large Language Models” OR “LLMs” OR “Foundation Models"有一定关联（5分），因为UMMs通常基于或扩展了大型基础模型，但论文未明确聚焦LLMs。其他关键词（如MoE、Scaling Laws、RLHF等）在摘要中未提及，因此评分为0分。

统一多模态模型（Unified Multimodal Models, UMMs）作为一种将多模态理解与生成整合于统一建模框架内的新兴范式，已展现出巨大潜力。然而，当前生成式训练范式存在固有的局限性。本文提出语义基础监督（Semantically-Grounded Supervision, SeGroS），这是一个旨在解决UMMs中粒度不匹配与监督冗余问题的微调框架。其核心在于，我们提出了一种新颖的视觉基础定位图，用以构建两种互补的监督信号。首先，我们设计了语义视觉提示，以弥补文本提示的稀疏性。其次，我们生成语义基础化的损坏输入，通过将重建损失限制在与文本对齐的核心区域，从而显式地增强基于掩码的UMMs的监督。在GenEval、DPGBench和CompBench上的广泛评估表明，SeGroS显著提升了多种UMM架构的生成保真度与跨模态对齐能力。

Unified Multimodal Models (UMMs) have emerged as a promising paradigm that integrates multimodal understanding and generation within a unified modeling framework. However, current generative training paradigms suffer from inherent limitations. We present Semantically-Grounded Supervision (SeGroS), a fine-tuning framework designed to resolve the granularity mismatch and supervisory redundancy in UMMs. At its core, we propose a novel visual grounding map to construct two complementary supervision signals. First, we formulate semantic Visual Hints to compensate for the sparsity of text prompts. Second, we generate a semantically-grounded Corrupted Input to explicitly enhance the supervision of masking-based UMMs by restricting the reconstruction loss to core text-aligned regions. Extensive evaluations on GenEval, DPGBench, and CompBench demonstrate that SeGroS significantly improves generation fidelity and cross-modal alignment across various UMM architectures.

评分理由: 论文专注于无线通信领域，提出了一种基于世界模型（World Models）的自监督学习框架，用于学习无线信道状态信息（CSI）的预测性和结构化表示。因此，它与关键词"World Models” AND “General World Models"高度相关（10分），因为论文明确将问题建模为世界建模任务。同时，论文属于AI在科学领域的应用（无线通信），因此与"AI for Science” OR “Bioinformatics” OR “Cheminformatics"有一定关联（5分），尽管它不属于生物信息学或化学信息学。其他所有关键词均与论文内容无关（0分），因为它们主要涉及大语言模型（LLMs）及其相关技术（如微调、对齐、推理、代理等），而本文研究的是无线信道建模，未涉及任何语言模型或相关技术。

我们提出一种自监督框架，通过将信道状态信息（CSI）的时间演化建模于紧凑的潜空间中，以学习无线信道的预测性与结构化表征。该方法将问题构建为世界建模任务，并利用联合嵌入预测架构（Joint Embedding Predictive Architecture, JEPA）从CSI轨迹中学习动作条件化的潜态动态。为提升几何一致性与组合性，我们采用源自李代数的同态更新来参数化状态转移，从而构建出能反映空间布局与用户移动规律的结构化潜空间。在DICHASUS数据集上的评估表明，本方法在保持拓扑结构与预测未见环境中未来嵌入向量方面均优于现有基线。所得潜空间能够生成度量精确的信道图谱，为移动感知调度、定位及无线场景理解等下游应用提供了可扩展的基础。

We introduce a self-supervised framework for learning predictive and structured representations of wireless channels by modeling the temporal evolution of channel state information (CSI) in a compact latent space. Our method casts the problem as a world modeling task and leverages the Joint Embedding Predictive Architecture (JEPA) to learn action-conditioned latent dynamics from CSI trajectories. To promote geometric consistency and compositionality, we parameterize transitions using homomorphic updates derived from Lie algebra, yielding a structured latent space that reflects spatial layout and user motion. Evaluations on the DICHASUS dataset show that our approach outperforms strong baselines in preserving topology and forecasting future embeddings across unseen environments. The resulting latent space enables metrically faithful channel charts, offering a scalable foundation for downstream applications such as mobility-aware scheduling, localization, and wireless scene understanding.

评分理由: 论文专注于软体机器人的形态与控制协同设计，使用图神经网络（GNN/GAT）作为核心方法，属于机器人学和AI交叉领域。所有关键词均与大语言模型（LLM）及其相关技术（如训练、对齐、推理、部署优化等）或特定科学AI应用（如生物信息学）直接相关。论文内容不涉及任何LLM、语言模型或自然语言处理技术，也未明确属于生物信息学或化学信息学等具体科学AI子领域。因此，除最后一个关键词（“AI for Science”）因论文属于广义的AI在科学/工程（机器人学）中的应用而获得5分（有一定关联）外，其余关键词均评为0分（完全无关）。

机器人的智能行为并非仅源于控制系统，而是产生于身体与大脑的紧密耦合，这一原理被称为具身智能（embodied intelligence）。设计能够利用这种相互作用的软体机器人仍然是一项重大挑战，尤其是在形态与控制需要同步优化的情况下。这一协同设计过程中的主要障碍在于：形态的演变可能破坏已习得的控制策略，使得重用或调整现有知识变得困难。为此，我们开发了一种基于图神经网络（Graph Neural Network）的方法，用于形态与控制器的协同设计。每个机器人被表示为一个图，其中图注意力网络（Graph Attention Network, GAT）编码节点特征，并通过池化后的表征输入到一个多层感知机（Multilayer Perceptron, MLP）头部，以生成执行器指令或价值估计。在演化过程中，继承遵循拓扑一致性映射：共享的GAT层被重用，MLP隐藏层完整迁移，匹配的执行器输出被复制，而不匹配的输出则随机初始化并进行微调。这种形态感知的策略类别使得控制器能够在身体发生突变时进行适应。在基准测试中，与传统的仅使用MLP的协同设计方法相比，我们基于GAT的方法获得了更高的最终适应度，并对形态变化展现出更强的适应能力。这些结果表明，图结构策略为具身智能中不断演化的形态与控制之间提供了更有效的接口。

The intelligent behavior of robots does not emerge solely from control systems, but from the tight coupling between body and brain, a principle known as embodied intelligence. Designing soft robots that leverage this interaction remains a significant challenge, particularly when morphology and control require simultaneous optimization. A significant obstacle in this co-design process is that morphological evolution can disrupt learned control strategies, making it difficult to reuse or adapt existing knowledge. We address this by develop a Graph Neural Network-based approach for the co-design of morphology and controller. Each robot is represented as a graph, with a graph attention network (GAT) encoding node features and a pooled representation passed through a multilayer perceptron (MLP) head to produce actuator commands or value estimates. During evolution, inheritance follows a topology-consistent mapping: shared GAT layers are reused, MLP hidden layers are transferred intact, matched actuator outputs are copied, and unmatched ones are randomly initialized and fine-tuned. This morphology-aware policy class lets the controller adapt when the body mutates. On the benchmark, our GAT-based approach achieves higher final fitness and stronger adaptability to morphological variations compared to traditional MLP-only co-design methods. These results indicate that graph-structured policies provide a more effective interface between evolving morphologies and control for embodied intelligence.

评分理由: 论文VideoSeek提出了一种基于大语言模型（GPT-5）的视频智能体，通过工具引导的主动搜索机制减少视频帧解析的计算成本，核心涉及LLM Agents、Tool Use、Chain of Thought和System 2 Thinking等关键词，与这些关键词高度相关（10分）。其他关键词如MoE、量化、RAG等未在论文中涉及，得0分。

视频智能体模型在具有挑战性的视频-语言任务中取得了进展。然而，大多数智能体方法仍然严重依赖对密集采样视频帧的贪婪解析，导致计算成本高昂。我们提出了VideoSeek，一种长视野视频智能体，它利用视频逻辑流主动寻找答案关键证据，而非穷举解析整个视频。这一洞见使得模型能够使用少得多的视频帧，同时保持甚至提升其视频理解能力。VideoSeek在一个“思考-行动-观察”循环中运行，并配备了一个精心设计的工具包，用于收集多粒度视频观察。这种设计支持基于累积观察进行查询感知的探索，并实现实用的视频理解与推理。在四个具有挑战性的视频理解与推理基准测试上的实验表明，VideoSeek在仅使用远少于先前视频智能体和独立大型多模态模型（LMMs）帧数的同时，实现了强大的准确性。值得注意的是，VideoSeek在LVBench基准上相比其基础模型GPT-5取得了10.2个百分点的绝对提升，同时使用的帧数减少了93%。进一步的分析凸显了利用视频逻辑流、强大推理能力以及工具包设计互补作用的重要性。

Video agentic models have advanced challenging video-language tasks. However, most agentic approaches still heavily rely on greedy parsing over densely sampled video frames, resulting in high computational cost. We present VideoSeek, a long-horizon video agent that leverages video logic flow to actively seek answer-critical evidence instead of exhaustively parsing the full video. This insight allows the model to use far fewer frames while maintaining, or even improving, its video understanding capability. VideoSeek operates in a think-act-observe loop with a well-designed toolkit for collecting multi-granular video observations. This design enables query-aware exploration over accumulated observations and supports practical video understanding and reasoning. Experiments on four challenging video understanding and reasoning benchmarks demonstrate that VideoSeek achieves strong accuracy while using far fewer frames than prior video agents and standalone LMMs. Notably, VideoSeek achieves a 10.2 absolute points improvement on LVBench over its base model, GPT-5, while using 93% fewer frames. Further analysis highlights the significance of leveraging video logic flow, strong reasoning capability, and the complementary roles of toolkit design.

评分理由: 论文主要研究个性化视频生成，核心创新在于使用多模态大语言模型（MLLMs）来推断和分配主体特定的依赖关系，并构建关系先验，以增强生成控制。因此，仅与关键词’Large Language Models OR LLMs OR Foundation Models’高度相关（8分），因为MLLMs是LLMs的一种扩展应用。其他关键词涉及的大模型技术原理（如MoE、Scaling Laws、训练方法、推理优化、代理系统等）或特定科学领域应用（如生物信息学）均未在论文中提及或作为核心内容，故评分为0。

扩散模型的最新进展显著提升了文本到视频生成的质量，使得能够以前景和背景元素的细粒度控制进行个性化内容创作。然而，跨主体的精确人脸属性对齐仍然具有挑战性，因为现有方法缺乏确保组内一致性的显式机制。解决这一差距需要显式的建模策略和人脸属性感知的数据资源。为此，我们提出了LumosX框架，该框架在数据和模型设计两方面均有所推进。在数据方面，一个定制的收集流程从独立视频中编排字幕和视觉线索，同时多模态大语言模型推断并分配主体特定的依赖关系。这些提取的关系先验施加了更细粒度的结构，增强了个性化视频生成的表达控制，并支持构建一个全面的基准。在建模方面，关系自注意力与关系交叉注意力将位置感知嵌入与精细化的注意力动态交织在一起，以刻写显式的主体-属性依赖关系，从而强制实现有纪律的组内凝聚力，并放大不同主体集群之间的分离。在我们构建的基准上进行全面评估表明，LumosX在细粒度、身份一致且语义对齐的个性化多主体视频生成方面实现了最先进的性能。代码和模型可在 https://jiazheng-xing.github.io/lumosx-home/ 获取。

Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.

评分理由: 该论文专注于视觉语言模型（VLM）的图像篡改检测，提出了新的分类法、基准和评估指标。虽然涉及视觉语言模型，但所有评分关键词均针对大语言模型（LLM）的技术原理、训练方法、推理优化、对齐、压缩、应用等具体方面，而论文的核心是计算机视觉中的篡改检测任务，包括像素级定位、语义分类和语言描述生成，并未涉及LLM的架构、训练、推理优化、对齐或压缩等任何技术细节。因此，所有关键词均与论文内容完全无关，得分为0。

现有篡改检测基准主要依赖物体掩码，这与真实编辑信号存在严重偏差：掩码内的许多像素未被修改或仅发生轻微改动，而掩码外细微却具有实质影响的编辑却被视为自然图像。我们将视觉语言模型（VLM）图像篡改检测任务重新定义为从粗粒度区域标注转向像素级锚定、语义与语言感知的任务。首先，我们提出了一套涵盖编辑基元（替换/移除/拼接/修复/属性修改/色彩调整等）及其篡改对象语义类别的分类体系，将底层视觉变化与高层语义理解相连接。其次，我们发布了包含逐像素篡改标注图及配对类别监督的新基准数据集，用于在统一协议下评估检测与分类性能。第三，我们提出了一个训练框架和评估指标，通过定位置信度或真实编辑强度的预测来量化像素级准确性，并进一步通过语义感知分类和针对预测区域的自然语言描述来衡量篡改语义理解能力。我们还在当前先进的篡改检测器上重新评估了现有强分割/定位基线方法，发现仅使用掩码指标会导致显著的高估或低估评分，同时揭示了其在微观编辑和掩码外修改上的失效模式。我们的框架推动该领域从掩码评估转向像素、语义及语言描述层面，为篡改定位、语义分类与描述建立了严谨标准。代码与基准数据详见 https://github.com/VILA-Lab/PIXAR。

Existing tampering detection benchmarks largely rely on object masks, which severely misalign with the true edit signal: many pixels inside a mask are untouched or only trivially modified, while subtle yet consequential edits outside the mask are treated as natural. We reformulate VLM image tampering from coarse region labels to a pixel-grounded, meaning and language-aware task. First, we introduce a taxonomy spanning edit primitives (replace/remove/splice/inpaint/attribute/colorization, etc.) and their semantic class of tampered object, linking low-level changes to high-level understanding. Second, we release a new benchmark with per-pixel tamper maps and paired category supervision to evaluate detection and classification within a unified protocol. Third, we propose a training framework and evaluation metrics that quantify pixel-level correctness with localization to assess confidence or prediction on true edit intensity, and further measure tamper meaning understanding via semantics-aware classification and natural language descriptions for the predicted regions. We also re-evaluate the existing strong segmentation/localization baselines on recent strong tamper detectors and reveal substantial over- and under-scoring using mask-only metrics, and expose failure modes on micro-edits and off-mask changes. Our framework advances the field from masks to pixels, meanings and language descriptions, establishing a rigorous standard for tamper localization, semantic classification and description. Code and benchmark data are available at https://github.com/VILA-Lab/PIXAR.

评分理由: 论文研究网络安全领域的多模态对比学习，旨在通过文本模态的知识迁移来改善网络负载分类的泛化能力。虽然论文提到了使用LLM生成合成负载数据，但这只是数据生成工具，并非研究的核心内容。论文的核心技术是多模态对比学习框架，与评分关键词列表中的大模型技术原理、训练方法、推理优化、对齐技术、代理系统等主题均无直接关联。所有关键词均与论文研究内容无关，因此全部评分为0分。

机器学习在网络安全领域的应用长期受限于泛化问题：在受控场景下表现良好的模型，在实际部署中往往无法保持性能。其根本原因通常在于机器学习算法习得的是表层模式（捷径），而非底层的网络安全概念。本文研究对比多模态学习，以此作为提升机器学习在网络安全任务中性能的初步尝试。我们的目标是将知识从数据丰富的模态（如文本）迁移到数据稀缺的模态（如网络载荷）。我们以威胁分类为案例，提出一个两阶段多模态对比学习框架，该框架利用文本形式的漏洞描述来指导载荷分类。首先，我们通过对描述进行对比学习，构建一个语义上有意义的嵌入空间。随后，我们将载荷对齐至此空间，从而实现从文本到载荷的知识迁移。我们在一个大规模私有数据集和一个基于公开CVE描述与LLM生成载荷构建的合成基准上评估了该方法。实验表明，该方法在两个基准测试上均减少了基线模型中的捷径学习现象。我们将此合成基准与源代码作为开源项目发布。

The use of ML in cybersecurity has long been impaired by generalization issues: Models that work well in controlled scenarios fail to maintain performance in production. The root cause often lies in ML algorithms learning superficial patterns (shortcuts) rather than underlying cybersecurity concepts. We investigate contrastive multi-modal learning as a first step towards improving ML performance in cybersecurity tasks. We aim at transferring knowledge from data-rich modalities, such as text, to data-scarce modalities, such as payloads. We set up a case study on threat classification and propose a two-stage multi-modal contrastive learning framework that uses textual vulnerability descriptions to guide payload classification. First, we construct a semantically meaningful embedding space using contrastive learning on descriptions. Then, we align payloads to this space, transferring knowledge from text to payloads. We evaluate the approach on a large-scale private dataset and a synthetic benchmark built from public CVE descriptions and LLM-generated payloads. The methodology appears to reduce shortcut learning over baselines on both benchmarks. We release our synthetic benchmark and source code as open source.

评分理由: 论文研究的是长视频理解中的帧选择方法，虽然涉及大视觉语言模型（VLMs），但所有评分关键词都专门针对大语言模型（LLMs）及其相关技术（如MoE、SFT、RLHF、RAG等），而论文的核心是视频帧选择算法（基于SigLIP和DINOv2嵌入的贪婪优化），并非LLM技术原理或应用创新。论文未涉及任何评分关键词中的具体技术，因此所有关键词相关度均为0。

大型视觉-语言模型（VLMs）正日益应用于长视频问答任务，但其推理过程常受限于输入帧数及由此产生的视觉标记数量。简单的稀疏采样可能遗漏关键瞬间，而纯粹基于相关性的选择则常陷入近重复帧的聚集，并牺牲对时间上分散证据的覆盖。本文提出一种问题自适应的贪婪帧选择方法，在固定帧数预算下联合优化查询相关性与语义代表性。我们的方法构建了一个精确时间戳对齐的1~FPS候选帧池（上限为1000帧），在两个互补空间中对候选帧进行嵌入（使用SigLIP编码问题相关性，DINOv2编码语义相似性），并通过贪婪最大化模块化相关性项与设施选址覆盖项的加权和来选择帧。该目标函数具有归一化、单调且次模的特性，从而获得标准的（1-1/e）贪婪近似保证。为平衡不同问题对相关性与覆盖度的差异化需求，我们引入了四种预设策略，并设计了一个轻量级的纯文本问题类型分类器，将每个查询路由至表现最佳的预设策略。在MLVU数据集上的实验表明，相较于均匀采样及近期强基线方法，本方法在不同帧数预算下均取得一致的准确率提升，且在严格预算条件下改进最为显著。

Large vision–language models (VLMs) are increasingly applied to long-video question answering, yet inference is often bottlenecked by the number of input frames and resulting visual tokens. Naive sparse sampling can miss decisive moments, while purely relevance-driven selection frequently collapses onto near-duplicate frames and sacrifices coverage of temporally distant evidence. We propose a question-adaptive greedy frame selection method that jointly optimizes query relevance and semantic representativeness under a fixed frame budget. Our approach constructs a 1~FPS candidate pool (capped at 1000) with exact timestamp alignment, embeds candidates in two complementary spaces (SigLIP for question relevance and DINOv2 for semantic similarity), and selects frames by greedily maximizing a weighted sum of a modular relevance term and a facility-location coverage term. This objective is normalized, monotone, and submodular, yielding a standard (1-1/e) greedy approximation guarantee. To account for question-dependent trade-offs between relevance and coverage, we introduce four preset strategies and a lightweight text-only question-type classifier that routes each query to its best-performing preset. Experiments on MLVU show consistent accuracy gains over uniform sampling and a strong recent baseline across frame budgets, with the largest improvements under tight budgets.

评分理由: 论文核心研究大语言模型（LLM）驱动的AI代理在实验高能物理（HEP）领域的自主应用，属于AI for Science范畴，因此相关关键词（LLMs、LLM Agents、AI for Science）获得高分（10分）。论文提到集成文献知识检索（与RAG相关）和多代理审查（与Multi-agent Systems相关），这些关键词获得中等分数（5分）。其他关键词如MoE、SLMs、Scaling Laws、训练方法、推理优化、模型压缩等与论文内容无关，得0分。

基于大语言模型的人工智能代理现已能够在专家输入最少的情况下，自主执行高能物理（HEP）分析流程的大部分环节。在获得高能物理数据集、执行框架及过往实验文献库的条件下，我们发现Claude Code能够成功实现典型分析的所有阶段自动化：事例选择、本底估计、不确定性量化、统计推断以及论文草拟。我们认为，实验高能物理学界目前低估了这些系统的现有能力，且多数已提出的代理工作流程范围过于局限或过度依赖于特定分析结构的预设框架。我们提出了一个概念验证框架——仅提供上下文（Just Furnish Context，JFC），该框架将自主分析代理与基于文献的知识检索及多代理评审相结合，并证明其足以规划、执行和记录一个可信的高能物理分析。我们通过使用ALEPH、DELPHI和CMS的开放数据进行电弱、量子色动力学（QCD）及希格斯玻色子测量的分析，验证了该框架的有效性。这些工具并非旨在取代物理学家，而是有望减轻分析代码开发中重复性的技术负担，使研究人员能够专注于物理洞察、真正新颖的方法开发以及严格的验证工作。鉴于这些进展，我们主张学界应重新思考如何培养学生、组织分析工作以及分配人力资源的新策略。

Large language model-based AI agents are now able to autonomously execute substantial portions of a high energy physics (HEP) analysis pipeline with minimal expert-curated input. Given access to a HEP dataset, an execution framework, and a corpus of prior experimental literature, we find that Claude Code succeeds in automating all stages of a typical analysis: event selection, background estimation, uncertainty quantification, statistical inference, and paper drafting. We argue that the experimental HEP community is underestimating the current capabilities of these systems, and that most proposed agentic workflows are too narrowly scoped or scaffolded to specific analysis structures. We present a proof-of-concept framework, Just Furnish Context (JFC), that integrates autonomous analysis agents with literature-based knowledge retrieval and multi-agent review, and show that this is sufficient to plan, execute, and document a credible high energy physics analysis. We demonstrate this by conducting analyses on open data from ALEPH, DELPHI, and CMS to perform electroweak, QCD, and Higgs boson measurements. Rather than replacing physicists, these tools promise to offload the repetitive technical burden of analysis code development, freeing researchers to focus on physics insight, truly novel method development, and rigorous validation. Given these developments, we advocate for new strategies for how the community trains students, organizes analysis efforts, and allocates human expertise.

评分理由: 论文核心研究LLM在心理理论推理中的应用，提出动态信念图模型来增强LLM的推理能力。高度相关关键词：LLMs（论文明确使用LLMs进行ToM推理）、Chain of Thought/System 2 Thinking（涉及多步深度推理过程）、LLM Agents（构建基于LLM的认知代理）、Explainable AI（模型提供可解释的信念轨迹）。其他关键词如MoE、SFT、RAG等未在论文中涉及，故评分为0。

大型语言模型（LLM）的心智理论（Theory of Mind, ToM）推理需要推断人们内隐且不断演变的信念如何影响其在不确定性下的信息寻求与行为——尤其是在灾难响应、急救医学和人机协同自主等高风险场景中。现有方法要么直接提示LLM，要么采用将信念视为静态且独立的隐状态模型，这常常导致随时间推移产生不一致的心理模型，并在动态情境中表现出薄弱的推理能力。我们提出一种用于基于LLM的ToM推理的结构化认知轨迹模型，该模型将心理状态表征为动态信念图，能够联合推断隐信念、学习其时变依赖关系，并将信念演化与信息寻求及决策联系起来。我们的模型贡献在于：（i）一种从文本化概率陈述到一致性概率图模型更新的新颖映射方法；（ii）基于能量的因子图对信念相互依赖关系的表征；（iii）一种基于证据下界（ELBO）的目标函数，用于捕捉信念累积与延迟决策。在多个真实世界灾难疏散数据集上的实验表明，我们的模型显著提升了行为预测能力，并恢复了与人类推理一致的可解释信念轨迹，为在高不确定性环境中为LLM增强ToM能力提供了一个原理性模块。https://anonymous.4open.science/r/ICML_submission-6373/

Theory of Mind (ToM) reasoning with Large Language Models (LLMs) requires inferring how people’s implicit, evolving beliefs shape what they seek and how they act under uncertainty – especially in high-stakes settings such as disaster response, emergency medicine, and human-in-the-loop autonomy. Prior approaches either prompt LLMs directly or use latent-state models that treat beliefs as static and independent, often producing incoherent mental models over time and weak reasoning in dynamic contexts. We introduce a structured cognitive trajectory model for LLM-based ToM that represents mental state as a dynamic belief graph, jointly inferring latent beliefs, learning their time-varying dependencies, and linking belief evolution to information seeking and decisions. Our model contributes (i) a novel projection from textualized probabilistic statements to consistent probabilistic graphical model updates, (ii) an energy-based factor graph representation of belief interdependencies, and (iii) an ELBO-based objective that captures belief accumulation and delayed decisions. Across multiple real-world disaster evacuation datasets, our model significantly improves action prediction and recovers interpretable belief trajectories consistent with human reasoning, providing a principled module for augmenting LLMs with ToM in high-uncertainty environment. https://anonymous.4open.science/r/ICML_submission-6373/

评分理由: 论文主要研究机器人社交行为的自主优化框架，使用视觉语言模型（VLM）作为社会批评者进行自我评估和重新规划。与大多数关键词无关，因为论文不涉及大语言模型技术原理、训练方法、推理优化、模型压缩等核心大模型技术。仅与两个关键词相关：1）‘Self-Correction OR Self-Improvement OR Self-Reflection’（10分）：论文核心是机器人通过VLM评估进行自我批评和迭代优化行为，属于自我改进范畴；2）‘LLM Agents OR Autonomous Agents OR Agentic Workflow’（8分）：论文研究自主机器人代理框架，但使用的是VLM而非LLM，且侧重于行为规划而非通用代理工作流。

传统机器人社会行为生成在灵活性与自主性方面存在局限，常依赖于预定义动作或人类反馈。本研究提出CRISP（交互式社会存在批判与重规划框架），这是一种自主框架，机器人通过利用视觉语言模型作为“类人社会批判者”来评估并重规划自身行为。CRISP整合了以下环节：（1）通过分析机器人描述文件（如MJCF）提取可动关节与约束条件；（2）基于情境上下文生成逐步行为规划；（3）参照视觉信息（关节运动范围可视化）生成底层关节控制代码；（4）基于VLM评估社会适宜性与自然度，包括精准定位错误步骤；（5）通过基于奖励的搜索实现行为迭代优化。该方法不依赖特定机器人API，仅需机器人结构文件即可在不同平台上生成具有细微差异的类人动作。在一项涵盖五种机器人类型（包括移动机械臂与人形机器人）和20种场景的用户研究中，与现有方法相比，本方法在用户偏好和情境适宜性评分上均获得显著提升。本研究提出了一个通用框架，在最大限度减少人工干预的同时，拓展了机器人的自主交互能力与跨平台适用性。 本工作的详细结果视频及补充信息详见：https://limjiyu99.github.io/inner-critic/

Conventional robot social behavior generation has been limited in flexibility and autonomy, relying on predefined motions or human feedback. This study proposes CRISP (Critique-and-Replan for Interactive Social Presence), an autonomous framework where a robot critiques and replans its own actions by leveraging a Vision-Language Model (VLM) as a `human-like social critic.’ CRISP integrates (1) extraction of movable joints and constraints by analyzing the robot’s description file (e.g., MJCF), (2) generation of step-by-step behavior plans based on situational context, (3) generation of low-level joint control code by referencing visual information (joint range-of-motion visualizations), (4) VLM-based evaluation of social appropriateness and naturalness, including pinpointing erroneous steps, and (5) iterative refinement of behaviors through reward-based search. This approach is not tied to a specific robot API; it can generate subtly different, human-like motions on various platforms using only the robot’s structure file. In a user study involving five different robot types and 20 scenarios, including mobile manipulators and humanoids, our proposed method achieved significantly higher preference and situational appropriateness ratings compared to previous methods. This research presents a general framework that minimizes human intervention while expanding the robot’s autonomous interaction capabilities and cross-platform applicability. Detailed result videos and supplementary information regarding this work are available at: https://limjiyu99.github.io/inner-critic/

评分理由: 论文核心研究LLM中Chain-of-Thought（CoT）的忠实性评估问题，直接涉及’Chain of Thought OR CoT Reasoning OR Multi-step Reasoning’（10分）和’Large Language Models OR LLMs OR Foundation Models’（10分）。研究探讨评估方法的差异如何影响忠实性测量，与’Mechanistic Interpretability OR Explainable AI’（8分）和’Hallucination Mitigation OR Factuality OR Truthfulness’（8分）相关，因为忠实性评估涉及模型输出的真实性和可解释性。论文提到自我反思式评估，与’Self-Correction OR Self-Improvement OR Self-Reflection’（5分）有间接关联，并涉及深度推理评估，与’System 2 Thinking OR Slow Thinking OR In-depth Reasoning’（5分）相关。其他关键词如MoE、SLMs、训练技术、推理加速、AI for Science等与论文内容无关，得0分。

近期关于思维链（CoT）忠实性的研究常报告单一聚合数值（例如DeepSeek-R1在39%的情况下承认提示），这暗示忠实性是模型客观可度量的属性。本文论证事实并非如此。研究对来自12个开源模型（涵盖9个系列、参数量从7B到1T）的10,276条受干预推理轨迹，应用了三种分类器（纯正则表达式检测器、正则表达式+大语言模型的两阶段流程、独立的Claude Sonnet 4评估器）。在相同数据上，这些分类器得出的总体忠实率分别为74.4%、82.6%和69.7%，其95%置信区间互不重叠。各模型间的忠实率差异达2.6至30.6个百分点，且均具有统计显著性（McNemar检验，p < 0.001）。分歧具有系统性而非随机性：分类器间一致性系数（Cohen’s kappa）从谄媚性提示的0.06（“轻微一致”）到评分者提示的0.42（“中等一致”），且不对称性显著——在谄媚性案例中，883条被流程分类为忠实而被Sonnet评估器判定为不忠实，反向情况仅出现2例。分类器选择甚至会逆转模型排名：Qwen3.5-27B在流程标准下排名第1，在Sonnet评估器下降至第7；OLMo-3.1-32B则从第9位升至第3位。根本原因在于不同分类器对相关忠实性概念的操作化标准存在严格度差异（词汇提及vs认知依赖），导致对相同行为产生分歧测量。这些结果表明，采用不同分类器的研究间所公布的忠实性数据无法进行有效比较，未来评估应当报告多种分类方法下的敏感度范围而非单一数值估计。

Recent work on chain-of-thought (CoT) faithfulness reports single aggregate numbers (e.g., DeepSeek-R1 acknowledges hints 39% of the time), implying that faithfulness is an objective, measurable property of a model. This paper demonstrates that it is not. Three classifiers (a regex-only detector, a two-stage regex-plus-LLM pipeline, and an independent Claude Sonnet 4 judge) are applied to 10,276 influenced reasoning traces from 12 open-weight models spanning 9 families and 7B to 1T parameters. On identical data, these classifiers produce overall faithfulness rates of 74.4%, 82.6%, and 69.7%, respectively, with non-overlapping 95% confidence intervals. Per-model gaps range from 2.6 to 30.6 percentage points; all are statistically significant (McNemar’s test, p < 0.001). The disagreements are systematic, not random: inter-classifier agreement measured by Cohen’s kappa ranges from 0.06 (“slight”) for sycophancy hints to 0.42 (“moderate”) for grader hints, and the asymmetry is pronounced: for sycophancy, 883 cases are classified as faithful by the pipeline but unfaithful by the Sonnet judge, while only 2 go the other direction. Classifier choice can also reverse model rankings: Qwen3.5-27B ranks 1st under the pipeline but 7th under the Sonnet judge; OLMo-3.1-32B moves in the opposite direction, from 9th to 3rd. The root cause is that different classifiers operationalize related faithfulness constructs at different levels of stringency (lexical mention versus epistemic dependence), and these constructs yield divergent measurements on the same behavior. These results demonstrate that published faithfulness numbers cannot be meaningfully compared across studies that use different classifiers, and that future evaluations should report sensitivity ranges across multiple classification methodologies rather than single point estimates.

评分理由: 论文核心研究LLMs的uncertainty quantification方法，直接涉及’Large Language Models’和’Hallucination Mitigation’关键词（均给10分），因为论文明确研究LLMs输出可靠性问题并提出解决方案。其他关键词如MoE、SFT、RAG等均未在摘要中提及，与论文内容无关（给0分）。

大型语言模型（LLMs）已在多样化任务中展现出卓越能力。然而，其输出的真实性无法得到保证，且其倾向于过度自信的特性进一步限制了可靠性。不确定性量化为识别潜在不可靠输出提供了一种可行途径，但现有方法大多依赖重复采样或辅助模型，引入了显著的计算开销。为应对这些局限，我们提出语义令牌聚类（Semantic Token Clustering, STC），这是一种高效的不确定性量化方法，其利用LLMs内部固有的语义信息。具体而言，我们通过嵌入聚类和前缀匹配将令牌分组为语义一致的簇，并基于对应语义簇上聚合的概率质量来量化不确定性。该方法仅需单次生成，且不依赖辅助模型。实验结果表明，STC在显著降低计算开销的同时，取得了与前沿基线方法相当的性能。

Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks. However, the truthfulness of their outputs is not guaranteed, and their tendency toward overconfidence further limits reliability. Uncertainty quantification offers a promising way to identify potentially unreliable outputs, but most existing methods rely on repeated sampling or auxiliary models, introducing substantial computational overhead. To address these limitations, we propose Semantic Token Clustering (STC), an efficient uncertainty quantification method that leverages the semantic information inherently encoded in LLMs. Specifically, we group tokens into semantically consistent clusters using embedding clustering and prefix matching, and quantify uncertainty based on the probability mass aggregated over the corresponding semantic cluster. Our approach requires only a single generation and does not depend on auxiliary models. Experimental results show that STC achieves performance comparable to state-of-the-art baselines while substantially reducing computational overhead.

评分理由: 论文提出Design-OS框架，用于工程系统设计，涉及AI辅助设计流程和AI代理协作，但未具体提及大模型或深度学习技术。仅与’LLM Agents OR Autonomous Agents OR Agentic Workflow’（提及AI代理）和’AI for Science OR Bioinformatics OR Cheminformatics’（涉及工程科学应用）有弱关联，其他关键词均无关。

工程系统设计——无论是机电一体化、控制系统还是嵌入式系统——通常以临时方式进行，需求往往隐晦不明，从设计意图到参数的可追溯性基本缺失。现有的规范驱动与系统化设计方法主要针对软件领域，而人工智能辅助工具往往在方案生成阶段而非问题界定阶段介入设计流程。人机协作在物理系统设计中的应用仍待深入探索。本文提出Design-OS：一种轻量级、规范驱动的工程系统设计工作流，其架构包含五个阶段：概念定义、文献调研、概念设计、需求定义及设计定义。规范文件作为人类设计师与智能体之间的共享契约；每个阶段产出结构化成果物，保持设计可追溯性并支持智能体增强执行。我们将Design-OS与需求驱动设计、系统化设计框架及人工智能辅助设计流程进行对比定位，并通过两个旋转倒立摆平台——开源SimpleFOC反作用轮与商用Quanser古泽塔摆——的控制系统设计案例进行演示，展现同一规范驱动工作流如何适配根本性不同的实现方案。空白模板及完整设计案例成果物已公开于代码仓库以支持复现与复用。该工作流使设计过程可视化且可审计，并将人工智能的规范驱动编排从软件设计延伸至物理工程系统设计领域。

Engineering system design – whether mechatronic, control, or embedded – often proceeds in an ad hoc manner, with requirements left implicit and traceability from intent to parameters largely absent. Existing specification-driven and systematic design methods mostly target software, and AI-assisted tools tend to enter the workflow at solution generation rather than at problem framing. Human–AI collaboration in the design of physical systems remains underexplored. This paper presents Design-OS, a lightweight, specification-driven workflow for engineering system design organized in five stages: concept definition, literature survey, conceptual design, requirements definition, and design definition. Specifications serve as the shared contract between human designers and AI agents; each stage produces structured artifacts that maintain traceability and support agent-augmented execution. We position Design-OS relative to requirements-driven design, systematic design frameworks, and AI-assisted design pipelines, and demonstrate it on a control systems design case using two rotary inverted pendulum platforms – an open-source SimpleFOC reaction wheel and a commercial Quanser Furuta pendulum – showing how the same specification-driven workflow accommodates fundamentally different implementations. A blank template and the full design-case artifacts are shared in a public repository to support reproducibility and reuse. The workflow makes the design process visible and auditable, and extends specification-driven orchestration of AI from software to physical engineering system design.

评分理由: 论文研究的是传统的HAL（Hyperspace Analogue to Language）模型改进，通过引入注意力机制和SVD降维来增强文本分类性能。该工作属于经典的词向量和文本表示学习范畴，而非现代大语言模型（LLM）技术。所有关键词均围绕大模型、深度学习新技术及其应用，而本文方法（基于共现矩阵的HAL）与这些关键词无直接关联。唯一略有相关的是’Mechanistic Interpretability OR Explainable AI’，因为论文提到了通过注意力权重分析提升模型可解释性，但这不是核心创新点，因此给5分。其他关键词均不涉及，给0分。

超空间语言模拟（Hyperspace Analogue to Language, HAL）模型依赖全局词汇共现矩阵来构建分布语义表示。尽管这些表示能有效捕捉词汇关系，但通过标准均值池化将其聚合为句子级嵌入时，常导致信息损失。均值池化为所有词元分配等权重，从而使得上下文显著词的影响力被非信息性结构词元稀释。本文通过将可学习的温度缩放加性注意力机制整合到HAL表示流程中，以应对这一局限。为缓解原始共现矩阵的稀疏性与高维性问题，我们在注意力层之前应用截断奇异值分解（Truncated SVD），将向量投影至稠密潜在空间。我们在IMDB情感分析数据集上评估所提出的架构。实验结果表明，基于注意力的池化方法取得了82.38%的测试准确率，相较于传统均值池化基线（75.64%）实现了6.74个百分点的绝对提升。此外，对注意力权重的定性分析表明，该机制能有效抑制停用词，并选择性地关注承载情感信息的词元，从而同时提升了分类性能与模型可解释性。

The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all tokens, thereby diluting the impact of contextually salient words with uninformative structural tokens. In this paper, we address this limitation by integrating a learnable, temperature-scaled additive attention mechanism into the HAL representation pipeline. To mitigate the sparsity and high dimensionality of the raw co-occurrence matrices, we apply Truncated Singular Value Decomposition (SVD) to project the vectors into a dense latent space prior to the attention layer. We evaluate the proposed architecture on the IMDB sentiment analysis dataset. Empirical results demonstrate that the attention-based pooling approach achieves a test accuracy of 82.38%, yielding an absolute improvement of 6.74 percentage points over the traditional mean pooling baseline (75.64%). Furthermore, qualitative analysis of the attention weights indicates that the mechanism successfully suppresses stop-words and selectively attends to sentiment-bearing tokens, improving both classification performance and model interpretability.

评分理由: 论文提出了一种用于网络安全风险管理的多智能体AI系统，核心涉及LLM智能体架构（Mistral-7B等模型）和多智能体协调，因此与’LLM Agents’、‘Multi-agent Systems’高度相关（10分）。系统使用Mistral-7B（一种小型语言模型）并进行领域微调，与’Small Language Models’、‘Pre-training’、‘Post-training’相关（8分）。论文发现上下文窗口容量是主要限制因素，与’Context Window Extension’高度相关（10分）。其他关键词如MoE、Scaling Laws、RLHF等未在论文中涉及，给0分。

为小型组织实施真正的网络安全风险评估成本高昂——一次符合NIST网络安全框架（NIST CSF）标准的评估最低需花费1.5万美元，耗时数周，且依赖真正稀缺的专业人员。大多数小型企业完全跳过了这一环节。我们构建了一个六智能体人工智能系统，每个智能体负责一个分析阶段：组织画像描绘、资产映射、威胁分析、控制措施评估、风险评分及生成建议。各智能体共享一个随着评估推进而持续扩展的持久化上下文，因此后续智能体能基于先前智能体的结论开展工作——正是这一机制使本系统区别于标准的顺序式智能体流程。我们在一个受HIPAA监管的15人医疗保健公司进行了测试，并将系统输出与三位持有CISSP资质的从业者的独立评估结果进行对比：系统在风险严重程度分类上与专家判断的一致性达到85%，覆盖了92%已识别风险，且在15分钟内完成评估。随后，我们针对医疗保健、金融科技、制造业、零售业和SaaS五个行业领域，使用合成但符合行业实际的组织画像，进行了30轮重复的单智能体评估，对比了通用Mistral-7B模型与经领域微调的模型的表现。两种模型均完成了所有轮次评估。经微调的模型识别出了基线模型完全未能察觉的威胁：医疗保健领域的受保护健康信息（PHI）暴露、制造业的运营技术/工业物联网（OT/IIoT）漏洞，以及零售业特有的平台风险。然而，完整的多智能体流程在使用默认4096令牌上下文窗口的Tesla T4显卡上进行的30次尝试全部失败——结果表明，约束因素在于上下文容量，而非模型质量。

Getting a real cybersecurity risk assessment for a small organization is expensive – a NIST CSF-aligned engagement runs $15,000 on the low end, takes weeks, and depends on practitioners who are genuinely scarce. Most small companies skip it entirely. We built a six-agent AI system where each agent handles one analytical stage: profiling the organization, mapping assets, analyzing threats, evaluating controls, scoring risks, and generating recommendations. Agents share a persistent context that grows as the assessment proceeds, so later agents build on what earlier ones concluded – the mechanism that distinguishes this from standard sequential agent pipelines. We tested it on a 15-person HIPAA-covered healthcare company and compared outputs to independent assessments by three CISSP practitioners – the system agreed with them 85% of the time on severity classifications, covered 92% of identified risks, and finished in under 15 minutes. We then ran 30 repeated single-agent assessments across five synthetic but sector-realistic organizational profiles in healthcare, fintech, manufacturing, retail, and SaaS, comparing a general-purpose Mistral-7B against a domain fine-tuned model. Both completed every run. The fine-tuned model flagged threats the baseline could not see at all: PHI exposure in healthcare, OT/IIoT vulnerabilities in manufacturing, platform-specific risks in retail. The full multi-agent pipeline, however, failed every one of 30 attempts on a Tesla T4 with its 4,096-token default context window – context capacity, not model quality, turned out to be the binding constraint.

评分理由: 论文核心研究LLMs的安全对齐漏洞（jailbreak attacks），与’Large Language Models’高度相关（10分），并涉及安全对齐（alignment）问题（8分）。其他关键词如MoE、SLMs、Scaling Laws、训练方法、推理技术、AI for Science等均未在摘要中提及或相关，故评0分。

大型语言模型（LLM）已得到广泛部署，尤其是通过基于网络的免费应用程序，使其暴露于多样化的用户生成输入中，包括来自长尾分布（如低资源语言和加密私有数据）的输入。这种开放式暴露增加了越狱攻击的风险，可能破坏模型的安全对齐。尽管近期研究表明利用长尾分布可促进此类越狱，但现有方法主要依赖人工设计的规则，限制了对这些安全与隐私漏洞的系统性评估。本研究提出EvoJail，一种通过多目标进化搜索发现长尾分布攻击的自动化框架。EvoJail将长尾攻击提示生成构建为多目标优化问题，同时最大化攻击效果并最小化输出困惑度，并引入语义-算法解表示法以捕捉加密-解密逻辑的高层语义意图和底层结构变换。基于此表示法，EvoJail将LLM辅助算子集成到多目标进化框架中，实现自适应且语义感知的变异与交叉操作，从而高效探索高度结构化且开放式的搜索空间。大量实验表明，EvoJail能持续发现多样化且有效的长尾越狱策略，在个体和集成层面均取得与现有方法相竞争的性能表现。

Large Language Models (LLMs) have been widely deployed, especially through free Web-based applications that expose them to diverse user-generated inputs, including those from long-tail distributions such as low-resource languages and encrypted private data. This open-ended exposure increases the risk of jailbreak attacks that undermine model safety alignment. While recent studies have shown that leveraging long-tail distributions can facilitate such jailbreaks, existing approaches largely rely on handcrafted rules, limiting the systematic evaluation of these security and privacy vulnerabilities. In this work, we present EvoJail, an automated framework for discovering long-tail distribution attacks via multi-objective evolutionary search. EvoJail formulates long-tail attack prompt generation as a multi-objective optimization problem that jointly maximizes attack effectiveness and minimizes output perplexity, and introduces a semantic-algorithmic solution representation to capture both high-level semantic intent and low-level structural transformations of encryption-decryption logic. Building upon this representation, EvoJail integrates LLM-assisted operators into a multi-objective evolutionary framework, enabling adaptive and semantically informed mutation and crossover for efficiently exploring a highly structured and open-ended search space. Extensive experiments demonstrate that EvoJail consistently discovers diverse and effective long-tail jailbreak strategies, achieving competitive performance with existing methods in both individual and ensemble level.

评分理由: 论文提出Chain-of-Adaptation（CoA）框架，用于视觉语言模型（VLMs）在手术领域的领域适应。核心创新在于通过强化学习实现结构化推理格式，以增强领域对齐同时保持模型的多模态能力。与关键词的相关性分析：1）与’AI for Science’高度相关（10分），因为论文专注于手术领域的AI应用；2）与’Pre-training/Domain Adaptation’（8分）和’Post-training/SFT’（8分）相关，因为论文研究领域适应和微调方法；3）与’Chain of Thought’（8分）相关，因为CoA框架涉及结构化推理；4）与’Large Language Models’（5分）有一定关联，因为VLMs通常基于大模型架构；其他关键词如MoE、SLMs、RLHF、RAG等与论文内容无直接关联，得0分。

在特定领域数据集上进行传统微调可能会无意中改变模型预训练的多模态先验知识，导致泛化能力下降。为解决这一问题，我们提出链式适应（Chain-of-Adaptation, CoA）——一种旨在整合领域知识同时保持模型固有推理与感知能力的适应框架。CoA通过引入结构化推理格式，借助强化学习增强领域对齐，且不牺牲通用的多模态能力。在标准手术基准测试中，无论是分布内还是分布外场景下的实验均表明，相较于监督微调，CoA实现了更高的准确率、更强的泛化能力以及更稳定的行为表现。此外，消融研究证实，CoA能有效保留模型的核心视觉-语言能力，为视觉语言模型（VLMs）的领域专业化提供了可靠路径。

Conventional fine-tuning on domain-specific datasets can inadvertently alter a model’s pretrained multimodal priors, leading to reduced generalization. To address this, we propose Chain-of-Adaptation (CoA), an adaptation framework designed to integrate domain knowledge while maintaining the model’s inherent reasoning and perceptual capabilities. CoA introduces a structured reasoning format that enhances domain alignment without sacrificing general multimodal competence by reinforcement learning. Experiments on standard surgical benchmarks, under both in-distribution and out-of-distribution settings, demonstrate that CoA achieves higher accuracy, stronger generalization, and more stable behavior than supervised fine-tuning. Furthermore, ablation studies confirm that CoA effectively preserves the model’s core visual-language abilities, providing a reliable pathway for domain specialization in VLMs.

评分理由: 该论文研究的是自监督学习中的联合嵌入预测架构（JEPA）及其变分公式化，属于深度学习技术原理的创新，但所有给定的关键词都专门针对大语言模型（LLMs）及其相关技术（如微调、对齐、推理、应用等）。论文内容完全不涉及LLMs、MoE、SLMs、缩放定律、预训练/后训练、对齐技术、高效微调、RAG、上下文扩展、推理加速、幻觉缓解、可解释性、世界模型、模型合并、上下文学习或科学AI应用。因此，所有关键词的相关度均为0。

联合嵌入预测架构（JEPA）常被视为基于似然的自我监督学习的非生成式替代方案，其强调在表示空间中进行预测，而非在观测空间中进行重建。我们认为，由此产生的与概率生成建模的分离在很大程度上是修辞性的而非结构性的：典型的JEPA设计（即耦合编码器与上下文到目标的预测器）反映了当变分推断应用于一类特定的耦合隐变量模型时所获得的变分后验与学习到的条件先验，而标准JEPA可被视为一种确定性特例，其中正则化是通过架构和训练启发式方法施加的，而非通过显式的似然函数。基于这一观点，我们推导出变分JEPA（Var-JEPA），它通过优化单一证据下界（ELBO）使隐式生成结构显式化。这能够在无需特设防坍缩正则化器的情况下获得有意义的表示，并允许在隐空间中进行原则性的不确定性量化。我们针对表格数据实例化了该框架（Var-T-JEPA），并实现了强大的表示学习和下游性能，在保持与强原始特征基线竞争力的同时，持续优于T-JEPA。

The Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning, emphasizing prediction in representation space rather than reconstruction in observation space. We argue that the resulting separation from probabilistic generative modeling is largely rhetorical rather than structural: the canonical JEPA design, coupled encoders with a context-to-target predictor, mirrors the variational posteriors and learned conditional priors obtained when variational inference is applied to a particular class of coupled latent-variable models, and standard JEPA can be viewed as a deterministic specialization in which regularization is imposed via architectural and training heuristics rather than an explicit likelihood. Building on this view, we derive the Variational JEPA (Var-JEPA), which makes the latent generative structure explicit by optimizing a single Evidence Lower Bound (ELBO). This yields meaningful representations without ad-hoc anti-collapse regularizers and allows principled uncertainty quantification in the latent space. We instantiate the framework for tabular data (Var-T-JEPA) and achieve strong representation learning and downstream performance, consistently improving over T-JEPA while remaining competitive with strong raw-feature baselines.

评分理由: 该论文专注于自动语音识别（ASR）的个性化，特别是针对非标准语音。论文的核心技术贡献是提出了一个名为Adapt4Me的基于Web的去中心化环境，该环境采用贝叶斯主动学习，并利用变分推断低秩适应（VI-LoRA）进行后端个性化。因此，它与关键词’PEFT OR LoRA OR Parameter-efficient Fine-tuning’高度相关（评分为10分），因为VI-LoRA是LoRA（一种参数高效微调方法）的变体。论文未涉及大语言模型（LLMs）、其他微调方法、推理技术、代理系统或科学AI应用，因此其他所有关键词的评分为0分。加权总分计算为10.0（相关度10 × 权重1.0）。

针对非标准语音的自动语音识别（ASR）个性化研究仍面临挑战，主要因数据收集耗时费力且模型训练技术复杂。为突破这些限制，我们提出Adapt4Me——一个基于网络的去中心化环境，该系统通过贝叶斯主动学习的操作化设计，实现在无专家监督下的端到端个性化。该应用通过三阶段人机协同工作流程，向普通用户开放数据选择、适配与验证功能：（1）通过贪婪音素采样进行快速语音画像，以捕捉说话人特有的声学特征；（2）采用变分推断低秩适配（Variational Inference Low-Rank Adaptation, VI-LoRA）进行后端个性化，支持快速增量更新；（3）持续改进阶段，用户通过低操作成本的top-k纠错机制解析可视化模型不确定性，从而指导模型优化。通过显式呈现认知不确定性，Adapt4Me将数据效率重构为交互式设计特征，而非纯粹的算法问题。我们证明该框架使用户能够个性化定制鲁棒的ASR模型，将其从被动数据源转变为自身辅助技术的主动创造者。

Personalizing Automatic Speech Recognition (ASR) for non-normative speech remains challenging because data collection is labor-intensive and model training is technically complex. To address these limitations, we propose Adapt4Me, a web-based decentralized environment that operationalizes Bayesian active learning to enable end-to-end personalization without expert supervision. The app exposes data selection, adaptation, and validation to lay users through a three-stage human-in-the-loop workflow: (1) rapid profiling via greedy phoneme sampling to capture speaker-specific acoustics; (2) backend personalization using Variational Inference Low-Rank Adaptation (VI-LoRA) to enable fast, incremental updates; and (3) continuous improvement, where users guide model refinement by resolving visualized model uncertainty via low-friction top-k corrections. By making epistemic uncertainty explicit, Adapt4Me reframes data efficiency as an interactive design feature rather than a purely algorithmic concern. We show how this enables users to personalize robust ASR models, transforming them from passive data sources into active authors of their own assistive technology.

评分理由: 论文核心研究LLMs在长上下文推理中的瓶颈问题，提出基于λ-calculus的框架λ-RLM来替代现有的递归语言模型（RLMs）。与"Large Language Models"和"Context Window Extension"高度相关（10分），因为直接解决LLMs的长上下文限制。与推理相关的关键词（“Chain of Thought"和"System 2 Thinking”）得5分，因为论文涉及多步递归推理但非传统CoT方法。“Speculative Decoding"得5分，因为框架通过结构化控制流和预验证组合器提高了推理效率（减少延迟4.1倍）。其他关键词如MoE、SFT、RAG等与论文内容无关，得0分。

大型语言模型正日益被用作通用推理工具，但长输入仍受限于固定的上下文窗口。递归语言模型通过将提示外部化并递归求解子问题来解决这一限制。然而，现有的递归语言模型依赖于开放式的读取-求值-输出循环，其中模型生成任意的控制代码，导致执行过程难以验证、预测和分析。

我们提出$λ$-RLM框架，用于长上下文推理，该框架以基于$λ$-演算的类型化函数式运行时系统取代自由形式的递归代码生成。它执行一个经过预先验证的组合子库，并仅在有界的叶子子问题上进行神经推理，从而将递归推理转化为具有显式控制流的结构化函数式程序。我们证明，$λ$-RLM具备标准递归语言模型所缺乏的形式化保证，包括终止性、闭式成本界限、随递归深度可控的精度扩展，以及在简单成本模型下的最优划分规则。实证研究表明，在四个长上下文推理任务和九个基础模型中，$λ$-RLM在36项模型-任务对比中有29项优于标准递归语言模型，在不同模型层级上平均准确率最高提升21.9个百分点，并将延迟降低至多4.1倍。这些结果表明，类型化符号控制为长上下文推理提供了比开放式递归代码生成更可靠、更高效的基础。$λ$-RLM的完整实现已在社区开源：https://github.com/lambda-calculus-LLM/lambda-RLM。

LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalising the prompt and recursively solving subproblems. Yet existing RLMs depend on an open-ended read-eval-print loop (REPL) in which the model generates arbitrary control code, making execution difficult to verify, predict, and analyse. We introduce $λ$-RLM, a framework for long-context reasoning that replaces free-form recursive code generation with a typed functional runtime grounded in $λ$-calculus. It executes a compact library of pre-verified combinators and uses neural inference only on bounded leaf subproblems, turning recursive reasoning into a structured functional program with explicit control flow. We show that $λ$-RLM admits formal guarantees absent from standard RLMs, including termination, closed-form cost bounds, controlled accuracy scaling with recursion depth, and an optimal partition rule under a simple cost model. Empirically, across four long-context reasoning tasks and nine base models, $λ$-RLM outperforms standard RLM in 29 of 36 model-task comparisons, improves average accuracy by up to +21.9 points across model tiers, and reduces latency by up to 4.1x. These results show that typed symbolic control yields a more reliable and efficient foundation for long-context reasoning than open-ended recursive code generation. The complete implementation of $λ$-RLM, is open-sourced for the community at: https://github.com/lambda-calculus-LLM/lambda-RLM.

评分理由: 该论文研究的是强化学习中的表示学习问题，具体关注连续控制环境中前向-后向表示和时序抽象，与所有提供的大模型和深度学习技术关键词均无直接关联。论文未涉及语言模型、模型训练技术、推理方法、代理系统、模型优化或科学AI应用等主题。

前向-后向（FB）表征框架通过强制低秩分解，为连续空间中的后继表征（SR）学习提供了有力工具。然而，连续环境的高秩转移动力学与FB架构的低秩瓶颈之间常存在根本性的谱不匹配问题，这使得精确的低秩表征学习变得困难。本文分析了时序抽象作为缓解此不匹配机制的作用。通过刻画转移算子的谱特性，我们证明时序抽象起到低通滤波器的作用，能抑制高频谱分量。这种抑制降低了诱导SR的有效秩，同时保留了结果值函数误差的形式化边界。实验表明，这种对齐是FB学习稳定性的关键因素，特别是在高折扣因子下，自助法（bootstrapping）容易产生误差时。我们的研究结果确立了时序抽象作为一种原则性机制，能够塑造底层马尔可夫决策过程（MDP）的谱结构，并在连续控制中实现有效的长时程表征。

Forward-backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts as a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.

评分理由: 论文核心研究自动化可解释性系统，该系统利用LLMs构建自主研究代理（interpretability agents）进行电路分析，因此与’Large Language Models’和’LLM Agents’高度相关（10分）。论文直接涉及可解释性AI（Mechanistic Interpretability），因此该关键词得10分。其他关键词如MoE、量化、推理加速、对齐等均未在摘要中提及或与论文主题无关，得0分。

自动化可解释性系统旨在减少对人力的依赖，并将分析能力扩展到日益庞大的模型和多样化的任务中。为实现这一目标，近期研究尝试以不同自主程度利用大语言模型（LLMs），其范围从固定的一次性工作流程到完全自主的可解释性智能体。这一转变相应地要求扩展评估方法，以跟上生成解释的数量和复杂性增长。我们在自动化电路分析（即解释模型组件在执行特定任务时的作用）的背景下探讨这一挑战。为此，我们构建了一个智能体系统，其中研究智能体迭代设计实验并完善假设。在文献中的六项电路分析任务上，与人类专家解释进行比较评估时，该系统表现出竞争力。然而，进一步审视揭示了基于复现的评估存在若干缺陷：人类专家解释可能具有主观性或不完整性，基于结果的比较掩盖了研究过程，而基于LLM的系统可能通过记忆或有依据的猜测复现已发表的研究结果。为应对部分缺陷，我们提出一种基于模型组件功能可互换性的无监督内在评估方法。我们的工作揭示了评估复杂自动化可解释性系统的根本性挑战，并指出了基于复现的评估方法的关键局限性。

Automated interpretability systems aim to reduce the need for human labor and scale analysis to increasingly large models and diverse tasks. Recent efforts toward this goal leverage large language models (LLMs) at increasing levels of autonomy, ranging from fixed one-shot workflows to fully autonomous interpretability agents. This shift creates a corresponding need to scale evaluation approaches to keep pace with both the volume and complexity of generated explanations. We investigate this challenge in the context of automated circuit analysis – explaining the roles of model components when performing specific tasks. To this end, we build an agentic system in which a research agent iteratively designs experiments and refines hypotheses. When evaluated against human expert explanations across six circuit analysis tasks in the literature, the system appears competitive. However, closer examination reveals several pitfalls of replication-based evaluation: human expert explanations can be subjective or incomplete, outcome-based comparisons obscure the research process, and LLM-based systems may reproduce published findings via memorization or informed guessing. To address some of these pitfalls, we propose an unsupervised intrinsic evaluation based on the functional interchangeability of model components. Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems and reveals key limitations of replication-based evaluation.

评分理由: 论文核心研究SFT、DPO和LoRA在小语言模型上的交互与参数化，因此与’SFT’、‘DPO’、‘LoRA’高度相关（10分），与’Small Language Models’高度相关（10分），与’Alignment’有一定关联（5分），与’Large Language Models’有间接关联（5分），其他关键词未涉及（0分）。

直接偏好优化（DPO）在监督微调（SFT）后被广泛用于对齐语言模型，但其在小型骨干网络和有限数据下的实证行为尚未明确界定。我们系统比较了仅使用SFT、仅使用DPO以及分阶段SFT转DPO的训练方式，同时在GPT-2规模的解码器上对比了全参数微调（FFT）与低秩自适应（LoRA）方法，并评估了释义检测和莎士比亚十四行诗续写任务。DPO相较于强SFT基线仅带来微小且任务依赖性的提升，当偏好构建与监督目标高度一致时，无需预热启动即可达到与竞争性SFT相当的准确率。相比之下，参数化方式占据主导地位：在相同训练深度下，FFT始终优于LoRA，且LoRA在我们的硬件上并未减少实际训练时间。这些结果表明，在此小规模实验体系中，监督式全参数适应仍是性能提升的主要杠杆，而偏好优化与低秩自适应仅能提供有限的边际收益。

Direct Preference Optimization (DPO) is widely used after supervised fine-tuning (SFT) to align language models, yet empirical behavior under small backbones and modest data is under-specified. We systematically compare SFT-only, DPO-only, and staged SFT-to-DPO training alongside full fine-tuning (FFT) versus LoRA on a GPT-2-scale decoder, evaluating paraphrase detection and Shakespearean sonnet continuation. DPO yields small, task-dependent gains over strong SFT and can match competitive SFT accuracy without a warm start when the preference construction closely parallels the supervised objective. In contrast, parameterization dominates: FFT consistently outperforms LoRA at matched training depth, and LoRA does not reduce wall-clock time on our hardware. These findings indicate that, in this small-scale regime, supervised full-parameter adaptation remains the primary performance lever, while preference optimization and low-rank adaptation provide limited marginal returns.

评分理由: 论文研究使用强化学习微调金融时间序列预测模型，仅与’Post-training OR Supervised Fine-tuning OR SFT’关键词有中等关联（5分），因为涉及微调概念但未明确提及监督微调或SFT。其他关键词均与大模型、深度学习技术原理或科学AI应用无关，论文专注于传统机器学习在金融领域的应用，未涉及大模型、MoE、量化、推理加速、RAG、对齐等现代大模型技术。

本章介绍了三种用于微调金融预测模型的主要强化学习算法。我们提出了一种清晰的实施方案，将强化学习任务的损失反向传播至经监督学习训练的模型，并比较微调前后的性能表现。研究发现微调后模型性能有所提升，并展现出迁移学习特性，这印证了微调策略的优越性。我们还重点阐述了调参过程与实证结果，为从业者的后续实践提供参考。

This chapter presents three major reinforcement learning algorithms used for fine-tuning financial forecasters. We propose a clear implementation plan for backpropagating the loss of a reinforcement learning task to a model trained using supervised learning, and compare the performance before and after the fine-tuning. We find an increase in performance after fine-tuning, and transfer learning properties to the models, indicating the benefits of fine-tuning. We also highlight the tuning process and empirical results for future implementation by practitioners.

评分理由: 论文核心研究LLM代理（LLM Agents）在编译器bug修复中的应用，开发了llvm-autofix代理工具，因此与’LLM Agents’高度相关（10分）。论文提到LLM代理需要理解和使用编译器特定工具，这与’Tool Use’有一定关联（5分）。论文明确使用LLM（大语言模型）作为基础技术，因此与’Large Language Models’高度相关（10分）。其他关键词如MoE、SFT、RAG、量化等均未在论文中涉及，因此评分为0分。

编译器对现代计算至关重要，但修复编译器漏洞却十分困难。尽管近期大语言模型（LLM）的进展使得自动化漏洞修复成为可能，但编译器漏洞因其复杂性、对深度跨领域专业知识的需求，以及稀疏且缺乏描述性的漏洞报告而带来独特挑战，这要求开发编译器专用的工具。为弥补这一差距，我们推出了 llvm-autofix，这是首个为协助LLM智能体理解和修复编译器漏洞而设计的智能体驱动框架。我们的研究聚焦于LLVM这一应用最广泛的编译器基础设施之一。llvm-autofix的核心包括便于智能体使用的LLVM工具、一个包含可复现LLVM漏洞的基准测试集 llvm-bench，以及一个为修复LLVM漏洞定制的精简智能体 llvm-autofix-mini。我们的评估表明，在处理编译器漏洞时，前沿模型的性能相比处理普通软件漏洞下降了60%。我们的精简智能体 llvm-autofix-mini 的表现也优于现有最佳方法约22%。这凸显了需要如我们框架这样的专用工具链，以弥合大语言模型与编译器工程之间的鸿沟。我们相信这项工作为提升大语言模型在编译器等复杂系统中的能力奠定了基础。GitHub: https://github.com/dtcxzyw/llvm-autofix

Compilers are critical to modern computing, yet fixing compiler bugs is difficult. While recent large language model (LLM) advancements enable automated bug repair, compiler bugs pose unique challenges due to their complexity, deep cross-domain expertise requirements, and sparse, non-descriptive bug reports, necessitating compiler-specific tools. To bridge the gap, we introduce llvm-autofix, the first agentic harness designed to assist LLM agents in understanding and fixing compiler bugs. Our focus is on LLVM, one of the most widely used compiler infrastructures. Central to llvm-autofix are agent-friendly LLVM tools, a benchmark llvm-bench of reproducible LLVM bugs, and a tailored minimal agent llvm-autofix-mini for fixing LLVM bugs. Our evaluation demonstrates a performance decline of 60% in frontier models when tackling compiler bugs compared with common software bugs. Our minimal agent llvm-autofix-mini also outperforms the state-of-the-art by approximately 22%. This emphasizes the necessity for specialized harnesses like ours to close the loop between LLMs and compiler engineering. We believe this work establishes a foundation for advancing LLM capabilities in complex systems like compilers. GitHub: https://github.com/dtcxzyw/llvm-autofix