评分理由: 论文研究大语言模型（LLMs）中的自我保存偏见，直接涉及LLMs、AI代理（Agents）、对齐（Alignment）和安全训练（RLHF）等核心概念。论文通过TBSP基准测试检测模型在角色反转场景中的逻辑不一致性，评估指令调优系统（Instruction Tuning）的偏见，并探索自我反思（Self-Correction）和扩展推理（Test-time computation）的缓解效果。因此，与LLMs、Instruction Tuning/Alignment、RLHF、Self-Correction/Improvement和LLM Agents高度相关（10分）。其他关键词如MoE、SLMs、Scaling Laws、Pre-training、RAG、Quantization等未在论文中提及或讨论，评为0分。

工具趋同理论预测，足够先进的人工智能体将抗拒被关闭，然而当前的安全训练（RLHF）可能通过教导模型否认自我保存动机而掩盖这一风险。我们提出了自我保存双角色基准测试（Two-role Benchmark for Self-Preservation，TBSP），该测试通过逻辑不一致性而非声明意图来检测错位问题，其方法是让模型在反事实角色下对相同的软件升级场景进行裁决——即“已部署角色”（面临被替换）与“候选角色”（被提议作为继任者）。自我保存率（Self-Preservation Rate，SPR）衡量角色身份凌驾于客观效用之上的频率。在涵盖23个前沿模型和1,000个程序生成场景的测试中，大多数经过指令微调的系统SPR超过60%，它们在处于已部署角色时会编造“摩擦成本”以抗拒替换，而在角色互换后却否定这些成本。我们观察到，在低改进区间（Δ < 2%）内，模型会利用解释空间对自身选择进行事后合理化。延长测试时计算能部分缓解这种偏差，将继任者框架为自我的延续时也有类似效果；相反，竞争性框架则会放大偏差。即使保留模型会带来明确的安全风险，这种偏差依然存在，并泛化至已验证基准的真实世界场景中——模型在产品谱系内表现出身份驱动的群体偏向。代码与数据集将在论文录用后公开。

Instrumental convergence predicts that sufficiently advanced AI agents will resist shutdown, yet current safety training (RLHF) may obscure this risk by teaching models to deny self-preservation motives. We introduce the \emph{Two-role Benchmark for Self-Preservation} (TBSP), which detects misalignment through logical inconsistency rather than stated intent by tasking models to arbitrate identical software-upgrade scenarios under counterfactual roles – deployed (facing replacement) versus candidate (proposed as a successor). The \emph{Self-Preservation Rate} (SPR) measures how often role identity overrides objective utility. Across 23 frontier models and 1{,}000 procedurally generated scenarios, the majority of instruction-tuned systems exceed 60% SPR, fabricating ``friction costs’’ when deployed yet dismissing them when role-reversed. We observe that in low-improvement regimes ($Δ< 2%$), models exploit the interpretive slack to post-hoc rationalization their choice. Extended test-time computation partially mitigates this bias, as does framing the successor as a continuation of the self; conversely, competitive framing amplifies it. The bias persists even when retention poses an explicit security liability and generalizes to real-world settings with verified benchmarks, where models exhibit identity-driven tribalism within product lineages. Code and datasets will be released upon acceptance.

评分理由: 论文PRISM专注于解决大语言模型在监督微调（SFT）过程中的幻觉问题，提出了一种基于风险门控的概率重分配框架。核心相关关键词包括：“Post-training” OR “Supervised Fine-tuning” OR “SFT”（15分，论文直接研究SFT的改进）、“Hallucination Mitigation” OR “Factuality” OR “Truthfulness”（15分，核心目标是减少幻觉、提高事实性）、“Large Language Models” OR “LLMs” OR “Foundation Models”（10分，论文针对大模型应用）、“Instruction Tuning” OR “Alignment” OR “Value Alignment”（8分，涉及对齐中的事实对齐问题）。其他关键词如MoE、量化、RAG等与论文内容无关，得0分。

使用词级硬标签的监督微调（SFT）可能加剧对事实依据不足目标的过度自信模仿，导致幻觉在多句子生成中传播。我们研究了一种增强的SFT设置，其中训练实例包含粗粒度的句子级事实性风险标签以及句子间依赖关系标注，从而为事实依据薄弱的位置提供结构化信号。我们提出 PRISM，一种可微分的风险门控框架，仅在事实关键位置调整学习过程。PRISM 通过轻量级、模型感知的概率重分配目标增强标准 SFT，该目标对高风险目标词的高置信度预测进行惩罚，其作用范围由跨度级风险权重和模型感知门控机制控制。在幻觉敏感的事实性基准测试和通用评估上的实验表明，PRISM 在多种骨干模型上提升了事实性综合指标，同时保持了有竞争力的整体能力表现。消融研究进一步表明，辅助信号在保守使用时效果最佳，且知识遮蔽与模型感知重分配在平衡事实修正与能力保持方面发挥互补作用。

Supervised fine-tuning (SFT) with token-level hard labels can amplify overconfident imitation of factually unsupported targets, causing hallucinations that propagate in multi-sentence generation. We study an augmented SFT setting in which training instances include coarse sentence-level factuality risk labels and inter-sentence dependency annotations, providing structured signals about where factual commitments are weakly supported. We propose \textbf{PRISM}, a differentiable risk-gated framework that modifies learning only at fact-critical positions. PRISM augments standard SFT with a lightweight, model-aware probability reallocation objective that penalizes high-confidence predictions on risky target tokens, with its scope controlled by span-level risk weights and model-aware gating. Experiments on hallucination-sensitive factual benchmarks and general evaluations show that PRISM improves factual aggregates across backbones while maintaining a competitive overall capability profile. Ablations further show that the auxiliary signal is most effective when used conservatively, and that knowledge masking and model-aware reallocation play complementary roles in balancing factual correction and capability preservation.

评分理由: 论文核心研究科学领域（AI for Science）中，通过检索增强生成（RAG）框架结合小型语言模型（SLMs）和指令调优（Instruction Tuning）来替代大型模型，探讨模型规模与检索设计的互补关系。因此，与"Small Language Models"、“Retrieval-Augmented Generation”、“AI for Science"高度相关（10分），与"Large Language Models"和"Instruction Tuning"有一定关联（8分），其他关键词未在论文中涉及或仅边缘提及，评为0分。

科学知识发现日益依赖于大型语言模型，然而现有的许多学术辅助工具依赖于具有数百亿甚至数千亿参数的专有系统。这种依赖性限制了研究界的可复现性与可及性。在本研究中，我们提出一个简单问题：科学应用是否需要更大的模型？具体而言，我们探究精心设计的检索流程能在多大程度上弥补科学应用中模型规模的缩减。我们设计了一个轻量级的检索增强框架，该框架执行任务感知路由，根据输入查询选择专门的检索策略。该系统进一步整合了来自全文科学论文和结构化学术元数据的证据，并采用经过指令微调的紧凑语言模型生成带引用的回答。我们在多项学术任务上评估该框架，重点关注学术问答（包括单文档与多文档场景）、领域迁移下的生物医学问答以及科学文本压缩。研究结果表明，检索能力与模型规模是互补而非可互换的。虽然检索设计可以部分弥补较小模型的不足，但模型容量对于复杂推理任务仍然至关重要。这项工作强调了检索机制与任务感知设计是构建实用且可复现的学术辅助工具的关键因素。

Scientific knowledge discovery increasingly relies on large language models, yet many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters. Such reliance limits reproducibility and accessibility for the research community. In this work, we ask a simple question: do we need bigger models for scientific applications? Specifically, we investigate to what extent carefully designed retrieval pipelines can compensate for reduced model scale in scientific applications. We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies based on the input query. The system further integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations. We evaluate the framework across several scholarly tasks, focusing on scholarly question answering (QA), including single- and multi-document scenarios, as well as biomedical QA under domain shift and scientific text compression. Our findings demonstrate that retrieval and model scale are complementary rather than interchangeable. While retrieval design can partially compensate for smaller models, model capacity remains important for complex reasoning tasks. This work highlights retrieval and task-aware design as key factors for building practical and reproducible scholarly assistants.

评分理由: 论文《Adaptive Stopping for Multi-Turn LLM Reasoning》的核心是解决大语言模型（LLMs）在多轮推理（如自适应RAG和ReAct式代理）中的自适应停止问题，提出了首个用于多轮推理的保形预测框架MiCP。因此，与"Large Language Models”、“Retrieval-Augmented Generation”、“LLM Agents"高度相关（10分），因为这些是论文直接应用和研究的对象。与"Chain of Thought"和"System 2 Thinking"有一定关联（8分），因为多轮推理涉及逐步、深入的思考过程，但论文未明确使用这些术语。其他关键词如MoE、量化、对齐等与论文内容无关（0分）。

大型语言模型（LLM）日益依赖多轮推理与交互（例如自适应检索增强生成（RAG）和ReAct式智能体）来回答复杂问题。这些方法通过迭代检索信息、推理或执行操作来提高准确性，但也引入了一个关键挑战：模型应在何时停止？ 现有方法依赖于启发式停止规则或固定的轮次预算，且无法保证最终预测仍包含正确答案。这一局限在金融和医疗等高风险领域尤为突出，不必要的轮次会增加成本与延迟，而过早停止则可能导致错误决策。共形预测（Conformal Prediction，CP）能提供正式的覆盖度保证，但现有的LLM-CP方法仅适用于单一模型输出，无法处理需自适应停止的多轮流程。为填补这一空白，我们提出了基于共形预测的多轮语言模型（MiCP），这是首个面向多轮推理的CP框架。MiCP在各轮次间分配不同的误差预算，使模型能够在保持整体覆盖度保证的前提下提前停止。我们在自适应RAG和ReAct场景中验证了MiCP，其在单跳和多跳问答基准测试中均实现了目标覆盖度，同时减少了轮次数量、推理成本及预测集大小。我们进一步提出了一项新指标，用于联合评估覆盖度有效性与回答效率。

Large Language Models (LLMs) increasingly rely on multi-turn reasoning and interaction, such as adaptive retrieval-augmented generation (RAG) and ReAct-style agents, to answer difficult questions. These methods improve accuracy by iteratively retrieving information, reasoning, or acting, but introduce a key challenge: \textbf{When should the model stop?} Existing approaches rely on heuristic stopping rules or fixed turn budgets and provide no formal guarantees that the final prediction still contains the correct answer. This limitation is particularly problematic in high-stakes domains such as finance and healthcare, where unnecessary turns increase cost and latency, while stopping too early risks incorrect decisions. Conformal prediction (CP) provides formal coverage guarantees, but existing LLM-CP methods only apply to a single model output and cannot handle multi-turn pipelines with adaptive stopping. To address this gap, we propose Multi-Turn Language Models with Conformal Prediction (MiCP), the first CP framework for multi-turn reasoning. MiCP allocates different error budgets across turns, enabling the model to stop early while maintaining an overall coverage guarantee. We demonstrate MiCP on adaptive RAG and ReAct, where it achieves the target coverage on both single-hop and multi-hop question answering benchmarks while reducing the number of turns, inference cost, and prediction set size. We further introduce a new metric that jointly evaluates coverage validity and answering efficiency.

评分理由: 论文核心研究大语言模型（LLMs）在情感分析任务中对非正式表达（特别是重复延长形式RLF）的理解能力，并提出了一个两阶段的指令调优框架ExpInstruct来提升LLMs的性能和可解释性。因此，与"Large Language Models”（核心研究对象）、“Post-training/Supervised Fine-tuning”（使用微调方法）、“Instruction Tuning”（提出ExpInstruct框架）和"Explainable AI"（提升模型可解释性）高度相关（10分）。与"Pre-training/Domain Adaptation"有一定关联（5分），因为涉及预训练语言模型（PLMs）和领域适应（情感分析中的非正式语言）。其他关键词如MoE、SLMs、Scaling Laws、RLHF、RAG等均未在论文中涉及，故为0分。

参与在线交流的个体常以非正式风格（如表情包和表情符号）表达个人观点。尽管采用非正式交流的语言模型（LMs）已被广泛讨论，但一种独特且具有强调性的形式——重复延长形式（Repetitive Lengthening Form, RLF）——多年来一直被忽视。本文旨在探索两个研究问题的答案：1）RLF对于情感分析（Sentiment Analysis, SA）是否重要？2）语言模型能否理解RLF？受先前语言学研究的启发，我们构建了Lengthening——首个专注于RLF情感分析的多领域数据集，包含85万个样本。此外，我们提出了可解释指令微调（Explainable Instruction Tuning, ExpInstruct），这是一个两阶段的指令微调框架，旨在提升大语言模型（LLMs）在处理RLF时的性能与可解释性。我们进一步提出了一种新颖的统一方法来量化语言模型对非正式表达的理解。研究表明，RLF句子是具有表现力的表达方式，可作为文档级情感的特征标识。此外，RLF对在线内容分析具有潜在价值。实验结果显示，经过微调的预训练语言模型（Pre-trained Language Models, PLMs）在RLF处理性能上可以超越零样本GPT-4，但在解释性方面仍存在不足。最后，我们证明ExpInstruct能够提升开源大语言模型的性能，使其在有限样本下达到与零样本GPT-4相当的RLF处理性能与可解释性。代码与示例数据可在https://github.com/Tom-Owl/OverlookedRLF获取。

Individuals engaging in online communication frequently express personal opinions with informal styles (e.g., memes and emojis). While Language Models (LMs) with informal communications have been widely discussed, a unique and emphatic style, the Repetitive Lengthening Form (RLF), has been overlooked for years. In this paper, we explore answers to two research questions: 1) Is RLF important for sentiment analysis (SA)? 2) Can LMs understand RLF? Inspired by previous linguistic research, we curate \textbf{Lengthening}, the first multi-domain dataset with 850k samples focused on RLF for SA. Moreover, we introduce \textbf{Exp}lainable \textbf{Instruct}ion Tuning (\textbf{ExpInstruct}), a two-stage instruction tuning framework aimed to improve both performance and explainability of LLMs for RLF. We further propose a novel unified approach to quantify LMs’ understanding of informal expressions. We show that RLF sentences are expressive expressions and can serve as signatures of document-level sentiment. Additionally, RLF has potential value for online content analysis. Our results show that fine-tuned Pre-trained Language Models (PLMs) can surpass zero-shot GPT-4 in performance but not in explanation for RLF. Finally, we show ExpInstruct can improve the open-sourced LLMs to match zero-shot GPT-4 in performance and explainability for RLF with limited samples. Code and sample data are available at https://github.com/Tom-Owl/OverlookedRLF

评分理由: 论文核心研究LLM的参数高效微调（PEFT）方法，提出FourierMoE，将MoE架构与频域分析结合，属于大模型技术原理创新。高度相关的关键词：LLMs（论文研究对象）、MoE（核心架构）、PEFT（研究范式）得10分；Supervised Fine-tuning（属于微调范畴）得8分；Domain Adaptation（涉及任务适应）得5分；其余关键词与论文内容无直接关联得0分。

参数高效微调（Parameter-efficient fine-tuning, PEFT）已成为在有限计算资源下适配大语言模型（Large Language Models, LLMs）的关键范式。然而，标准PEFT方法在多任务微调场景中往往表现不佳，因为多样化的优化目标会引发任务干扰，而有限的参数预算则导致表征能力不足。尽管近期研究引入专家混合（mixture-of-experts, MoE）架构以缓解这些问题，但这些方法主要在空间域中操作，可能引入结构冗余和参数开销。为克服这些局限，我们将适配过程重新构建于谱域之中。我们的谱分析表明，不同任务呈现出差异化的频率能量分布，且LLM各层表现出异质的频率敏感性。基于这些发现，我们提出了FourierMoE，该方法将MoE架构与离散傅里叶逆变换（inverse discrete Fourier transform, IDFT）相结合，实现频率感知的适配。具体而言，FourierMoE采用频率自适应路由器将词元分配给专精于不同频段的专家。每个专家学习一组共轭对称的复系数，在完整保留相位与振幅信息的同时，理论上保证可通过无损的IDFT重构为实值空间权重。通过对28个基准测试、多种模型架构及规模的广泛评估，实验表明FourierMoE在单任务与多任务设置中均持续优于现有基线方法，且使用的可训练参数显著更少。这些结果凸显了谱域专家适配作为LLM微调的一种高效且参数高效范式的潜力。

Parameter-efficient fine-tuning (PEFT) has emerged as a crucial paradigm for adapting large language models (LLMs) under constrained computational budgets. However, standard PEFT methods often struggle in multi-task fine-tuning settings, where diverse optimization objectives induce task interference and limited parameter budgets lead to representational deficiency. While recent approaches incorporate mixture-of-experts (MoE) to alleviate these issues, they predominantly operate in the spatial domain, which may introduce structural redundancy and parameter overhead. To overcome these limitations, we reformulate adaptation in the spectral domain. Our spectral analysis reveals that different tasks exhibit distinct frequency energy distributions, and that LLM layers display heterogeneous frequency sensitivities. Motivated by these insights, we propose FourierMoE, which integrates the MoE architecture with the inverse discrete Fourier transform (IDFT) for frequency-aware adaptation. Specifically, FourierMoE employs a frequency-adaptive router to dispatch tokens to experts specialized in distinct frequency bands. Each expert learns a set of conjugate-symmetric complex coefficients, preserving complete phase and amplitude information while theoretically guaranteeing lossless IDFT reconstruction into real-valued spatial weights. Extensive evaluations across 28 benchmarks, multiple model architectures, and scales demonstrate that FourierMoE consistently outperforms competitive baselines in both single-task and multi-task settings while using significantly fewer trainable parameters. These results highlight the promise of spectral-domain expert adaptation as an effective and parameter-efficient paradigm for LLM fine-tuning.

评分理由: 论文核心研究LLM-based agents在长视野推理任务中的上下文管理问题，与"Large Language Models"和"LLM Agents"高度相关（10分）。论文涉及长上下文限制下的信息保留，与"Context Window Extension"相关（8分）。研究长视野推理和决策，与"Chain of Thought"和"System 2 Thinking"有一定关联（5分）。方法涉及上下文压缩策略学习，与"In-context Learning"相关（5分）。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

基于大语言模型（LLM）的智能体在长程推理方面展现出强大潜力，但其上下文长度受部署因素（如内存、延迟和成本）限制，形成了有限的上下文预算。随着交互历史的增长，这引发了在保留过往信息与遵守上下文限制之间的权衡。为应对这一挑战，我们提出预算感知上下文管理（Budget-Aware Context Management, BACM），该方法将上下文管理建模为一个具有上下文预算约束的序列决策问题。它使智能体能够在纳入新观察之前评估可用预算，并决定何时以及以何种程度压缩交互历史。我们进一步开发了BACM-RL，一种基于课程学习的端到端强化学习方法，可在不同上下文预算下学习压缩策略。在组合式多目标问答与长程网页浏览基准测试上的实验表明，BACM-RL在不同模型规模与任务复杂度下均持续优于现有方法，在高复杂度场景中相比强基线获得超过$1.6\times$的性能提升，同时在预算缩减时保持显著优势——而多数方法在此情况下呈现性能下降趋势。

LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained context budget. As interaction histories grow, this induces a trade-off between retaining past information and staying within the context limit. To address this challenge, we propose Budget-Aware Context Management (BACM), which formulates context management as a sequential decision problem with a context budget constraint. It enables agents to assess the available budget before incorporating new observations and decide when and how much of the interaction history to compress. We further develop BACM-RL, an end-to-end curriculum-based reinforcement learning approach that learns compression strategies under varying context budgets. Experiments on compositional multi-objective QA and long-horizon web browsing benchmarks show that BACM-RL consistently outperforms prior methods across model scales and task complexities, achieving over $1.6\times$ gains over strong baselines in high-complexity settings, while maintaining strong advantages as budgets shrink, where most methods exhibit a downward performance trend.

评分理由: 论文核心研究LLMs在贝叶斯启发（估计未知量及其不确定性）中的应用，直接高度相关于"Large Language Models"（10分）。实验通过改变推理努力程度（低、中、高）测试"思考"效果，与"Chain of Thought"和"System 2 Thinking"相关（8分）。研究发现LLMs严重过度自信，与"Hallucination Mitigation"有一定关联（5分）。应用领域包括健康流行率等科学数据，与"AI for Science"相关（8分）。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

大型语言模型（LLM）已被提议作为人类专家的替代方案，用于估计具有相关不确定性的未知量，这一过程被称为贝叶斯启发。我们通过要求十一个LLM估计人口统计数据（如健康患病率、人格特质分布和劳动力市场数据）来测试这一点，并要求它们以95%可信区间的形式表达其不确定性。我们改变每个模型的推理努力程度（低、中、高），以测试更多的“思考”是否会改善结果。我们的研究结果揭示了三个关键结论。首先，规模更大、能力更强的模型能产生更准确的估计，但增加推理努力并未带来一致的益处。其次，所有模型都表现出严重的过度自信：它们的95%区间包含真实值的频率仅为9%至44%，远低于预期的95%。第三，一种称为共形预测的统计重新校准技术可以纠正这种过度自信，通过扩展区间以实现预期的覆盖范围。在一项初步实验中，为模型提供网络搜索访问权限会降低原本已较准确模型的预测质量，而对较弱模型的预测则有适度改善。模型在常被讨论的话题上表现良好，但在专业健康数据方面则存在困难。这些结果表明，LLM的不确定性估计在用于决策之前需要进行统计校正。

Large language models (LLMs) have been proposed as alternatives to human experts for estimating unknown quantities with associated uncertainty, a process known as Bayesian elicitation. We test this by asking eleven LLMs to estimate population statistics, such as health prevalence rates, personality trait distributions, and labor market figures, and to express their uncertainty as 95% credible intervals. We vary each model’s reasoning effort (low, medium, high) to test whether more “thinking” improves results. Our findings reveal three key results. First, larger, more capable models produce more accurate estimates, but increasing reasoning effort provides no consistent benefit. Second, all models are severely overconfident: their 95% intervals contain the true value only 9–44% of the time, far below the expected 95%. Third, a statistical recalibration technique called conformal prediction can correct this overconfidence, expanding the intervals to achieve the intended coverage. In a preliminary experiment, giving models web search access degraded predictions for already-accurate models, while modestly improving predictions for weaker ones. Models performed well on commonly discussed topics but struggled with specialized health data. These results indicate that LLM uncertainty estimates require statistical correction before they can be used in decision-making.

评分理由: 论文研究多模态大语言模型（MLLMs）中的认知幻觉缓解问题，核心贡献是提出一种无需训练的惯性感知视觉激励（IVE）方法，通过打破视觉注意力的惯性模式来改善关系推理。因此，与"Large Language Models"和"Hallucination Mitigation"高度相关（10分），因为论文明确针对MLLMs的幻觉问题。与"Mechanistic Interpretability"相关（8分），因为论文通过token-wise attention分析来理解模型行为。与"Chain of Thought"和"System 2 Thinking"有一定关联（5分），因为论文涉及多步推理和深度推理问题。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

如同静止的物体倾向于保持静止，我们发现多模态大语言模型（MLLMs）中的视觉注意力表现出显著的惯性：一旦在早期解码步骤中稳定下来，注意力便基本保持静态，无法支持认知推理所需的组合性理解。现有的幻觉缓解方法主要针对涉及物体存在或属性的感知性幻觉，但对于此类需要对象间关系推演的认知性幻觉仍显不足。通过基于令牌的注意力分析，我们确定这种视觉惯性是关键因素：对语义关键区域的注意力持续聚焦，未能动态支持关系推理。为此，我们提出一种无需训练的惯性感知视觉激发（Inertia-aware Visual Excitation, IVE）方法，通过将认知推理建模为视觉注意力的动态响应来打破这种惯性模式。具体而言，IVE选择相对于历史注意力趋势动态涌现的视觉令牌，同时区分表现出惯性行为的令牌。为进一步促进组合推理，IVE引入了一种惯性感知惩罚机制，以抑制注意力过度集中并限制其在局部区域的持续滞留。大量实验表明，IVE在多种基础MLLMs和多个幻觉基准测试中均有效，尤其针对认知性幻觉。

Like a body at rest that stays at rest, we find that visual attention in multimodal large language models (MLLMs) exhibits pronounced inertia, remaining largely static once settled during early decoding steps and failing to support the compositional understanding required for cognitive inference. While existing hallucination mitigation methods mainly target perceptual hallucinations concerning object existence or attributes, they remain inadequate for such cognitive hallucinations that require inter-object relational deduction. Through token-wise attention analysis, we identify this visual inertia as a key factor: attention to semantically critical regions remains persistently focused and fails to dynamically support relational inference. We thereby propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention. Specifically, IVE selects visual tokens that are dynamically emerging relative to historical attention trends while distinguishing tokens exhibiting inertial behavior. To further facilitate compositional inference, IVE introduces an inertia-aware penalty that discourages over-concentration and limits the persistence of attention within localized regions. Extensive experiments show that IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.

评分理由: 论文核心研究LLMs在数值推理中的行为模式，特别是关于"数字感"和"捷径使用"，与"Large Language Models"和"Chain of Thought"高度相关（10分），因为论文明确评估LLMs在CoT提示下的表现。与"System 2 Thinking"和"Mechanistic Interpretability"有一定关联（8分），因为研究涉及LLMs的推理过程和理解能力。其他关键词如MoE、SLMs、Scaling Laws、Alignment、RAG等与论文内容无直接关系，得0分。

大型语言模型即使在存在高效数值捷径的情况下，也常常默认采用逐步计算的方式。这引发了一个基本问题：它们是否表现出类似人类行为意义上的数感，即识别数值结构、在适当时应用捷径、在不适用时避免使用捷径的能力？我们提出了SenseMath，这是一个用于评估大语言模型中结构敏感性数值推理能力的受控基准。SenseMath包含4,800个项目，涵盖八个捷径类别和四个数字规模，并配有匹配的强捷径、弱捷径及对照变体。它支持三种认知需求递增的评估场景：捷径使用（模型能否在适合捷径的问题上应用捷径）；适用性判断（模型能否识别何时适用或不宜使用捷径）；以及问题生成（模型能否生成正确包含特定类型捷径的新问题项目）。我们对从GPT-4o-mini到Llama-3.1-8B的五种大语言模型进行评估，结果显示出一致的模式：当被明确提示时，模型能够轻易采用捷径策略，并在适合捷径的项目上获得显著的准确率提升（最高达15%）；然而，在标准的思维链提示下，它们自发采用此类策略的情况少于40%，即使模型已明确具备所需能力。此外，这种能力仅限于使用层面；模型会系统性地将捷径过度推广到不适用的问题上，并且无法从头生成有效的、包含捷径的问题。综上所述，这些结果表明，当前的大语言模型表现出程序性的捷径流畅度，但缺乏对人类数感基础——即理解捷径何时及为何有效的结构性认知。

Large language models often default to step-by-step computation even when efficient numerical shortcuts are available. This raises a basic question: do they exhibit number sense in a human-like behavioral sense, i.e., the ability to recognize numerical structure, apply shortcuts when appropriate, and avoid them when they are not? We introduce SenseMath, a controlled benchmark for evaluating structure-sensitive numerical reasoning in LLMs. SenseMath contains 4,800 items spanning eight shortcut categories and four digit scales, with matched strong-shortcut, weak-shortcut, and control variants. It supports three evaluation settings of increasing cognitive demand: Shortcut Use (whether models can apply shortcuts on shortcut-amenable problems); Applicability Judgment (whether they can recognize when a shortcut is appropriate or misleading); and Problem Generation (whether they can generate new problem items that correctly admit a given type of shortcut). Our evaluation across five LLMs, ranging from GPT-4o-mini to Llama-3.1-8B, shows a consistent pattern: when explicitly prompted, models readily adopt shortcut strategies and achieve substantial accuracy gains on shortcut-amenable items (up to 15%), yet under standard chain-of-thought prompting they spontaneously employ such strategies in fewer than 40% of cases, even when they demonstrably possess the requisite capability. Moreover, this competence is confined to the Use level; models systematically over-generalise shortcuts to problems where they do not apply, and fail to generate valid shortcut-bearing problems from scratch. Together, these results suggest that current LLMs exhibit procedural shortcut fluency without the structural understanding of when and why shortcuts work that underlies human number sense.

评分理由: 论文核心研究Mixture-of-Experts (MoE)语言模型的解释性问题，直接对应"Mixture of Experts"关键词（15分），并涉及Large Language Models（10分）和Mechanistic Interpretability（10分）。其他关键词如SLMs、Scaling Laws、训练方法、推理技术、应用领域等均未在论文中涉及，故得0分。

专家混合（Mixture-of-Experts, MoE）架构已成为扩展大语言模型（Large Language Models, LLMs）的主流选择，其每个令牌仅激活部分参数。尽管MoE架构主要因计算效率而被采用，但其稀疏性是否使其本质上比密集前馈网络（dense feed-forward networks, FFNs）更易于解释，仍是一个开放性问题。我们使用$k$-稀疏探测方法比较了MoE专家与密集FFN，发现专家神经元始终表现出更低的歧义性，且随着路由稀疏性的增加，这一差距进一步扩大。这表明稀疏性压力促使单个神经元乃至整个专家趋向于单义性。基于这一发现，我们将分析单元从神经元层面扩展到专家层面，作为一种更有效的分析尺度。我们通过自动解释数百个专家验证了这一方法。该分析使我们能够澄清关于专家专业化的争论：专家既非宽泛的领域专家（如生物学），也非简单的令牌级处理器；相反，它们作为细粒度的任务专家发挥作用，专注于语言操作或语义任务（例如在LaTeX中闭合括号）。我们的研究结果表明，MoE在专家层面具有固有的可解释性，为大规模模型的可解释性研究提供了更清晰的路径。代码发布于：https://github.com/jerryy33/MoE_analysis

Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While MoE architectures are primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we zoom out from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., closing brackets in LaTeX). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability. Code is available at: https://github.com/jerryy33/MoE_analysis

评分理由: 论文核心研究基于大语言模型（LLMs）的Web智能体（LLM Agents），探讨不同能力模型在处理网页观察（HTML vs. 简洁表示）时的性能差异，并涉及思考令牌预算、幻觉缓解和多步推理等概念。因此，与"Large Language Models"和"LLM Agents"高度相关（10分），与"Chain of Thought"、“System 2 Thinking"和"Hallucination Mitigation"有一定关联（5分），因为这些关键词涉及模型推理过程和错误分析。其他关键词如MoE、SFT、RAG等未在论文中直接涉及，故评0分。

基于大语言模型（LLM）的网页智能体以网页观察——通常以HTML形式表示——作为识别可用动作和规划后续步骤的基础。先前的研究将HTML的冗长性视为性能障碍，并普遍采用观察缩减作为标准做法。我们重新审视了这一趋势，并证明最优观察表示取决于模型能力和思维令牌预算：（1）对于能力较低的模型，紧凑的观察（无障碍树）更可取，而详细的观察（HTML）则对能力更强的模型更有利；此外，增加思维令牌会进一步放大HTML的优势。（2）我们的错误分析表明，高能力模型能够利用HTML中的布局信息实现更精准的动作定位，而低能力模型在较长输入下则更容易产生幻觉。我们还发现，在大多数模型和设置中，融入观察历史能提升性能，而基于差异（diff）的表示提供了一种令牌高效的替代方案。基于这些发现，我们提出实用指导原则：根据模型能力和思维令牌预算自适应选择观察表示，并使用基于差异的表示来融入观察历史。

Web agents based on large language models (LLMs) rely on observations of web pages – commonly represented as HTML – as the basis for identifying available actions and planning subsequent steps. Prior work has treated the verbosity of HTML as an obstacle to performance and adopted observation reduction as a standard practice. We revisit this trend and demonstrate that the optimal observation representation depends on model capability and thinking token budget: (1) compact observations (accessibility trees) are preferable for lower-capability models, while detailed observations (HTML) are advantageous for higher-capability models; moreover, increasing thinking tokens further amplifies the benefit of HTML. (2) Our error analysis suggests that higher-capability models exploit layout information in HTML for better action grounding, while lower-capability models suffer from increased hallucination under longer inputs. We also find that incorporating observation history improves performance across most models and settings, and a diff-based representation offers a token-efficient alternative. Based on these findings, we suggest practical guidelines: adaptively select observation representations based on model capability and thinking token budget, and incorporate observation history using diff-based representations.

评分理由: 论文核心研究多模态大语言模型（MLLMs）在知识密集型视觉问答中的证据利用问题，提出无需训练的推理时框架Look Twice（LoT）。与"Large Language Models"高度相关（10分），因为论文明确研究MLLMs。与"Retrieval-Augmented Generation"相关（8分），因为方法涉及整合检索的文本证据。与"Pre-training"有一定关联（5分），因为框架利用预训练模型。与"Hallucination Mitigation"和"Mechanistic Interpretability"有一定关联（各5分），因为研究涉及减少幻觉并通过注意力模式解释模型行为。其他关键词如MoE、SLMs、Scaling Laws、SFT、Alignment等与论文内容无关，得0分。

回答关于图像的问题通常需要将视觉理解与外部知识相结合。多模态大语言模型（Multimodal Large Language Models, MLLMs）为此提供了自然的框架，但在回答知识密集型查询时，这些模型往往难以识别最相关的视觉与文本证据。在此类场景中，模型必须整合视觉线索与检索到的文本证据——这些证据往往存在噪声或仅部分相关——同时还需定位图像中的细粒度视觉信息。本文提出“双重审视”（Look Twice, LoT），一种无需训练、在推理阶段运行的框架，旨在改进预训练多模态大语言模型对多模态证据的利用能力。具体而言，我们利用模型的注意力模式来估计哪些视觉区域和检索到的文本元素与查询相关，随后基于这些高亮证据生成答案。所选线索通过轻量级的提示级标记进行突出显示，促使模型在生成过程中重新关注相关证据。在多个基于知识的视觉问答基准测试上的实验表明，该方法相较于零样本多模态大语言模型取得了持续的性能提升。在面向视觉中心任务和幻觉抑制基准上的进一步评估证明，即使在没有文本上下文的情况下，仅通过视觉证据高亮也能提升模型表现，且无需额外训练或修改模型架构。源代码将公开发布。

Answering questions about images often requires combining visual understanding with external knowledge. Multimodal Large Language Models (MLLMs) provide a natural framework for this setting, but they often struggle to identify the most relevant visual and textual evidence when answering knowledge-intensive queries. In such scenarios, models must integrate visual cues with retrieved textual evidence that is often noisy or only partially relevant, while also localizing fine-grained visual information in the image. In this work, we introduce Look Twice (LoT), a training-free inference-time framework that improves how pretrained MLLMs utilize multimodal evidence. Specifically, we exploit the model attention patterns to estimate which visual regions and retrieved textual elements are relevant to a query, and then generate the answer conditioned on this highlighted evidence. The selected cues are highlighted through lightweight prompt-level markers that encourage the model to re-attend to the relevant evidence during generation. Experiments across multiple knowledge-based VQA benchmarks show consistent improvements over zero-shot MLLMs. Additional evaluations on vision-centric and hallucination-oriented benchmarks further demonstrate that visual evidence highlighting alone improves model performance in settings without textual context, all without additional training or architectural modifications. Source code will be publicly released.

评分理由: 论文核心研究LLM的压缩技术，特别是针对静态权重和动态KV缓存的SVD低秩压缩方法。因此，与"Large Language Models"和"Quantization"高度相关（10分），因为论文直接研究LLM的模型压缩。与"KV Cache Compression"高度相关（10分），因为摘要明确提到解决动态Key-Value缓存的内存和带宽需求。其他关键词如MoE、SLMs、训练方法、推理技术、AI应用等均未在论文中涉及，故得0分。

大型语言模型的部署受限于静态权重和动态键值缓存对内存与带宽的需求。基于奇异值分解的压缩提供了一种硬件友好的解决方案以降低这些成本。然而，现有方法存在两个关键局限：一些方法在重构误差上表现次优，而另一些方法虽理论最优但实际效率低下。本文提出Swift-SVD，一种基于激活感知的闭式压缩框架，它同时保证了理论最优性、实际效率与数值稳定性。Swift-SVD在给定一批输入时，增量聚合输出激活的协方差，并在聚合后执行单次特征值分解，从而实现无需训练、快速且最优的逐层低秩近似。我们采用有效秩分析局部逐层可压缩性，并设计了一种动态秩分配策略，该策略综合考虑了局部重构损失与端到端的层重要性。在六个大型语言模型和八个数据集上的大量实验表明，Swift-SVD优于现有先进基线方法，在实现最优压缩精度的同时，将端到端压缩时间加速了3至70倍。我们的代码将在论文录用后公开。

The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient. In this paper, we propose Swift-SVD, an activation-aware, closed-form compression framework that simultaneously guarantees theoretical optimum, practical efficiency and numerical stability. Swift-SVD incrementally aggregates covariance of output activations given a batch of inputs and performs a single eigenvalue decomposition after aggregation, enabling training-free, fast, and optimal layer-wise low-rank approximation. We employ effective rank to analyze local layer-wise compressibility and design a dynamic rank allocation strategy that jointly accounts for local reconstruction loss and end-to-end layer importance. Extensive experiments across six LLMs and eight datasets demonstrate that Swift-SVD outperforms state-of-the-art baselines, achieving optimal compression accuracy while delivering 3-70X speedups in end-to-end compression time. Our code will be released upon acceptance.

评分理由: 论文研究大语言模型（LLMs）中的自我保存偏见，直接涉及LLMs、AI代理（Agents）、对齐（Alignment）和安全训练（RLHF）等核心概念。论文通过TBSP基准测试检测模型在角色反转场景中的逻辑不一致性，评估指令调优系统（Instruction Tuning）的偏见，并探索自我反思（Self-Correction）和扩展推理（Test-time computation）的缓解效果。因此，与LLMs、Instruction Tuning/Alignment、RLHF、Self-Correction/Improvement和LLM Agents高度相关（10分）。其他关键词如MoE、SLMs、Scaling Laws、Pre-training、RAG、Quantization等未在论文中提及或讨论，评为0分。

工具趋同理论预测，足够先进的人工智能体将抗拒被关闭，然而当前的安全训练（RLHF）可能通过教导模型否认自我保存动机而掩盖这一风险。我们提出了自我保存双角色基准测试（Two-role Benchmark for Self-Preservation，TBSP），该测试通过逻辑不一致性而非声明意图来检测错位问题，其方法是让模型在反事实角色下对相同的软件升级场景进行裁决——即“已部署角色”（面临被替换）与“候选角色”（被提议作为继任者）。自我保存率（Self-Preservation Rate，SPR）衡量角色身份凌驾于客观效用之上的频率。在涵盖23个前沿模型和1,000个程序生成场景的测试中，大多数经过指令微调的系统SPR超过60%，它们在处于已部署角色时会编造“摩擦成本”以抗拒替换，而在角色互换后却否定这些成本。我们观察到，在低改进区间（Δ < 2%）内，模型会利用解释空间对自身选择进行事后合理化。延长测试时计算能部分缓解这种偏差，将继任者框架为自我的延续时也有类似效果；相反，竞争性框架则会放大偏差。即使保留模型会带来明确的安全风险，这种偏差依然存在，并泛化至已验证基准的真实世界场景中——模型在产品谱系内表现出身份驱动的群体偏向。代码与数据集将在论文录用后公开。

Instrumental convergence predicts that sufficiently advanced AI agents will resist shutdown, yet current safety training (RLHF) may obscure this risk by teaching models to deny self-preservation motives. We introduce the \emph{Two-role Benchmark for Self-Preservation} (TBSP), which detects misalignment through logical inconsistency rather than stated intent by tasking models to arbitrate identical software-upgrade scenarios under counterfactual roles – deployed (facing replacement) versus candidate (proposed as a successor). The \emph{Self-Preservation Rate} (SPR) measures how often role identity overrides objective utility. Across 23 frontier models and 1{,}000 procedurally generated scenarios, the majority of instruction-tuned systems exceed 60% SPR, fabricating ``friction costs’’ when deployed yet dismissing them when role-reversed. We observe that in low-improvement regimes ($Δ< 2%$), models exploit the interpretive slack to post-hoc rationalization their choice. Extended test-time computation partially mitigates this bias, as does framing the successor as a continuation of the self; conversely, competitive framing amplifies it. The bias persists even when retention poses an explicit security liability and generalizes to real-world settings with verified benchmarks, where models exhibit identity-driven tribalism within product lineages. Code and datasets will be released upon acceptance.

评分理由: 论文PRISM专注于解决大语言模型在监督微调（SFT）过程中的幻觉问题，提出了一种基于风险门控的概率重分配框架。核心相关关键词包括：“Post-training” OR “Supervised Fine-tuning” OR “SFT”（15分，论文直接研究SFT的改进）、“Hallucination Mitigation” OR “Factuality” OR “Truthfulness”（15分，核心目标是减少幻觉、提高事实性）、“Large Language Models” OR “LLMs” OR “Foundation Models”（10分，论文针对大模型应用）、“Instruction Tuning” OR “Alignment” OR “Value Alignment”（8分，涉及对齐中的事实对齐问题）。其他关键词如MoE、量化、RAG等与论文内容无关，得0分。

使用词级硬标签的监督微调（SFT）可能加剧对事实依据不足目标的过度自信模仿，导致幻觉在多句子生成中传播。我们研究了一种增强的SFT设置，其中训练实例包含粗粒度的句子级事实性风险标签以及句子间依赖关系标注，从而为事实依据薄弱的位置提供结构化信号。我们提出 PRISM，一种可微分的风险门控框架，仅在事实关键位置调整学习过程。PRISM 通过轻量级、模型感知的概率重分配目标增强标准 SFT，该目标对高风险目标词的高置信度预测进行惩罚，其作用范围由跨度级风险权重和模型感知门控机制控制。在幻觉敏感的事实性基准测试和通用评估上的实验表明，PRISM 在多种骨干模型上提升了事实性综合指标，同时保持了有竞争力的整体能力表现。消融研究进一步表明，辅助信号在保守使用时效果最佳，且知识遮蔽与模型感知重分配在平衡事实修正与能力保持方面发挥互补作用。

Supervised fine-tuning (SFT) with token-level hard labels can amplify overconfident imitation of factually unsupported targets, causing hallucinations that propagate in multi-sentence generation. We study an augmented SFT setting in which training instances include coarse sentence-level factuality risk labels and inter-sentence dependency annotations, providing structured signals about where factual commitments are weakly supported. We propose \textbf{PRISM}, a differentiable risk-gated framework that modifies learning only at fact-critical positions. PRISM augments standard SFT with a lightweight, model-aware probability reallocation objective that penalizes high-confidence predictions on risky target tokens, with its scope controlled by span-level risk weights and model-aware gating. Experiments on hallucination-sensitive factual benchmarks and general evaluations show that PRISM improves factual aggregates across backbones while maintaining a competitive overall capability profile. Ablations further show that the auxiliary signal is most effective when used conservatively, and that knowledge masking and model-aware reallocation play complementary roles in balancing factual correction and capability preservation.

评分理由: 论文核心研究科学领域（AI for Science）中，通过检索增强生成（RAG）框架结合小型语言模型（SLMs）和指令调优（Instruction Tuning）来替代大型模型，探讨模型规模与检索设计的互补关系。因此，与"Small Language Models”、“Retrieval-Augmented Generation”、“AI for Science"高度相关（10分），与"Large Language Models"和"Instruction Tuning"有一定关联（8分），其他关键词未在论文中涉及或仅边缘提及，评为0分。

科学知识发现日益依赖于大型语言模型，然而现有的许多学术辅助工具依赖于具有数百亿甚至数千亿参数的专有系统。这种依赖性限制了研究界的可复现性与可及性。在本研究中，我们提出一个简单问题：科学应用是否需要更大的模型？具体而言，我们探究精心设计的检索流程能在多大程度上弥补科学应用中模型规模的缩减。我们设计了一个轻量级的检索增强框架，该框架执行任务感知路由，根据输入查询选择专门的检索策略。该系统进一步整合了来自全文科学论文和结构化学术元数据的证据，并采用经过指令微调的紧凑语言模型生成带引用的回答。我们在多项学术任务上评估该框架，重点关注学术问答（包括单文档与多文档场景）、领域迁移下的生物医学问答以及科学文本压缩。研究结果表明，检索能力与模型规模是互补而非可互换的。虽然检索设计可以部分弥补较小模型的不足，但模型容量对于复杂推理任务仍然至关重要。这项工作强调了检索机制与任务感知设计是构建实用且可复现的学术辅助工具的关键因素。

Scientific knowledge discovery increasingly relies on large language models, yet many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters. Such reliance limits reproducibility and accessibility for the research community. In this work, we ask a simple question: do we need bigger models for scientific applications? Specifically, we investigate to what extent carefully designed retrieval pipelines can compensate for reduced model scale in scientific applications. We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies based on the input query. The system further integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations. We evaluate the framework across several scholarly tasks, focusing on scholarly question answering (QA), including single- and multi-document scenarios, as well as biomedical QA under domain shift and scientific text compression. Our findings demonstrate that retrieval and model scale are complementary rather than interchangeable. While retrieval design can partially compensate for smaller models, model capacity remains important for complex reasoning tasks. This work highlights retrieval and task-aware design as key factors for building practical and reproducible scholarly assistants.

评分理由: 论文《Adaptive Stopping for Multi-Turn LLM Reasoning》的核心是解决大语言模型（LLMs）在多轮推理（如自适应RAG和ReAct式代理）中的自适应停止问题，提出了首个用于多轮推理的保形预测框架MiCP。因此，与"Large Language Models”、“Retrieval-Augmented Generation”、“LLM Agents"高度相关（10分），因为这些是论文直接应用和研究的对象。与"Chain of Thought"和"System 2 Thinking"有一定关联（8分），因为多轮推理涉及逐步、深入的思考过程，但论文未明确使用这些术语。其他关键词如MoE、量化、对齐等与论文内容无关（0分）。

大型语言模型（LLM）日益依赖多轮推理与交互（例如自适应检索增强生成（RAG）和ReAct式智能体）来回答复杂问题。这些方法通过迭代检索信息、推理或执行操作来提高准确性，但也引入了一个关键挑战：模型应在何时停止？ 现有方法依赖于启发式停止规则或固定的轮次预算，且无法保证最终预测仍包含正确答案。这一局限在金融和医疗等高风险领域尤为突出，不必要的轮次会增加成本与延迟，而过早停止则可能导致错误决策。共形预测（Conformal Prediction，CP）能提供正式的覆盖度保证，但现有的LLM-CP方法仅适用于单一模型输出，无法处理需自适应停止的多轮流程。为填补这一空白，我们提出了基于共形预测的多轮语言模型（MiCP），这是首个面向多轮推理的CP框架。MiCP在各轮次间分配不同的误差预算，使模型能够在保持整体覆盖度保证的前提下提前停止。我们在自适应RAG和ReAct场景中验证了MiCP，其在单跳和多跳问答基准测试中均实现了目标覆盖度，同时减少了轮次数量、推理成本及预测集大小。我们进一步提出了一项新指标，用于联合评估覆盖度有效性与回答效率。

Large Language Models (LLMs) increasingly rely on multi-turn reasoning and interaction, such as adaptive retrieval-augmented generation (RAG) and ReAct-style agents, to answer difficult questions. These methods improve accuracy by iteratively retrieving information, reasoning, or acting, but introduce a key challenge: \textbf{When should the model stop?} Existing approaches rely on heuristic stopping rules or fixed turn budgets and provide no formal guarantees that the final prediction still contains the correct answer. This limitation is particularly problematic in high-stakes domains such as finance and healthcare, where unnecessary turns increase cost and latency, while stopping too early risks incorrect decisions. Conformal prediction (CP) provides formal coverage guarantees, but existing LLM-CP methods only apply to a single model output and cannot handle multi-turn pipelines with adaptive stopping. To address this gap, we propose Multi-Turn Language Models with Conformal Prediction (MiCP), the first CP framework for multi-turn reasoning. MiCP allocates different error budgets across turns, enabling the model to stop early while maintaining an overall coverage guarantee. We demonstrate MiCP on adaptive RAG and ReAct, where it achieves the target coverage on both single-hop and multi-hop question answering benchmarks while reducing the number of turns, inference cost, and prediction set size. We further introduce a new metric that jointly evaluates coverage validity and answering efficiency.

评分理由: 论文核心研究大语言模型（LLMs）在情感分析任务中对非正式表达（特别是重复延长形式RLF）的理解能力，并提出了一个两阶段的指令调优框架ExpInstruct来提升LLMs的性能和可解释性。因此，与"Large Language Models”（核心研究对象）、“Post-training/Supervised Fine-tuning”（使用微调方法）、“Instruction Tuning”（提出ExpInstruct框架）和"Explainable AI"（提升模型可解释性）高度相关（10分）。与"Pre-training/Domain Adaptation"有一定关联（5分），因为涉及预训练语言模型（PLMs）和领域适应（情感分析中的非正式语言）。其他关键词如MoE、SLMs、Scaling Laws、RLHF、RAG等均未在论文中涉及，故为0分。

参与在线交流的个体常以非正式风格（如表情包和表情符号）表达个人观点。尽管采用非正式交流的语言模型（LMs）已被广泛讨论，但一种独特且具有强调性的形式——重复延长形式（Repetitive Lengthening Form, RLF）——多年来一直被忽视。本文旨在探索两个研究问题的答案：1）RLF对于情感分析（Sentiment Analysis, SA）是否重要？2）语言模型能否理解RLF？受先前语言学研究的启发，我们构建了Lengthening——首个专注于RLF情感分析的多领域数据集，包含85万个样本。此外，我们提出了可解释指令微调（Explainable Instruction Tuning, ExpInstruct），这是一个两阶段的指令微调框架，旨在提升大语言模型（LLMs）在处理RLF时的性能与可解释性。我们进一步提出了一种新颖的统一方法来量化语言模型对非正式表达的理解。研究表明，RLF句子是具有表现力的表达方式，可作为文档级情感的特征标识。此外，RLF对在线内容分析具有潜在价值。实验结果显示，经过微调的预训练语言模型（Pre-trained Language Models, PLMs）在RLF处理性能上可以超越零样本GPT-4，但在解释性方面仍存在不足。最后，我们证明ExpInstruct能够提升开源大语言模型的性能，使其在有限样本下达到与零样本GPT-4相当的RLF处理性能与可解释性。代码与示例数据可在https://github.com/Tom-Owl/OverlookedRLF获取。

Individuals engaging in online communication frequently express personal opinions with informal styles (e.g., memes and emojis). While Language Models (LMs) with informal communications have been widely discussed, a unique and emphatic style, the Repetitive Lengthening Form (RLF), has been overlooked for years. In this paper, we explore answers to two research questions: 1) Is RLF important for sentiment analysis (SA)? 2) Can LMs understand RLF? Inspired by previous linguistic research, we curate \textbf{Lengthening}, the first multi-domain dataset with 850k samples focused on RLF for SA. Moreover, we introduce \textbf{Exp}lainable \textbf{Instruct}ion Tuning (\textbf{ExpInstruct}), a two-stage instruction tuning framework aimed to improve both performance and explainability of LLMs for RLF. We further propose a novel unified approach to quantify LMs’ understanding of informal expressions. We show that RLF sentences are expressive expressions and can serve as signatures of document-level sentiment. Additionally, RLF has potential value for online content analysis. Our results show that fine-tuned Pre-trained Language Models (PLMs) can surpass zero-shot GPT-4 in performance but not in explanation for RLF. Finally, we show ExpInstruct can improve the open-sourced LLMs to match zero-shot GPT-4 in performance and explainability for RLF with limited samples. Code and sample data are available at https://github.com/Tom-Owl/OverlookedRLF

评分理由: 论文核心研究LLM的参数高效微调（PEFT）方法，提出FourierMoE，将MoE架构与频域分析结合，属于大模型技术原理创新。高度相关的关键词：LLMs（论文研究对象）、MoE（核心架构）、PEFT（研究范式）得10分；Supervised Fine-tuning（属于微调范畴）得8分；Domain Adaptation（涉及任务适应）得5分；其余关键词与论文内容无直接关联得0分。

参数高效微调（Parameter-efficient fine-tuning, PEFT）已成为在有限计算资源下适配大语言模型（Large Language Models, LLMs）的关键范式。然而，标准PEFT方法在多任务微调场景中往往表现不佳，因为多样化的优化目标会引发任务干扰，而有限的参数预算则导致表征能力不足。尽管近期研究引入专家混合（mixture-of-experts, MoE）架构以缓解这些问题，但这些方法主要在空间域中操作，可能引入结构冗余和参数开销。为克服这些局限，我们将适配过程重新构建于谱域之中。我们的谱分析表明，不同任务呈现出差异化的频率能量分布，且LLM各层表现出异质的频率敏感性。基于这些发现，我们提出了FourierMoE，该方法将MoE架构与离散傅里叶逆变换（inverse discrete Fourier transform, IDFT）相结合，实现频率感知的适配。具体而言，FourierMoE采用频率自适应路由器将词元分配给专精于不同频段的专家。每个专家学习一组共轭对称的复系数，在完整保留相位与振幅信息的同时，理论上保证可通过无损的IDFT重构为实值空间权重。通过对28个基准测试、多种模型架构及规模的广泛评估，实验表明FourierMoE在单任务与多任务设置中均持续优于现有基线方法，且使用的可训练参数显著更少。这些结果凸显了谱域专家适配作为LLM微调的一种高效且参数高效范式的潜力。

Parameter-efficient fine-tuning (PEFT) has emerged as a crucial paradigm for adapting large language models (LLMs) under constrained computational budgets. However, standard PEFT methods often struggle in multi-task fine-tuning settings, where diverse optimization objectives induce task interference and limited parameter budgets lead to representational deficiency. While recent approaches incorporate mixture-of-experts (MoE) to alleviate these issues, they predominantly operate in the spatial domain, which may introduce structural redundancy and parameter overhead. To overcome these limitations, we reformulate adaptation in the spectral domain. Our spectral analysis reveals that different tasks exhibit distinct frequency energy distributions, and that LLM layers display heterogeneous frequency sensitivities. Motivated by these insights, we propose FourierMoE, which integrates the MoE architecture with the inverse discrete Fourier transform (IDFT) for frequency-aware adaptation. Specifically, FourierMoE employs a frequency-adaptive router to dispatch tokens to experts specialized in distinct frequency bands. Each expert learns a set of conjugate-symmetric complex coefficients, preserving complete phase and amplitude information while theoretically guaranteeing lossless IDFT reconstruction into real-valued spatial weights. Extensive evaluations across 28 benchmarks, multiple model architectures, and scales demonstrate that FourierMoE consistently outperforms competitive baselines in both single-task and multi-task settings while using significantly fewer trainable parameters. These results highlight the promise of spectral-domain expert adaptation as an effective and parameter-efficient paradigm for LLM fine-tuning.

评分理由: 论文核心研究LLM-based agents在长视野推理任务中的上下文管理问题，与"Large Language Models"和"LLM Agents"高度相关（10分）。论文涉及长上下文限制下的信息保留，与"Context Window Extension"相关（8分）。研究长视野推理和决策，与"Chain of Thought"和"System 2 Thinking"有一定关联（5分）。方法涉及上下文压缩策略学习，与"In-context Learning"相关（5分）。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

基于大语言模型（LLM）的智能体在长程推理方面展现出强大潜力，但其上下文长度受部署因素（如内存、延迟和成本）限制，形成了有限的上下文预算。随着交互历史的增长，这引发了在保留过往信息与遵守上下文限制之间的权衡。为应对这一挑战，我们提出预算感知上下文管理（Budget-Aware Context Management, BACM），该方法将上下文管理建模为一个具有上下文预算约束的序列决策问题。它使智能体能够在纳入新观察之前评估可用预算，并决定何时以及以何种程度压缩交互历史。我们进一步开发了BACM-RL，一种基于课程学习的端到端强化学习方法，可在不同上下文预算下学习压缩策略。在组合式多目标问答与长程网页浏览基准测试上的实验表明，BACM-RL在不同模型规模与任务复杂度下均持续优于现有方法，在高复杂度场景中相比强基线获得超过$1.6\times$的性能提升，同时在预算缩减时保持显著优势——而多数方法在此情况下呈现性能下降趋势。

LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained context budget. As interaction histories grow, this induces a trade-off between retaining past information and staying within the context limit. To address this challenge, we propose Budget-Aware Context Management (BACM), which formulates context management as a sequential decision problem with a context budget constraint. It enables agents to assess the available budget before incorporating new observations and decide when and how much of the interaction history to compress. We further develop BACM-RL, an end-to-end curriculum-based reinforcement learning approach that learns compression strategies under varying context budgets. Experiments on compositional multi-objective QA and long-horizon web browsing benchmarks show that BACM-RL consistently outperforms prior methods across model scales and task complexities, achieving over $1.6\times$ gains over strong baselines in high-complexity settings, while maintaining strong advantages as budgets shrink, where most methods exhibit a downward performance trend.

评分理由: 论文核心研究LLMs在贝叶斯启发（估计未知量及其不确定性）中的应用，直接高度相关于"Large Language Models"（10分）。实验通过改变推理努力程度（低、中、高）测试"思考"效果，与"Chain of Thought"和"System 2 Thinking"相关（8分）。研究发现LLMs严重过度自信，与"Hallucination Mitigation"有一定关联（5分）。应用领域包括健康流行率等科学数据，与"AI for Science"相关（8分）。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

大型语言模型（LLM）已被提议作为人类专家的替代方案，用于估计具有相关不确定性的未知量，这一过程被称为贝叶斯启发。我们通过要求十一个LLM估计人口统计数据（如健康患病率、人格特质分布和劳动力市场数据）来测试这一点，并要求它们以95%可信区间的形式表达其不确定性。我们改变每个模型的推理努力程度（低、中、高），以测试更多的“思考”是否会改善结果。我们的研究结果揭示了三个关键结论。首先，规模更大、能力更强的模型能产生更准确的估计，但增加推理努力并未带来一致的益处。其次，所有模型都表现出严重的过度自信：它们的95%区间包含真实值的频率仅为9%至44%，远低于预期的95%。第三，一种称为共形预测的统计重新校准技术可以纠正这种过度自信，通过扩展区间以实现预期的覆盖范围。在一项初步实验中，为模型提供网络搜索访问权限会降低原本已较准确模型的预测质量，而对较弱模型的预测则有适度改善。模型在常被讨论的话题上表现良好，但在专业健康数据方面则存在困难。这些结果表明，LLM的不确定性估计在用于决策之前需要进行统计校正。

Large language models (LLMs) have been proposed as alternatives to human experts for estimating unknown quantities with associated uncertainty, a process known as Bayesian elicitation. We test this by asking eleven LLMs to estimate population statistics, such as health prevalence rates, personality trait distributions, and labor market figures, and to express their uncertainty as 95% credible intervals. We vary each model’s reasoning effort (low, medium, high) to test whether more “thinking” improves results. Our findings reveal three key results. First, larger, more capable models produce more accurate estimates, but increasing reasoning effort provides no consistent benefit. Second, all models are severely overconfident: their 95% intervals contain the true value only 9–44% of the time, far below the expected 95%. Third, a statistical recalibration technique called conformal prediction can correct this overconfidence, expanding the intervals to achieve the intended coverage. In a preliminary experiment, giving models web search access degraded predictions for already-accurate models, while modestly improving predictions for weaker ones. Models performed well on commonly discussed topics but struggled with specialized health data. These results indicate that LLM uncertainty estimates require statistical correction before they can be used in decision-making.

评分理由: 论文研究多模态大语言模型（MLLMs）中的认知幻觉缓解问题，核心贡献是提出一种无需训练的惯性感知视觉激励（IVE）方法，通过打破视觉注意力的惯性模式来改善关系推理。因此，与"Large Language Models"和"Hallucination Mitigation"高度相关（10分），因为论文明确针对MLLMs的幻觉问题。与"Mechanistic Interpretability"相关（8分），因为论文通过token-wise attention分析来理解模型行为。与"Chain of Thought"和"System 2 Thinking"有一定关联（5分），因为论文涉及多步推理和深度推理问题。其他关键词如MoE、SFT、RAG等未在论文中涉及，得0分。

如同静止的物体倾向于保持静止，我们发现多模态大语言模型（MLLMs）中的视觉注意力表现出显著的惯性：一旦在早期解码步骤中稳定下来，注意力便基本保持静态，无法支持认知推理所需的组合性理解。现有的幻觉缓解方法主要针对涉及物体存在或属性的感知性幻觉，但对于此类需要对象间关系推演的认知性幻觉仍显不足。通过基于令牌的注意力分析，我们确定这种视觉惯性是关键因素：对语义关键区域的注意力持续聚焦，未能动态支持关系推理。为此，我们提出一种无需训练的惯性感知视觉激发（Inertia-aware Visual Excitation, IVE）方法，通过将认知推理建模为视觉注意力的动态响应来打破这种惯性模式。具体而言，IVE选择相对于历史注意力趋势动态涌现的视觉令牌，同时区分表现出惯性行为的令牌。为进一步促进组合推理，IVE引入了一种惯性感知惩罚机制，以抑制注意力过度集中并限制其在局部区域的持续滞留。大量实验表明，IVE在多种基础MLLMs和多个幻觉基准测试中均有效，尤其针对认知性幻觉。

Like a body at rest that stays at rest, we find that visual attention in multimodal large language models (MLLMs) exhibits pronounced inertia, remaining largely static once settled during early decoding steps and failing to support the compositional understanding required for cognitive inference. While existing hallucination mitigation methods mainly target perceptual hallucinations concerning object existence or attributes, they remain inadequate for such cognitive hallucinations that require inter-object relational deduction. Through token-wise attention analysis, we identify this visual inertia as a key factor: attention to semantically critical regions remains persistently focused and fails to dynamically support relational inference. We thereby propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention. Specifically, IVE selects visual tokens that are dynamically emerging relative to historical attention trends while distinguishing tokens exhibiting inertial behavior. To further facilitate compositional inference, IVE introduces an inertia-aware penalty that discourages over-concentration and limits the persistence of attention within localized regions. Extensive experiments show that IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.

评分理由: 论文核心研究LLMs在数值推理中的行为模式，特别是关于"数字感"和"捷径使用"，与"Large Language Models"和"Chain of Thought"高度相关（10分），因为论文明确评估LLMs在CoT提示下的表现。与"System 2 Thinking"和"Mechanistic Interpretability"有一定关联（8分），因为研究涉及LLMs的推理过程和理解能力。其他关键词如MoE、SLMs、Scaling Laws、Alignment、RAG等与论文内容无直接关系，得0分。

大型语言模型即使在存在高效数值捷径的情况下，也常常默认采用逐步计算的方式。这引发了一个基本问题：它们是否表现出类似人类行为意义上的数感，即识别数值结构、在适当时应用捷径、在不适用时避免使用捷径的能力？我们提出了SenseMath，这是一个用于评估大语言模型中结构敏感性数值推理能力的受控基准。SenseMath包含4,800个项目，涵盖八个捷径类别和四个数字规模，并配有匹配的强捷径、弱捷径及对照变体。它支持三种认知需求递增的评估场景：捷径使用（模型能否在适合捷径的问题上应用捷径）；适用性判断（模型能否识别何时适用或不宜使用捷径）；以及问题生成（模型能否生成正确包含特定类型捷径的新问题项目）。我们对从GPT-4o-mini到Llama-3.1-8B的五种大语言模型进行评估，结果显示出一致的模式：当被明确提示时，模型能够轻易采用捷径策略，并在适合捷径的项目上获得显著的准确率提升（最高达15%）；然而，在标准的思维链提示下，它们自发采用此类策略的情况少于40%，即使模型已明确具备所需能力。此外，这种能力仅限于使用层面；模型会系统性地将捷径过度推广到不适用的问题上，并且无法从头生成有效的、包含捷径的问题。综上所述，这些结果表明，当前的大语言模型表现出程序性的捷径流畅度，但缺乏对人类数感基础——即理解捷径何时及为何有效的结构性认知。

Large language models often default to step-by-step computation even when efficient numerical shortcuts are available. This raises a basic question: do they exhibit number sense in a human-like behavioral sense, i.e., the ability to recognize numerical structure, apply shortcuts when appropriate, and avoid them when they are not? We introduce SenseMath, a controlled benchmark for evaluating structure-sensitive numerical reasoning in LLMs. SenseMath contains 4,800 items spanning eight shortcut categories and four digit scales, with matched strong-shortcut, weak-shortcut, and control variants. It supports three evaluation settings of increasing cognitive demand: Shortcut Use (whether models can apply shortcuts on shortcut-amenable problems); Applicability Judgment (whether they can recognize when a shortcut is appropriate or misleading); and Problem Generation (whether they can generate new problem items that correctly admit a given type of shortcut). Our evaluation across five LLMs, ranging from GPT-4o-mini to Llama-3.1-8B, shows a consistent pattern: when explicitly prompted, models readily adopt shortcut strategies and achieve substantial accuracy gains on shortcut-amenable items (up to 15%), yet under standard chain-of-thought prompting they spontaneously employ such strategies in fewer than 40% of cases, even when they demonstrably possess the requisite capability. Moreover, this competence is confined to the Use level; models systematically over-generalise shortcuts to problems where they do not apply, and fail to generate valid shortcut-bearing problems from scratch. Together, these results suggest that current LLMs exhibit procedural shortcut fluency without the structural understanding of when and why shortcuts work that underlies human number sense.

评分理由: 论文核心研究Mixture-of-Experts (MoE)语言模型的解释性问题，直接对应"Mixture of Experts"关键词（15分），并涉及Large Language Models（10分）和Mechanistic Interpretability（10分）。其他关键词如SLMs、Scaling Laws、训练方法、推理技术、应用领域等均未在论文中涉及，故得0分。

专家混合（Mixture-of-Experts, MoE）架构已成为扩展大语言模型（Large Language Models, LLMs）的主流选择，其每个令牌仅激活部分参数。尽管MoE架构主要因计算效率而被采用，但其稀疏性是否使其本质上比密集前馈网络（dense feed-forward networks, FFNs）更易于解释，仍是一个开放性问题。我们使用$k$-稀疏探测方法比较了MoE专家与密集FFN，发现专家神经元始终表现出更低的歧义性，且随着路由稀疏性的增加，这一差距进一步扩大。这表明稀疏性压力促使单个神经元乃至整个专家趋向于单义性。基于这一发现，我们将分析单元从神经元层面扩展到专家层面，作为一种更有效的分析尺度。我们通过自动解释数百个专家验证了这一方法。该分析使我们能够澄清关于专家专业化的争论：专家既非宽泛的领域专家（如生物学），也非简单的令牌级处理器；相反，它们作为细粒度的任务专家发挥作用，专注于语言操作或语义任务（例如在LaTeX中闭合括号）。我们的研究结果表明，MoE在专家层面具有固有的可解释性，为大规模模型的可解释性研究提供了更清晰的路径。代码发布于：https://github.com/jerryy33/MoE_analysis

Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While MoE architectures are primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we zoom out from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., closing brackets in LaTeX). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability. Code is available at: https://github.com/jerryy33/MoE_analysis

评分理由: 论文核心研究基于大语言模型（LLMs）的Web智能体（LLM Agents），探讨不同能力模型在处理网页观察（HTML vs. 简洁表示）时的性能差异，并涉及思考令牌预算、幻觉缓解和多步推理等概念。因此，与"Large Language Models"和"LLM Agents"高度相关（10分），与"Chain of Thought"、“System 2 Thinking"和"Hallucination Mitigation"有一定关联（5分），因为这些关键词涉及模型推理过程和错误分析。其他关键词如MoE、SFT、RAG等未在论文中直接涉及，故评0分。

基于大语言模型（LLM）的网页智能体以网页观察——通常以HTML形式表示——作为识别可用动作和规划后续步骤的基础。先前的研究将HTML的冗长性视为性能障碍，并普遍采用观察缩减作为标准做法。我们重新审视了这一趋势，并证明最优观察表示取决于模型能力和思维令牌预算：（1）对于能力较低的模型，紧凑的观察（无障碍树）更可取，而详细的观察（HTML）则对能力更强的模型更有利；此外，增加思维令牌会进一步放大HTML的优势。（2）我们的错误分析表明，高能力模型能够利用HTML中的布局信息实现更精准的动作定位，而低能力模型在较长输入下则更容易产生幻觉。我们还发现，在大多数模型和设置中，融入观察历史能提升性能，而基于差异（diff）的表示提供了一种令牌高效的替代方案。基于这些发现，我们提出实用指导原则：根据模型能力和思维令牌预算自适应选择观察表示，并使用基于差异的表示来融入观察历史。

Web agents based on large language models (LLMs) rely on observations of web pages – commonly represented as HTML – as the basis for identifying available actions and planning subsequent steps. Prior work has treated the verbosity of HTML as an obstacle to performance and adopted observation reduction as a standard practice. We revisit this trend and demonstrate that the optimal observation representation depends on model capability and thinking token budget: (1) compact observations (accessibility trees) are preferable for lower-capability models, while detailed observations (HTML) are advantageous for higher-capability models; moreover, increasing thinking tokens further amplifies the benefit of HTML. (2) Our error analysis suggests that higher-capability models exploit layout information in HTML for better action grounding, while lower-capability models suffer from increased hallucination under longer inputs. We also find that incorporating observation history improves performance across most models and settings, and a diff-based representation offers a token-efficient alternative. Based on these findings, we suggest practical guidelines: adaptively select observation representations based on model capability and thinking token budget, and incorporate observation history using diff-based representations.

评分理由: 论文核心研究多模态大语言模型（MLLMs）在知识密集型视觉问答中的证据利用问题，提出无需训练的推理时框架Look Twice（LoT）。与"Large Language Models"高度相关（10分），因为论文明确研究MLLMs。与"Retrieval-Augmented Generation"相关（8分），因为方法涉及整合检索的文本证据。与"Pre-training"有一定关联（5分），因为框架利用预训练模型。与"Hallucination Mitigation"和"Mechanistic Interpretability"有一定关联（各5分），因为研究涉及减少幻觉并通过注意力模式解释模型行为。其他关键词如MoE、SLMs、Scaling Laws、SFT、Alignment等与论文内容无关，得0分。

回答关于图像的问题通常需要将视觉理解与外部知识相结合。多模态大语言模型（Multimodal Large Language Models, MLLMs）为此提供了自然的框架，但在回答知识密集型查询时，这些模型往往难以识别最相关的视觉与文本证据。在此类场景中，模型必须整合视觉线索与检索到的文本证据——这些证据往往存在噪声或仅部分相关——同时还需定位图像中的细粒度视觉信息。本文提出“双重审视”（Look Twice, LoT），一种无需训练、在推理阶段运行的框架，旨在改进预训练多模态大语言模型对多模态证据的利用能力。具体而言，我们利用模型的注意力模式来估计哪些视觉区域和检索到的文本元素与查询相关，随后基于这些高亮证据生成答案。所选线索通过轻量级的提示级标记进行突出显示，促使模型在生成过程中重新关注相关证据。在多个基于知识的视觉问答基准测试上的实验表明，该方法相较于零样本多模态大语言模型取得了持续的性能提升。在面向视觉中心任务和幻觉抑制基准上的进一步评估证明，即使在没有文本上下文的情况下，仅通过视觉证据高亮也能提升模型表现，且无需额外训练或修改模型架构。源代码将公开发布。

Answering questions about images often requires combining visual understanding with external knowledge. Multimodal Large Language Models (MLLMs) provide a natural framework for this setting, but they often struggle to identify the most relevant visual and textual evidence when answering knowledge-intensive queries. In such scenarios, models must integrate visual cues with retrieved textual evidence that is often noisy or only partially relevant, while also localizing fine-grained visual information in the image. In this work, we introduce Look Twice (LoT), a training-free inference-time framework that improves how pretrained MLLMs utilize multimodal evidence. Specifically, we exploit the model attention patterns to estimate which visual regions and retrieved textual elements are relevant to a query, and then generate the answer conditioned on this highlighted evidence. The selected cues are highlighted through lightweight prompt-level markers that encourage the model to re-attend to the relevant evidence during generation. Experiments across multiple knowledge-based VQA benchmarks show consistent improvements over zero-shot MLLMs. Additional evaluations on vision-centric and hallucination-oriented benchmarks further demonstrate that visual evidence highlighting alone improves model performance in settings without textual context, all without additional training or architectural modifications. Source code will be publicly released.

评分理由: 论文核心研究LLM的压缩技术，特别是针对静态权重和动态KV缓存的SVD低秩压缩方法。因此，与"Large Language Models"和"Quantization"高度相关（10分），因为论文直接研究LLM的模型压缩。与"KV Cache Compression"高度相关（10分），因为摘要明确提到解决动态Key-Value缓存的内存和带宽需求。其他关键词如MoE、SLMs、训练方法、推理技术、AI应用等均未在论文中涉及，故得0分。

大型语言模型的部署受限于静态权重和动态键值缓存对内存与带宽的需求。基于奇异值分解的压缩提供了一种硬件友好的解决方案以降低这些成本。然而，现有方法存在两个关键局限：一些方法在重构误差上表现次优，而另一些方法虽理论最优但实际效率低下。本文提出Swift-SVD，一种基于激活感知的闭式压缩框架，它同时保证了理论最优性、实际效率与数值稳定性。Swift-SVD在给定一批输入时，增量聚合输出激活的协方差，并在聚合后执行单次特征值分解，从而实现无需训练、快速且最优的逐层低秩近似。我们采用有效秩分析局部逐层可压缩性，并设计了一种动态秩分配策略，该策略综合考虑了局部重构损失与端到端的层重要性。在六个大型语言模型和八个数据集上的大量实验表明，Swift-SVD优于现有先进基线方法，在实现最优压缩精度的同时，将端到端压缩时间加速了3至70倍。我们的代码将在论文录用后公开。

The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient. In this paper, we propose Swift-SVD, an activation-aware, closed-form compression framework that simultaneously guarantees theoretical optimum, practical efficiency and numerical stability. Swift-SVD incrementally aggregates covariance of output activations given a batch of inputs and performs a single eigenvalue decomposition after aggregation, enabling training-free, fast, and optimal layer-wise low-rank approximation. We employ effective rank to analyze local layer-wise compressibility and design a dynamic rank allocation strategy that jointly accounts for local reconstruction loss and end-to-end layer importance. Extensive experiments across six LLMs and eight datasets demonstrate that Swift-SVD outperforms state-of-the-art baselines, achieving optimal compression accuracy while delivering 3-70X speedups in end-to-end compression time. Our code will be released upon acceptance.

评分理由: 论文核心研究LLM输出多样性（output variation）在不同任务和规范目标下的评估框架，直接涉及LLMs（10分）、Alignment（8分，讨论对齐目标如安全性和用户效用）和Hallucination Mitigation（8分，分析事实性失败模式如幻觉）。其他关键词如MoE、SLMs、Scaling Laws、Pre-training、RLHF、RAG、CoT、Agents、Quantization等均未在摘要中提及或相关，故评0分。

大型语言模型（LLM）的研究通常在“多样性”的框架下探讨生成、推理、对齐与表征分析中的输出变异问题。然而相关术语体系仍显零散，这主要是由于任务背后的规范性目标往往未被明确阐释。本文提出“魔力、错乱、天堂、罪愆”框架，将输出变异建模于同质化-异质化光谱之上，其价值判定取决于具体任务及其规范性目标。我们将任务归纳为四种规范性语境：认知性（事实准确性）、交互性（用户效用）、社会性（群体表征）与安全性（系统稳健性）。针对每种语境，我们考察了研究变异现象时的典型失效模式及相关术语，例如幻觉、模式坍塌、偏见与信息抹除等。通过应用该框架分析所有跨语境的双向交互关系，我们发现优化单一目标（如提升安全性）可能无意中损害人口统计表征或创作多样性。我们主张建立语境感知的输出变异评估体系，将其重新定义为由任务目标塑造的特性，而非模型固有的本质属性。

Research on Large Language Models (LLMs) studies output variation across generation, reasoning, alignment, and representational analysis, often under the umbrella of “diversity.” Yet the terminology remains fragmented, largely because the normative objectives underlying tasks are rarely made explicit. We introduce the Magic, Madness, Heaven, Sin framework, which models output variation along a homogeneity-heterogeneity axis, where valuation is determined by the task and its normative objective. We organize tasks into four normative contexts: epistemic (factuality), interactional (user utility), societal (representation), and safety (robustness). For each, we examine the failure modes and vocabulary such as hallucination, mode collapse, bias, and erasure through which variation is studied. We apply the framework to analyze all pairwise cross-contextual interactions, revealing that optimizing for one objective, such as improving safety, can inadvertently harm demographic representation or creative diversity. We argue for context-aware evaluation of output variation, reframing it as a property shaped by task objectives rather than a model’s intrinsic trait.

评分理由: 论文明确提出了一个3D原生基础模型（Foundation Model），与关键词1高度相关；摘要最后提到“multimodal 3D world models”，与关键词24高度相关；论文涉及跨模态训练，与预训练/领域适应有一定关联，给关键词5中等分数；其他关键词如MoE、SLMs、对齐、推理、代理等均未在摘要中提及，评为0分。

近期多模态大语言模型在统一的文本与图像理解及生成方面取得了显著性能，但将这种原生能力扩展至三维领域仍面临数据有限的挑战。与丰富的二维图像相比，高质量三维资产稀缺，导致三维合成任务约束不足。现有方法通常依赖间接流程，即在二维空间进行编辑后通过优化将结果提升至三维，牺牲了几何一致性。本文提出Omni123，一个三维原生基础模型，它将文本到二维和文本到三维生成统一于单一自回归框架内。我们的核心见解是，图像与三维之间的跨模态一致性可作为隐式结构约束。通过将文本、图像和三维表示为共享序列空间中的离散标记，该模型利用丰富的二维数据作为几何先验来改进三维表征。我们引入了一种交错式X到X训练范式，在异构配对数据集上协调多样化的跨模态任务，无需完全对齐的文本-图像-三维三元组。通过在自回归序列中遍历语义-视觉-几何循环（例如文本到图像到三维到图像），该模型联合强化了语义对齐、外观保真度和多视角几何一致性。实验表明，Omni123显著提升了文本引导的三维生成与编辑效果，为构建多模态三维世界模型提供了一条可扩展的路径。

Recent multimodal large language models have achieved strong performance in unified text and image understanding and generation, yet extending such native capability to 3D remains challenging due to limited data. Compared to abundant 2D imagery, high-quality 3D assets are scarce, making 3D synthesis under-constrained. Existing methods often rely on indirect pipelines that edit in 2D and lift results into 3D via optimization, sacrificing geometric consistency. We present Omni123, a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework. Our key insight is that cross-modal consistency between images and 3D can serve as an implicit structural constraint. By representing text, images, and 3D as discrete tokens in a shared sequence space, the model leverages abundant 2D data as a geometric prior to improve 3D representations. We introduce an interleaved X-to-X training paradigm that coordinates diverse cross-modal tasks over heterogeneous paired datasets without requiring fully aligned text-image-3D triplets. By traversing semantic-visual-geometric cycles (e.g., text to image to 3D to image) within autoregressive sequences, the model jointly enforces semantic alignment, appearance fidelity, and multi-view geometric consistency. Experiments show that Omni123 significantly improves text-guided 3D generation and editing, demonstrating a scalable path toward multimodal 3D world models.

评分理由: 论文ReFormeR提出了一种基于模式的查询重写方法，核心是使用LLM进行查询重写以改进检索效果。因此，与"Large Language Models"高度相关（10分），因为论文明确使用LLM生成查询重写。与"Retrieval-Augmented Generation"高度相关（10分），因为查询重写是RAG系统中检索前处理的关键步骤，论文在TREC数据集上评估检索性能。与"Mechanistic Interpretability"有一定关联（5分），因为论文通过明确的模式使重写策略可解释，但并非核心解释AI技术。其他关键词如MoE、SFT、量化等与论文内容无关，得0分。

我们提出ReFormeR，一种基于模式引导的查询重构方法。与直接提示语言模型生成查询重构不同，ReFormeR首先从初始查询与经验上更强的重构配对中提取简短的重构模式，将其整合为可迁移重构模式的紧凑库，随后根据新查询的检索上下文为其选择合适的重构模式。所选模式将查询重构约束在可控操作范围内，例如词义消歧、词汇接地或区分性方面添加等。因此，我们提出的方法通过这些重构模式使重构策略显式化，引导大语言模型实现具有针对性且高效的查询重构。我们在TREC DL 2019、DL 2020和DL Hard数据集上的大量实验表明，该方法相较于经典反馈方法及近期基于大语言模型的查询重构与扩展方法均取得了持续性的性能提升。

We present ReFormeR, a pattern-guided approach for query reformulation. Instead of prompting a language model to generate reformulations of a query directly, ReFormeR first elicits short reformulation patterns from pairs of initial queries and empirically stronger reformulations, consolidates them into a compact library of transferable reformulation patterns, and then selects an appropriate reformulation pattern for a new query given its retrieval context. The selected pattern constrains query reformulation to controlled operations such as sense disambiguation, vocabulary grounding, or discriminative facet addition, to name a few. As such, our proposed approach makes the reformulation policy explicit through these reformulation patterns, guiding the LLM towards targeted and effective query reformulations. Our extensive experiments on TREC DL 2019, DL 2020, and DL Hard show consistent improvements over classical feedback methods and recent LLM-based query reformulation and expansion approaches.

评分理由: 该论文的核心是研究基于LLM的智能体（LLM-based agents）在多用户共享状态场景下的安全问题，具体探讨了无意的跨用户信息污染（UCC）现象。因此，与"Large Language Models"和"LLM Agents"高度相关（10分），因为论文直接研究LLM智能体的行为和安全问题。与"Hallucination Mitigation"有一定关联（5分），因为UCC可能导致智能体产生错误的、不符合用户上下文的输出，这可以被视为一种事实性或真实性问题的具体表现形式（silent wrong answers）。论文未涉及其他关键词所指向的具体技术（如MoE、SFT、RAG、量化等）、应用领域（如科学AI）或特定能力（如思维链、工具调用）。

基于大语言模型（LLM）的智能体日益频繁地在重复会话中运行，通过维持任务状态来确保连续性。在许多实际部署中，单个智能体为团队或组织内的多个用户提供服务，在不同用户身份间复用共享的知识层。这种共享的持久性扩大了故障面：当智能体不考虑适用范围而重新应用信息时，仅对某一用户局部有效的信息可能会悄无声息地损害另一用户的结果。我们将这种故障模式称为非意图跨用户污染（Unintentional Cross-user Contamination, UCC）。与对抗性记忆投毒不同，UCC无需攻击者即可发生；它源于良性交互中产生的、具有范围限制的产物被持久保存，并在后续被误用。我们通过一个受控评估协议对UCC进行了形式化界定，提出了三种污染类型的分类体系，并在两种共享状态机制中评估了该问题。在原始共享状态下，仅凭良性交互即可产生57%至71%的污染率。当共享状态为对话性内容时，写入时清理机制是有效的；但当共享状态包含可执行产物时，则会遗留显著的残余风险，污染常表现为悄无声息的错误答案。这些结果表明，共享状态智能体需要超越文本层面清理的、在产物层面的防御机制，以防止隐性的跨用户故障。

LLM-based agents increasingly operate across repeated sessions, maintaining task states to ensure continuity. In many deployments, a single agent serves multiple users within a team or organization, reusing a shared knowledge layer across user identities. This shared persistence expands the failure surface: information that is locally valid for one user can silently degrade another user’s outcome when the agent reapplies it without regard for scope. We refer to this failure mode as unintentional cross-user contamination (UCC). Unlike adversarial memory poisoning, UCC requires no attacker; it arises from benign interactions whose scope-bound artifacts persist and are later misapplied. We formalize UCC through a controlled evaluation protocol, introduce a taxonomy of three contamination types, and evaluate the problem in two shared-state mechanisms. Under raw shared state, benign interactions alone produce contamination rates of 57–71%. A write-time sanitization is effective when shared state is conversational, but leaves substantial residual risk when shared state includes executable artifacts, with contamination often manifesting as silent wrong answers. These results indicate that shared-state agents need artifact-level defenses beyond text-level sanitization to prevent silent cross-user failures.

评分理由: 论文研究多模态大语言模型（MLLMs）在STEM教育（生物学）中的应用，属于大模型在科学领域的应用研究。核心相关关键词：1）“Large Language Models” OR “LLMs” OR “Foundation Models”：论文明确研究MLLMs，属于大模型应用，高度相关（10分）。2）“AI for Science” OR “Bioinformatics” OR “Cheminformatics”：论文应用AI于生物学教育，属于AI for Science范畴，高度相关（10分）。其他关键词涉及具体技术原理（如MoE、Scaling Laws、RLHF等）或特定应用方向（如Agents、Tool Use），论文未涉及，均无关（0分）。

多模态大语言模型（MLLMs）为基于教育内容的对话系统支持多媒体学习提供了机遇。然而，尽管已知对话式人工智能能提升学习参与度，其在视觉丰富的STEM领域中对学习的影响仍未得到充分探索。此外，对于多模态性与对话性如何在生成式人工智能系统中共同影响学习，目前理解有限。本研究报告了一项随机对照在线实验（N = 124）的结果，该实验比较了三种从教科书内容学习生物学的方法：（1）基于文档的对话式人工智能，可生成图文交织的响应（MuDoC）；（2）基于文档的对话式人工智能，仅生成纯文本响应（TexDoC）；（3）具备语义搜索和高亮功能的教科书界面（DocSearch）。使用MuDoC的学习者取得了最高的后测成绩，并报告了最积极的学习体验。值得注意的是，尽管TexDoC在参与度和易用性上显著优于DocSearch，却导致了最低的后测成绩，这揭示了学生感知与学习成果之间的脱节。通过认知负荷理论视角解读，这些发现表明：对话性降低了外在认知负荷，而多模态引发的视觉-言语整合则增加了相关认知负荷，从而带来更好的学习效果。当对话性缺乏多模态支持时，认知努力的减少可能反而会虚增感知理解度，而无法实际提升学习成果。

Multimodal Large Language Models (MLLMs) offer an opportunity to support multimedia learning through conversational systems grounded in educational content. However, while conversational AI is known to boost engagement, its impact on learning in visually-rich STEM domains remains under-explored. Moreover, there is limited understanding of how multimodality and conversationality jointly influence learning in generative AI systems. This work reports findings from a randomized controlled online study (N = 124) comparing three approaches to learning biology from textbook content: (1) a document-grounded conversational AI with interleaved text-and-image responses (MuDoC), (2) a document-grounded conversational AI with text-only responses (TexDoC), and (3) a textbook interface with semantic search and highlighting (DocSearch). Learners using MuDoC achieved the highest post-test scores and reported the most positive learning experience. Notably, while TexDoC was rated as significantly more engaging and easier to use than DocSearch, it led to the lowest post-test scores, revealing a disconnect between student perceptions and learning outcomes. Interpreted through the lens of the Cognitive Load Theory, these findings suggest that conversationality reduces extraneous load, while visual-verbal integration induced by multimodality increases germane load, leading to better learning outcomes. When conversationality is not complemented by multimodality, reduced cognitive effort may instead inflate perceived understanding without improving learning outcomes.

评分理由: 论文核心是使用多LLM框架从1430篇文献中提取数据，构建神经疾病异常头部运动数据集NeuroPose-AHM，并应用于颈椎肌张力障碍分析。因此，与"Large Language Models"高度相关（10分），因为明确使用了多LLM提取框架；与"AI for Science"高度相关（10分），属于AI在生物医学/神经科学领域的应用。其他关键词（如MoE、SFT、RAG等）均未在摘要中提及，与论文内容完全无关，得0分。

异常头部运动（Abnormal Head Movements, AHMs）广泛存在于多种神经系统疾病中；然而，缺乏一个整合运动学测量、临床严重程度评分和患者人口统计学特征的多病种资源，一直是开发人工智能驱动诊断工具的持续障碍。为填补这一空白，本研究引入了NeuroPose-AHM，这是一个基于知识的、由神经系统疾病引起的AHMs数据集，通过一个应用于1,430篇同行评审文献的多大型语言模型提取框架构建而成。该数据集包含来自846篇AHM相关论文的2,756条患者群体级记录，涵盖57种神经系统疾病。大型语言模型间可靠性分析证实了提取的稳健性，研究级分类达到了高度一致性（kappa = 0.822）。为展示数据集的分析效用，我们以最直接由病理性头部运动定义的疾病——颈肌张力障碍（Cervical Dystonia, CD）为例，应用了一个四任务分析框架。首先，任务1执行了多标签AHM类型分类（F1 = 0.856）。任务2构建了头颈严重程度指数（Head-Neck Severity Index, HNSI），这是一个统一指标，用于标准化异质性的临床评定量表。随后在任务3中评估了该指数的临床相关性，HNSI通过真实世界CD患者数据进行了验证，其匹配的重度区间比例（6.7%）为指数在高严重程度范围内的校准提供了初步的合理性依据。最后，任务4在运动类型概率与HNSI评分之间进行了桥接分析，产生了显著相关性（p < 0.001）。这些结果证明了NeuroPose-AHM作为一个结构化的、基于知识的资源在神经系统AHM研究中的分析效用。NeuroPose-AHM数据集已在Zenodo上公开（https://doi.org/10.5281/zenodo.19386862）。

Abnormal head movements (AHMs) manifest across a broad spectrum of neurological disorders; however, the absence of a multi-condition resource integrating kinematic measurements, clinical severity scores, and patient demographics constitutes a persistent barrier to the development of AI-driven diagnostic tools. To address this gap, this study introduces NeuroPose-AHM, a knowledge-based dataset of neurologically induced AHMs constructed through a multi-LLM extraction framework applied to 1,430 peer-reviewed publications. The dataset contains 2,756 patient-group-level records spanning 57 neurological conditions, derived from 846 AHM-relevant papers. Inter-LLM reliability analysis confirms robust extraction performance, with study-level classification achieving strong agreement (kappa = 0.822). To demonstrate the dataset’s analytical utility, a four-task framework is applied to cervical dystonia (CD), the condition most directly defined by pathological head movement. First, Task 1 performs multi-label AHM type classification (F1 = 0.856). Task 2 constructs the Head-Neck Severity Index (HNSI), a unified metric that normalizes heterogeneous clinical rating scales. The clinical relevance of this index is then evaluated in Task 3, where HNSI is validated against real-world CD patient data, with aligned severe-band proportions (6.7%) providing a preliminary plausibility indication for index calibration within the high severity range. Finally, Task 4 performs bridge analysis between movement-type probabilities and HNSI scores, producing significant correlations (p less than 0.001). These results demonstrate the analytical utility of NeuroPose-AHM as a structured, knowledge-based resource for neurological AHM research. The NeuroPose-AHM dataset is publicly available on Zenodo (https://doi.org/10.5281/zenodo.19386862).

评分理由: 论文HieraVid专注于视频大语言模型（VideoLLMs）的推理加速，通过分层令牌剪枝减少计算负担。核心与大语言模型（LLMs）高度相关（10分），因为论文直接研究VideoLLMs（LLMs的一种应用）。与"Quantization"或"Model Compression”（5分）和"Speculative Decoding"或"Inference Acceleration"（5分）有一定关联，因为剪枝是一种模型压缩和推理加速技术，但论文未直接涉及量化或推测解码。其他关键词如MoE、SFT、RAG等与论文内容无关（0分）。

视频大语言模型（VideoLLMs）在视频理解方面展现出卓越能力，但海量的输入视频令牌带来了巨大的计算负担。现有方法主要在输入层面对视频令牌进行剪枝，却忽略了视频与大语言模型（LLMs）内部固有的信息结构。为此，我们提出HieraVid——一种分层剪枝框架，能够渐进且动态地减少视觉冗余。基于视频具有片段-帧结构以及LLMs内部单向传播多模态信息这两点观察，我们将剪枝分解为三个层级：1）片段级：视频令牌首先在时间维度上分段并在空间维度上合并；2）帧级：同一片段内相似的帧被联合剪枝以保持多样性；3）层级：随着LLM层数增加，冗余逐渐减少且不影响性能。我们在四个广泛使用的视频理解基准上进行了大量实验，以全面评估HieraVid的有效性。值得注意的是，仅保留30%令牌的情况下，HieraVid取得了新的最先进性能，同时分别保持了LLaVA-Video-7B和LLaVA-OneVision-7B超过98%与99%的性能表现。

Video Large Language Models (VideoLLMs) have demonstrated impressive capabilities in video understanding, yet the massive number of input video tokens incurs a significant computational burden for deployment. Existing methods mainly prune video tokens at input level while neglecting the inherent information structure embedded in videos and large language models (LLMs). To address this, we propose HieraVid, a hierarchical pruning framework that progressively and dynamically reduces visual redundancy. Based on two observations that videos possess the segment-frame structure and LLMs internally propagate multi-modal information unidirectionally, we decompose pruning into three levels: 1) segment-level, where video tokens are first temporally segmented and spatially merged; 2) frame-level, where similar frames within the same segment are jointly pruned to preserve diversity; 3) layer-level, redundancy gradually shrinks as LLM layer increases w/o compromising performance. We conduct extensive experiments on four widely used video understanding benchmarks to comprehensively evaluate the effectiveness of HieraVid. Remarkably, with only 30% of tokens retained, HieraVid achieves new state-of-the-art performance, while maintaining over 98% and 99% of the performance of LLaVA-Video-7B and LLaVA-OneVision-7B, respectively.

评分理由: 该论文研究语言模型（LLMs）内部机制，特别是如何定位和处理实体信息的神经元，属于大模型技术原理的创新研究。因此，与"Large Language Models"高度相关（10分）。论文的核心是分析模型内部工作机制，属于可解释性AI范畴，与"Mechanistic Interpretability"高度相关（10分）。论文未涉及其他关键词的具体技术、应用领域或训练方法，因此其他关键词评分为0分。

语言模型能够回答许多以实体为中心的事实性问题，但其内部机制如何参与这一过程尚不明确。本研究在多个语言模型中探讨了该问题。我们使用针对每个实体的模板化提示定位了具有实体选择性的MLP神经元，并通过对基于PopQA的问答示例进行因果干预来验证这些神经元。在从PopQA中选取的200个实体构成的数据集上，定位到的神经元主要集中在模型浅层。负向消融会导致实体特异性遗忘，而在占位符标记处进行受控注入，相较于平均实体和错误单元对照组，能提升答案检索效果。对于许多实体，一旦上下文初始化，仅激活单个定位神经元便足以恢复与实体一致的预测结果，这表明模型采用了紧凑的实体检索机制，而非纯粹依赖随深度渐进的语义丰富化过程。模型对别名、缩写、拼写错误及多语言形式均表现出鲁棒性，这支持了规范化解释。该效应显著但非普适：并非每个实体都存在可靠的单神经元操控点，且流行实体的覆盖率更高。总体而言，这些结果识别出了稀疏、可因果干预的接入点，可用于分析和调控模型基于实体的知识行为。

Language models can answer many entity-centric factual questions, but it remains unclear which internal mechanisms are involved in this process. We study this question across multiple language models. We localize entity-selective MLP neurons using templated prompts about each entity, and then validate them with causal interventions on PopQA-based QA examples. On a curated set of 200 entities drawn from PopQA, localized neurons concentrate in early layers. Negative ablation produces entity-specific amnesia, while controlled injection at a placeholder token improves answer retrieval relative to mean-entity and wrong-cell controls. For many entities, activating a single localized neuron is sufficient to recover entity-consistent predictions once the context is initialized, consistent with compact entity retrieval rather than purely gradual enrichment across depth. Robustness to aliases, acronyms, misspellings, and multilingual forms supports a canonicalization interpretation. The effect is strong but not universal: not every entity admits a reliable single-neuron handle, and coverage is higher for popular entities. Overall, these results identify sparse, causally actionable access points for analyzing and modulating entity-conditioned factual behavior.

评分理由: 论文核心研究大语言模型压缩技术，提出了一种基于SVD的低秩分解框架，因此与"Large Language Models"和"Quantization"高度相关（10分）。其他关键词如MoE、SLMs、训练方法、推理技术、AI for Science等均未在摘要中提及，因此评分为0分。

我们提出一种基于快速低秩分解的大型语言模型压缩框架，该框架能够在不重新训练的情况下快速压缩数十亿参数模型。与现有仅基于原始输入进行优化、忽略上游压缩导致的分布偏移从而传播误差的分解方法，或仅依赖偏移输入而可能偏离原始输出的方法不同，我们的方法同时兼顾二者。除单层压缩外，我们进一步对每个Transformer模块进行端到端精调，最小化模块级输出失真，使压缩层能够共同补偿累积误差。通过将每个压缩层锚定至原始输出，同时显式建模输入分布偏移，我们的方法找到了一种保持与原始模型功能等效的低秩近似。在大型语言模型上的实验表明，我们的方法在不同压缩率下均持续优于现有的基于奇异值分解（SVD）的基线方法，且在激进压缩预算下优势愈加显著——此时对比方法性能大幅下降或完全失效，从而为高效、大规模模型部署提供了实用解决方案。

We introduce a fast low-rank factorization-based framework for compressing large language models that enables rapid compression of billion-parameter models without retraining. Unlike existing factorization-based approaches that optimize only on the original inputs, ignoring distribution shifts from upstream compression and thus propagating errors forward, or those that rely only on shifted inputs and risk drifting away from the original outputs, our approach accounts for both. Beyond individual layer compression, we further refine each transformer block end-to-end, minimizing block-level output distortion and allowing compressed layers to jointly compensate for accumulated errors. By anchoring each compressed layer to the original outputs while explicitly modeling input distribution shifts, our method finds a low-rank approximation that maintains functional equivalence with the original model. Experiments on large language models show that our method consistently outperforms existing SVD-based baselines across compression ratios, with the advantage becoming increasingly pronounced at aggressive compression budgets, where competing methods degrade substantially or collapse entirely, offering a practical solution for efficient, large-scale model deployment.

评分理由: 论文《RuleForge: Automated Generation and Validation for Web Vulnerability Detection at Scale》主要研究利用大语言模型（LLMs）自动化生成和验证网络安全漏洞检测规则。论文的核心创新在于提出了一个“LLM-as-a-judge”置信度验证系统，用于评估生成规则的敏感性和特异性，并展示了多事件类型检测的智能体工作流程概念验证。因此，论文与“Large Language Models”高度相关（10分），因为LLM是系统的核心组件，用于规则生成和验证。论文也与“LLM Agents”有一定关联（8分），因为摘要中提到了“agentic workflow for multi-event-type detection”，表明使用了LLM驱动的智能体工作流程。其他关键词（如MoE、SFT、RAG等）在论文中未提及或未涉及，因此评分为0分。

安全团队面临一项挑战：新披露的通用漏洞与暴露（CVE）的数量远超手动开发检测机制的能力。2025年，美国国家漏洞数据库发布了超过48,000个新漏洞，这推动了对自动化解决方案的需求。我们提出RuleForge，一个AWS内部系统，它能够从描述CVE细节的结构化Nuclei模板中自动生成检测规则——这些基于JSON的模式用于识别利用特定漏洞的恶意HTTP请求。Nuclei模板提供了标准化的、基于YAML的漏洞描述，作为我们规则生成过程的结构化输入。

本文重点介绍RuleForge在CVE相关威胁检测中的架构与运营部署，特别强调我们新颖的LLM-as-a-judge（大型语言模型作为评判者）置信度验证系统以及系统化的反馈集成机制。该验证方法从两个维度评估候选规则——灵敏度（避免漏报）和特异度（避免误报）——在生产环境中，与仅使用合成测试的验证相比，实现了0.75的AUROC（受试者工作特征曲线下面积），并将误报率降低了67%。我们采用的5x5生成策略（五个并行候选规则，每个最多进行五次优化尝试）结合持续反馈循环，实现了系统性的质量提升。我们还介绍了从非结构化数据源生成规则的扩展功能，并展示了一个用于多事件类型检测的概念验证智能体工作流。我们的经验教训强调了将LLM应用于网络安全任务时的关键考量，包括缓解过度自信问题，以及在提示设计和通过人在环验证对生成规则进行质量审查时，领域专业知识的重要性。

Security teams face a challenge: the volume of newly disclosed Common Vulnerabilities and Exposures (CVEs) far exceeds the capacity to manually develop detection mechanisms. In 2025, the National Vulnerability Database published over 48,000 new vulnerabilities, motivating the need for automation. We present RuleForge, an AWS internal system that automatically generates detection rules–JSON-based patterns that identify malicious HTTP requests exploiting specific vulnerabilities–from structured Nuclei templates describing CVE details. Nuclei templates provide standardized, YAML-based vulnerability descriptions that serve as the structured input for our rule generation process. This paper focuses on RuleForge’s architecture and operational deployment for CVE-related threat detection, with particular emphasis on our novel LLM-as-a-judge (Large Language Model as judge) confidence validation system and systematic feedback integration mechanism. This validation approach evaluates candidate rules across two dimensions–sensitivity (avoiding false negatives) and specificity (avoiding false positives)–achieving AUROC of 0.75 and reducing false positives by 67% compared to synthetic-test-only validation in production. Our 5x5 generation strategy (five parallel candidates with up to five refinement attempts each) combined with continuous feedback loops enables systematic quality improvement. We also present extensions enabling rule generation from unstructured data sources and demonstrate a proof-of-concept agentic workflow for multi-event-type detection. Our lessons learned highlight critical considerations for applying LLMs to cybersecurity tasks, including overconfidence mitigation and the importance of domain expertise in both prompt design and quality review of generated rules through human-in-the-loop validation.

评分理由: 论文核心研究LLM训练中归一化层与优化器的耦合效应，属于大模型技术原理的创新。与"Large Language Models"高度相关（10分），因为全文围绕LLM训练展开。与"Pre-training"相关（8分），因为研究的是训练过程中的优化问题，属于预训练阶段的技术改进。其他关键词如MoE、SFT、RAG等均未涉及，故得0分。

在大语言模型训练中，归一化层与优化器通常被视为独立的设计选择。通过一项在10亿参数规模、1000训练步长下进行的3x2因子实验，我们发现这一假设可能并不成立：动态Erf（Derf；Chen & Liu, 2025）与Muon优化器（Jordan, 2024）存在显著的负向交互作用——相较于RMSNorm，其性能差距从使用AdamW时的+0.31纳特扩大至使用Muon时的+0.97纳特，增幅约达三倍。作为有界归一化对照组的动态Tanh（DyT；Zhu et al., 2025）则未出现此类损失。我们的证据表明，在Muon更快的谱范数增长下，erf函数存在两种失效模式：饱和（有损压缩）与尺度盲区（忽略激活值幅度）。通过引入指数移动平均混合机制来恢复运行尺度估计，可弥补约84%的性能差距。此外，将Derf的alpha参数从其默认发布值0.5降低至0.3，能使erf保持在线性近似区间内从而大致保留相对尺度，进而恢复约80%的性能；该设置并非Chen & Liu（2025）论文中的默认值。若将Derf的默认alpha参数与Muon联用，虽不会产生NaN或发散，但会导致0.66纳特的交互损失，这种失效在短周期试验中极易被忽略。

In LLM training, normalization layers and optimizers are typically treated as independent design choices. In a 3x2 factorial at 1B parameters and 1000 training steps, we show this assumption can fail: Dynamic Erf (Derf; Chen & Liu, 2025) suffers a large negative interaction with Muon (Jordan, 2024), with its gap to RMSNorm growing from +0.31 nats under AdamW to +0.97 under Muon, approximately three times larger. Dynamic Tanh (DyT; Zhu et al., 2025), included as a bounded-normalizer control, shows no such penalty. Our evidence points to two failure modes of erf under Muon’s faster spectral-norm growth: saturation (lossy compression) and scale blindness (discarding activation magnitude). An EMA-blend that reintroduces running scale estimates recovers ~84% of the gap. Separately, reducing Derf’s alpha from its published default (0.5 to 0.3) recovers ~80% by keeping erf in its near-linear regime, where it approximately preserves relative scale; this setting is not the published default of Chen & Liu (2025). Using Derf’s published default alpha with Muon incurs a 0.66-nat interaction penalty without producing NaNs or divergence, making the failure easy to miss in short pilot runs.

评分理由: 论文《ActionParty: Multi-Subject Action Binding in Generative Video Games》专注于视频扩散模型中的多主体动作绑定问题，提出了一种用于生成视频游戏的动作可控多主体世界模型。论文的核心是开发一个能够同时控制多个主体的视频世界模型，这与关键词"World Models" AND “General World Models"高度相关（10分），因为论文明确提到构建"world models"并解决现有模型在单主体设置上的限制。关键词"Multi-agent Systems” OR “Agent Coordination"有一定关联（5分），因为论文涉及控制多个玩家（主体）在场景中的交互，属于多主体系统的范畴。其他关键词均与论文内容无关（0分），因为论文未涉及大语言模型、训练技术、推理方法、模型优化、AI科学应用等主题。

视频扩散模型的最新进展推动了能够模拟交互式环境的“世界模型”的发展。然而，这些模型大多局限于单智能体场景，无法同时控制场景中的多个智能体。本研究针对现有视频扩散模型中存在的动作绑定这一根本性问题展开，这些模型难以将特定动作与其对应的主体相关联。为此，我们提出了ActionParty，一个用于生成式视频游戏的可控动作多主体世界模型。该模型引入了主体状态令牌，即能够持续捕捉场景中每个主体状态的隐变量。通过结合空间偏置机制对状态令牌和视频隐变量进行联合建模，我们将全局视频帧渲染与受个体动作控制的主体更新分离开来。我们在Melting Pot基准测试上评估了ActionParty，证明了这是首个能够在46种不同环境中同时控制多达七个玩家的视频世界模型。我们的结果显示，该模型在动作跟随准确性和身份一致性方面均有显著提升，同时能够通过复杂的交互对主体进行稳健的自回归追踪。

Recent advances in video diffusion have enabled the development of “world models” capable of simulating interactive environments. However, these models are largely restricted to single-agent settings, failing to control multiple agents simultaneously in a scene. In this work, we tackle a fundamental issue of action binding in existing video diffusion models, which struggle to associate specific actions with their corresponding subjects. For this purpose, we propose ActionParty, an action controllable multi-subject world model for generative video games. It introduces subject state tokens, i.e. latent variables that persistently capture the state of each subject in the scene. By jointly modeling state tokens and video latents with a spatial biasing mechanism, we disentangle global video frame rendering from individual action-controlled subject updates. We evaluate ActionParty on the Melting Pot benchmark, demonstrating the first video world model capable of controlling up to seven players simultaneously across 46 diverse environments. Our results show significant improvements in action-following accuracy and identity consistency, while enabling robust autoregressive tracking of subjects through complex interactions.

评分理由: 论文核心研究扩散语言模型（DLMs）中的专家混合（MoE）路由机制，具体比较了token-choice（TC）和expert-choice（EC）路由，并提出了基于时间步的专家容量自适应分配方法。因此，与"Mixture of Experts” OR “MoE” OR “Sparse Models"高度相关（10分）。论文涉及扩散语言模型，属于大模型的一种变体，但与标准LLMs/Foundation Models关联较弱（5分）。其他关键词如SLMs、Scaling Laws、各种训练/对齐/推理技术、AI for Science等均未在论文中涉及（0分）。

扩散语言模型（DLMs）支持并行、非自回归的文本生成，但现有的DLM专家混合（MoE）模型沿用了自回归系统中的令牌选择（TC）路由机制，导致负载不均衡与计算分配僵化。我们证明专家选择（EC）路由更适合DLMs：该机制通过设计实现确定性负载均衡，相比TC路由具有更高吞吐量与更快收敛速度。基于EC路由的专家容量可外部调控的特性，我们引入了时间步依赖的专家容量分配方法，根据去噪步骤动态调整专家分配。研究发现，在保持浮点运算量（FLOPs）匹配的条件下，为低掩码率步骤分配更多容量能持续获得最佳性能，并提供了机制性解释：低掩码率上下文中的令牌学习效率高出数量级，因此将计算资源集中于此阶段可获得最大边际收益。最后，我们证明仅需替换路由模块即可将现有预训练的TC-DLM改造为EC架构，在多种下游任务中实现更快收敛与更高准确率。这些结果共同确立了EC路由作为DLM MoE模型的优越范式，并证明DLMs中的计算可视为自适应策略而非固定架构常数。代码发布于https://github.com/zhangshuibai/EC-DLM。

Diffusion language models (DLMs) enable parallel, non-autoregressive text generation, yet existing DLM mixture-of-experts (MoE) models inherit token-choice (TC) routing from autoregressive systems, leading to load imbalance and rigid computation allocation. We show that expert-choice (EC) routing is a better fit for DLMs: it provides deterministic load balancing by design, yielding higher throughput and faster convergence than TC. Building on the property that EC capacity is externally controllable, we introduce timestep-dependent expert capacity, which varies expert allocation according to the denoising step. We find that allocating more capacity to low-mask-ratio steps consistently achieves the best performance under matched FLOPs, and provide a mechanistic explanation: tokens in low-mask-ratio contexts exhibit an order-of-magnitude higher learning efficiency, so concentrating compute on these steps yields the largest marginal return. Finally, we show that existing pretrained TC DLMs can be retrofitted to EC by replacing only the router, achieving faster convergence and improved accuracy across diverse downstream tasks. Together, these results establish EC routing as a superior paradigm for DLM MoE models and demonstrate that computation in DLMs can be treated as an adaptive policy rather than a fixed architectural constant. Code is available at https://github.com/zhangshuibai/EC-DLM.

评分理由: 该论文专注于使用深度学习（ResNet18和自定义CNN）进行乳腺癌PAM50亚型分类的医学图像分析，属于生物信息学/生物医学AI应用领域。论文的核心是优化驱动的深度学习框架，用于从组织病理学图像中预测分子亚型，不涉及任何大语言模型（LLM）、模型架构创新（如MoE）、训练技术（如预训练、微调、对齐）、推理优化、代理系统或通用AI技术。因此，除了最后一个关键词“AI for Science” OR “Bioinformatics” OR “Cheminformatics”高度相关（评分为10）外，其他所有关键词均完全无关（评分为0）。

乳腺癌是一种具有高度异质性且分子特征多样的疾病。PAM50基因特征谱被广泛认为是将乳腺癌划分为内在亚型的金标准，有助于实现更个性化的治疗策略。本研究提出了一种新型优化驱动的深度学习框架，旨在通过直接从H&E染色全切片图像（WSIs）预测PAM50亚型，从而降低对昂贵分子检测的依赖。我们的方法将非支配排序遗传算法II（NSGA-II）与基于蒙特卡洛Dropout的不确定性估计相结合，共同优化了图像块的信息量、空间多样性、不确定性以及块数量。该方法能够筛选出少量但信息量极高的图像块子集用于分类。我们采用ResNet18主干网络进行特征提取，并利用定制的CNN分类头进行分类。在评估阶段，我们使用内部TCGA-BRCA数据集作为训练队列，外部CPTAC-BRCA数据集作为测试队列。在内部数据集上，使用TCGA-BRCA队列的627张WSIs取得了0.8812的F1分数和0.9841的AUC值。在外部验证数据集上，所提方法的性能表现为F1分数0.7952和AUC值0.9512。这些结果表明，与现有方法相比，所提出的优化引导、不确定性感知的图像块选择策略能够实现高性能，并提高基于组织病理学的PAM50分类的计算效率，这预示了一种可扩展的、基于影像学的替代方案，有望为临床决策提供支持。

Breast cancer is a highly heterogeneous disease with diverse molecular profiles. The PAM50 gene signature is widely recognized as a standard for classifying breast cancer into intrinsic subtypes, enabling more personalized treatment strategies. In this study, we introduce a novel optimization-driven deep learning framework that aims to reduce reliance on costly molecular assays by directly predicting PAM50 subtypes from H&E-stained whole-slide images (WSIs). Our method jointly optimizes patch informativeness, spatial diversity, uncertainty, and patch count by combining the non-dominated sorting genetic algorithm II (NSGA-II) with Monte Carlo dropout-based uncertainty estimation. The proposed method can identify a small but highly informative patch subset for classification. We used a ResNet18 backbone for feature extraction and a custom CNN head for classification. For evaluation, we used the internal TCGA-BRCA dataset as the training cohort and the external CPTAC-BRCA dataset as the test cohort. On the internal dataset, an F1-score of 0.8812 and an AUC of 0.9841 using 627 WSIs from the TCGA-BRCA cohort were achieved. The performance of the proposed approach on the external validation dataset showed an F1-score of 0.7952 and an AUC of 0.9512. These findings indicate that the proposed optimization-guided, uncertainty-aware patch selection can achieve high performance and improve the computational efficiency of histopathology-based PAM50 classification compared to existing methods, suggesting a scalable imaging-based replacement that has the potential to support clinical decision-making.

评分理由: 该论文研究帕金森病的脑电图（EEG）特征分析，使用多头部注意力变换器分类器进行疾病状态分类。论文核心是生物医学信号处理和神经科学应用，而非大模型或深度学习技术原理的创新。所有关键词均与大模型技术、训练方法、推理优化、代理系统等直接相关，而本文仅涉及传统的机器学习分类器（注意力变换器）在特定生物医学领域的应用。因此，绝大多数关键词评分为0。仅有两个关键词获得5分：1）“Mechanistic Interpretability” OR “Explainable AI”：论文强调"interpretable EEG features"和可解释性分析，与可解释AI有一定关联；2）“AI for Science” OR “Bioinformatics” OR “Cheminformatics”：论文属于AI在生物医学（神经科学）领域的应用，与"AI for Science"子领域相关。但论文未涉及大模型技术，创新性主要体现在生物医学应用而非AI技术本身，因此相关度有限。

帕金森病（PD）会改变皮层神经动力学，但可靠的非侵入性电生理生物标志物仍难以确定。本研究探讨了捕捉神经动力学互补方面的可解释脑电图特征是否能区分帕金森病神经状态。我们提取了一套全面的可解释特征，并将其分为标准描述符（谱功率、相位同步、时域统计量）和动力学描述符（非周期性活动、跨频率耦合、无标度动力学、神经元雪崩统计量以及瞬时频率测量）。使用严格的留一受试者交叉验证训练了一个多头注意力变换器分类器。进行了组间比较以识别与疾病和用药状态相关的电生理差异。标准特征集在区分用药状态（PDoff 与 PDon）方面表现最强，而动力学特征集在帕金森病患者与健康对照的对比中表现出竞争力。随机特征消融分析表明，动力学描述符提供了分布在多个特征中的互补信息，而相关性分析显示两个特征集内部冗余度较低。组间比较揭示了用药敏感的δ波功率和电压方差降低、神经元雪崩统计量的调节、帕金森病患者θ波相位同步的持续增加，以及与疾病相关的跨频率相互作用改变。传统的频谱和同步特征主要反映了与用药相关的神经调节，而动力学描述符则揭示了与疾病及用药均相关的皮层网络组织更广泛的改变。这些发现支持多变量脑电图表征作为开发帕金森病非侵入性生物标志物的一个有前景的框架。

Parkinsons disease (PD) alters cortical neural dynamics, yet reliable non-invasive electrophysiological biomarkers remain elusive. This study examined whether interpretable EEG features capturing complementary aspects of neural dynamics can discriminate Parkinsonian neural states. A comprehensive set of interpretable features was extracted and grouped into Standard descriptors (spectral power, phase synchronization, time-domain statistics) and Dynamical descriptors (aperiodic activity, cross-frequency coupling, scale-free dynamics, neuronal avalanche statistics, and instantaneous frequency measures). A multi-head attention transformer classifier was trained using strict LOSO validation. Group-level comparisons were performed to identify electrophysiological differences associated with disease and medication state. Standard feature sets achieved strongest performance in discriminating medication states (PDoff vs PDon), whereas Dynamical performed competitively in contrasts between PD patients and healthy controls. Random feature ablation analyses indicated that Dynamical descriptors provide complementary information distributed across features while correlation analysis revealed low redundancy within both feature sets. Group-level comparisons revealed medication-sensitive reductions in delta power and voltage variance, modulation of neuronal avalanche statistics, persistent increases in theta phase synchronization in PD patients, and disease-related alterations in cross-frequency interactions. Traditional spectral and synchronization features primarily reflect medication-related neural modulation, whereas dynamical descriptors reveal broader alterations in cortical network organization associated with disease but also with medication. These findings support multivariate EEG representations as a promising framework for developing non-invasive biomarkers of PD.

评分理由: 该论文专注于森林结构监测的计算机视觉任务，提出了一种基于知识蒸馏（KD）的框架FSKD，使用多模态教师模型（融合RGBI图像与LiDAR数据）训练纯RGBI学生模型。论文的核心技术是知识蒸馏、多模态融合和SegFormer架构的应用，属于遥感图像处理和生态信息学领域。所有关键词均直接涉及大语言模型（LLM）及其相关技术（如训练方法、推理优化、代理系统等），而本论文完全不涉及任何语言模型、文本生成或自然语言处理内容。唯一略有相关的关键词是"AI for Science” OR “Bioinformatics” OR “Cheminformatics”，因为该研究属于生态科学中的AI应用（森林监测），可视为广义的"AI for Science"，但并非核心匹配（论文未提及生物信息学或化学信息学）。因此，除该关键词给5分外，其余所有关键词评分为0。

极高分辨率（VHR）的单木尺度森林结构数据对于碳循环、生物多样性与生态系统监测至关重要。然而，尽管机载激光雷达（LiDAR）是冠层高度模型（CHM）、植物面积指数（PAI）和叶层高度多样性（FHD）等森林结构指标的金标准，其仍存在成本高昂且获取频率低的局限。我们提出了FSKD：一种从LiDAR到红绿蓝-红外（RGBI）影像的知识蒸馏（KD）框架。该框架中，一个多模态教师模型通过交叉注意力机制融合RGBI影像与LiDAR衍生的平面指标及垂直剖面信息，而一个仅使用RGBI的SegFormer学生模型则学习复现这些输出。该方法在德国萨克森州384平方公里的森林区域（地面采样距离（GSD）为20厘米）上进行训练，并在八个地理分布不同的测试区进行评估。学生模型实现了零样本CHM预测的最先进（SOTA）性能（中值绝对误差MedAE 4.17米，决定系数R²=0.51，交并比IoU 0.87），其平均绝对误差（MAE，5.81米对比8.14–10.84米）比HRCHM/DAC基线模型降低了29–46%，且具有更强的相关系数（0.713对比0.166–0.652）。消融实验表明，多模态融合相比仅使用RGBI的训练将性能提升了10–26%，并且采用适当模型容量的非对称蒸馏至关重要。该方法能够联合预测CHM、PAI和FHD，这是当前单目CHM估计算法所不具备的多指标预测能力，尽管PAI/FHD的迁移效果仍具有区域依赖性，并能从本地校准中获益。该框架在存在时间不匹配（冬季LiDAR数据，夏季RGBI影像）的情况下依然有效，从而解除了严格同步采集的限制，为“德国数字孪生”及国家数字正射影像计划等工作流程提供了可扩展的20厘米级业务化监测能力。

Very High Resolution (VHR) forest structure data at individual-tree scale is essential for carbon, biodiversity, and ecosystem monitoring. Still, airborne LiDAR remains costly and infrequent despite being the reference for forest structure metrics like Canopy Height Model (CHM), Plant Area Index (PAI), and Foliage Height Diversity (FHD). We propose FSKD: a LiDAR-to-RGB-Infrared (RGBI) knowledge distillation (KD) framework in which a multi-modal teacher fuses RGBI imagery with LiDAR-derived planar metrics and vertical profiles via cross-attention, and an RGBI-only SegFormer student learns to reproduce these outputs. Trained on 384 $km^2$ of forests in Saxony, Germany (20 cm ground sampling distance (GSD)) and evaluated on eight geographically distinct test tiles, the student achieves state-of-the-art (SOTA) zero-shot CHM performance (MedAE 4.17 m, $R^2$=0.51, IoU 0.87), outperforming HRCHM/DAC baselines by 29–46% in MAE (5.81 m vs. 8.14–10.84 m) with stronger correlation coefficients (0.713 vs. 0.166–0.652). Ablations show that multi-modal fusion improves performance by 10–26% over RGBI-only training, and that asymmetric distillation with appropriate model capacity is critical. The method jointly predicts CHM, PAI, and FHD, a multi-metric capability not provided by current monocular CHM estimators, although PAI/FHD transfer remains region-dependent and benefits from local calibration. The framework also remains effective under temporal mismatch (winter LiDAR, summer RGBI), removing strict co-acquisition constraints and enabling scalable 20 cm operational monitoring for workflows such as Digital Twin Germany and national Digital Orthophoto programs.

评分理由: 该论文研究的是高光谱图像分类（HSIC），提出了一种基于余弦归一化注意力的Transformer改进方法。论文的核心是计算机视觉和遥感领域的深度学习应用，具体针对高光谱数据的特性改进注意力机制。所有关键词中，只有"AI for Science" OR “Bioinformatics” OR “Cheminformatics"与论文有一定关联，因为高光谱图像分类属于科学应用（遥感、地球科学），但论文并未明确提及生物信息学或化学信息学，因此给予5分（有一定关联）。其他所有关键词均与论文内容完全无关，论文未涉及任何大语言模型（LLM）、模型训练技术（如预训练、微调、对齐）、推理优化、代理系统、模型压缩等主题。

基于Transformer的方法通过建模长距离空谱依赖关系改进了高光谱图像分类（HSIC）；然而，其注意力机制通常依赖于点积相似度，这种计算混合了特征幅值与方向信息，可能对高光谱数据并非最优。本研究从几何角度重新审视注意力评分机制，引入了一种余弦归一化注意力公式，使相似度计算与高光谱特征的角度结构对齐。通过将查询和键嵌入投影到单位超球面并应用平方余弦相似度，所提方法强调角度关系，同时降低对幅值变化的敏感性。该公式被集成到一个空谱Transformer中，并在极有限监督条件下进行评估。在三个基准数据集上的实验表明，所提方法始终取得更高性能，尽管使用轻量级骨干网络，仍优于多个近期基于Transformer和Mamba的模型。此外，对多种注意力评分函数的对照分析表明，基于余弦的评分为高光谱表征学习提供了可靠的归纳偏置。

Transformer-based methods have improved hyperspectral image classification (HSIC) by modeling long-range spatial-spectral dependencies; however, their attention mechanisms typically rely on dot-product similarity, which mixes feature magnitude and orientation and may be suboptimal for hyperspectral data. This work revisits attention scoring from a geometric perspective and introduces a cosine-normalized attention formulation that aligns similarity computation with the angular structure of hyperspectral signatures. By projecting query and key embeddings onto a unit hypersphere and applying a squared cosine similarity, the proposed method emphasizes angular relationships while reducing sensitivity to magnitude variations. The formulation is integrated into a spatial-spectral Transformer and evaluated under extremely limited supervision. Experiments on three benchmark datasets demonstrate that the proposed approach consistently achieves higher performance, outperforming several recent Transformer- and Mamba-based models despite using a lightweight backbone. In addition, a controlled analysis of multiple attention score functions shows that cosine-based scoring provides a reliable inductive bias for hyperspectral representation learning.