个性化幻觉:LLM 如何编造用户画像,以及为何自我监控会误导
The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads
一项新研究揭示,个性化大语言模型普遍存在过度推断(OI)现象,即编造超出证据支持的用户属性。在 MirageBench 基准测试中,12 个模型均有 35%–49% 的推断被判定为虚构(均值 41.6%)。更关键的是,模型自我评估的 OI 与外部评测结果呈负相关(rho = -0.60),表明自我报告的可信度是误导性信号,外部验证才是更可靠的个性化基础。
12个主流LLM在个性化时平均有41.8%的推断是凭空编造的,且模型自己报告的过度推断程度与实际测得的程度呈负相关,打破了依赖自审来评估模型可信度的做法。
Personalized LLMs with persistent memory are increasingly deployed, yet the faithfulness of their user models remains unexamined. We study over-inference (OI): the phenomenon where LLMs fabricate user attributes beyond what evidence supports. We introduce MirageBench, comprising 150 personas balanced across stereotypical, counter-stereotypical, and neutral profiles, 6 personalization tasks spanning an ``imagination gradient'', a four-way faithfulness taxonomy operationalized by an independent judge (validated against a blind human annotator on 400 claims: Cohen's kappa = 0.863 four-class, kappa = 0.900 binary), and a leaderboard of 12 models across 7 families on 143616 judged claims. We find that over-inference is pervasive: every one of the 12 models over-infers 35%--49% of its claims (cross-model mean 41.6%; claim-weighted 41.8%), with no model in this evaluation escaping it. Most strikingly, we surface a Self-Monitoring Inversion: at the model-selection level, models' self-assessed OI is negatively rank-correlated with their judge-measured OI (rho = -0.60, p = 0.044; exploratory, wide bootstrap CI [-0.90, +0.06], n = 12). The models that report the least over-inference tend to be flagged as fabricating the most, so self-reported confidence is a misleading signal for comparing models, even though within a single model self-audit still ranks that model's own claims moderately well (AUROC 0.58--0.83). We further show that OI is task-dependent (27%--59%) and that, in a multi-turn pilot, inferred attributes accumulate approximately linearly with little revision. MirageBench positions external verification, rather than model self-report, as a more reliable foundation for trustworthy personalization.
来源:HuggingFace Daily Papers(社区热门论文) · arxiv.org