GoLongRL：面向能力的长期上下文强化学习与多任务对齐

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

AI 导读

GoLongRL是一个全开源的长期上下文强化学习方案，聚焦于使用可验证奖励的强化学习。该工作提出了面向能力的数据构建方法，公开发布了包含23K样本的数据集、完整构建管线及训练代码。数据集依据长期上下文能力分类，涵盖9种任务类型，由真实文档生成的问答对构成；实验证明该数据集性能优于闭源的QwenLong-L1.5数据集。训练得到的Qwen3-30B-A3B模型在长期上下文任务上达到了与DeepSeek-R1-0528等先进模型可比的性能。此外，提出了TMN-Reweight多任务优化方法，通过任务级归一化和难度自适应加权，在提升平均性能的同时保持或增强了通用能力。

推荐理由

开源长上下文RL的配方直接放出来了，数据集+代码全都有。更狠的是单靠数据多样性就干掉了闭源竞品，甚至摸到了DeepSeek-R1的水平，做长上下文的值得复现。

正文 · 原文

We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL methods often treat data construction as a matter of designing increasingly complex retrieval paths, leading to homogeneous task coverage and reward formulations that inadequately reflect practical long-context requirements. Our work offers two contributions. (1) Capability-oriented data construction with full open release. We openly release a dataset of 23K RLVR samples, the complete construction pipeline, and all training code. Guided by a taxonomy of long-context capabilities, the dataset spans 9 task types, each paired with its natural evaluation metric. It comprises curated open-source samples from established corpora and synthetic samples whose QA pairs are generated from real source documents such as books, academic papers, and multi-turn dialogues. Under the same vanilla GRPO setup, our dataset alone outperforms the closed-source QwenLong-L1.5 dataset. Moreover, our Qwen3-30B-A3B model trained on this data delivers long-context performance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, suggesting that broader coverage and greater reward diversity substantially benefit long-context capability improvement. (2) TMN-Reweight for heterogeneous multitask optimization. To address optimization challenges from heterogeneous rewards, we propose TMN-Reweight, which combines task-level mean normalization for cross-task reward scale alignment with difficulty-adaptive weighting for more reliable advantage estimation. TMN-Reweight further improves average performance over vanilla GRPO, with general capabilities preserved or improved across reported evaluations.