跳到正文
北京时间
原文
HuggingFace Daily Papers(社区热门论文)·· 2026-06-03精选AI 评分79

Echo-Infinity:学习演化记忆实现实时无限视频生成

Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation

AI 导读

Echo-Infinity 是一个自回归(AR)框架,用于实时无限视频生成。它用可学习的演化记忆替代人工缓存策略,通过注意力机制和门控更新 Memory Query,与视频扩散 Transformer(DiTs)端到端优化,支持任意压缩比且计算量不随视频长度增加。同时引入 Unified Relative RoPE Recipe,锚定 sink 帧从 id 0 开始、最新帧 id 不超过预训练最大时间 RoPE id,解除有限 RoPE 约束并缩小外推差距。在长/短视频生成中达到 SOTA,首次实现 24 小时(超 130 万帧)实时滚动生成。

推荐理由

论文把长视频生成的记忆机制从手动压缩换成了可学习的动态演化,首次做到24小时实时无限生成,这对视频生成走出‘短视频玩具’阶段是个决定性的信号。

正文 · 原文

We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation that employs a learnable evolving memory to dynamically filter, abstract, and compress any-length history at constant cost. Existing methods mainly curate memory with predefined KV-cache schedules, fixed-ratio heuristic compression, or inference-time RoPE adaptation. These designs inevitably lose historical information and amplify compounding errors due to their limited cache window and ignorance of autoregressive generation noise. Inspired by human memory consolidation, Echo-Infinity replaces handcrafted memory curation with learnable Memory Query, which are updated by attention and a gating mechanism when past frames are evicted from the local window. The queries are optimized end-to-end with the video diffusion transformers (DiTs), forming an evolving memory that supports arbitrary compression ratios with constant computation independent of video length. They also act as a generalizable generation prior, improving quality even when only the optimized initial state is used. We further introduce Unified Relative RoPE Recipe, which anchors the sink frames to start from id 0 and lets the newest frame id grow at most to the DiTs' pretrained maximum temporal RoPE id throughout training and inference, freeing the model from the finite RoPE constraint and closing the train-test RoPE extrapolation gap. In long and short video generation, Echo-Infinity achieves state-of-the-art performance, and, to our knowledge, demonstrates promising 24-hour (>1.3 M frames) real-time rollouts for the first time, suggesting a practical path toward infinite video generation.

来源:HuggingFace Daily Papers(社区热门论文) · arxiv.org