演讲者以“Robotics: Endgame”为题,提出解决物理AGI的路线图,直接类比LLM的成功路径。核心观点包括视频世界模型作为第二预训练范式、世界行动模型(WAM)、机器人数据收集策略(类似FSD的物理数据飞轮)、EgoScale和灵巧性缩放定律、物理强化学习 bridging the last mile,以及DreamDojo端到端神经物理引擎。预测物理AGI的实现比预期更近,并提及2016年参与OpenAI DGX-1签署与Jensen和Elon的个人经历。
Jim Fan 这 20 分钟把机器人做成了 LLM 的平行故事,从 World Action Models 到 Dexterity Scaling Law,信息密度大到建议 0.5 倍速,做硬件的该换地图了。
I promise this will be the best 20 min you spend today! Robotics: Endgame, the sequel to my last year's Sequoia AI Ascent talk, "Physical Turing Test". I laid out the roadmap for solving Physical AGI as a simple parallel to the LLM success story. Be a good scientist, copy homework ;)
And stay till the end, more easter eggs and predictions for your polymarket!
00:30 DGX-1 origin story at OpenAI, I was there in 2016 signing with Jensen and Elon. Heading to the Computer History Museum!
01:42 The Great Parallel
03:31 Robotics, the Endgame
03:39 Why VLAs fall short
04:32 Video world models as the 2nd pretraining paradigm
06:09 World Action Models (WAM)
07:46 Strategies for robot data collection and the FSD equivalent to physical data flywheel for robot manipulation
11:06 EgoScale and the Dexterity Scaling Law we discovered recently
14:00 Physical RL: bridging the last mile
15:39 DreamDojo: an end-to-end neural physics engine for scaling RL in silico
17:00 Civilizational Technology Tree and my predictions for the near future. Spoiler: it's closer than you think.
Thanks to my friends at Sequoia for inviting me back to AI Ascent this year! I had a blast! Last year's talk is attached in the thread if you missed it.
来源:Jim Fan · x.com