Velox：学习4D几何与外观的表示
Velox: Learning Representations of 4D Geometry and Appearance
Velox提出一个学习4D对象潜在表示的框架，该表示具备描述性、压缩性与易获取性。它仅需非结构化动态点云作为输入，通过编码器将时空彩色点云压缩为动态形状标记，并利用两个互补解码器进行监督：4D表面解码器建模随时间变化的表面分布以捕捉几何信息，高斯解码器则负责外观重建。该方法在保持高保真度的同时提升了下游任务的效率。
苹果把动态点云的几何和外观塞进一个可压缩的latent space，思路干净但领域垂直，做3D视觉和AR的可以跟一下，其他人不用急着读。
AuthorsAnagh Malik†, Dorian Chan, Xiaoming Zhao, David B. Lindell†, Oncel Tuzel, Jen-Hao Rick Chang
We introduce a framework for learning latent representations of 4D objects which are descriptive, faithfully capturing object geometry and appearance; compressive, aiding in downstream efficiency; and accessible, requiring minimal input, i.e., an unstructured dynamic point cloud, to construct. Specifically, Velox trains an encoder to compress spatiotemporal color point clouds into a set of dynamic shape tokens. These tokens are supervised using two complementary decoders: a 4D surface decoder, which models the time-varying surface distribution capturing the geometry; and a Gaussian decoder, which maps the tokens to 3D Gaussians, helping learn appearance. To demonstrate the utility of our representation, we evaluate it across three downstream tasks — video-to-4D generation, 3D tracking, and cloth simulation via image-to-4D generation — and observe strong performances in all settings.
- † University of Toronto
来源：Apple Machine Learning Research（RSS） · machinelearning.apple.com