谷歌上周正式向公众发布了其首个原生多模态嵌入模型Gemini Embedding 2。该模型如同“通用翻译器”,能将文本、图像、视频和音频数据转化为独特的数字向量。其核心突破在于不再依赖关键词匹配,而是基于语义将不同模态的数据映射到同一空间,从而理解内容间的深层联系。开发者已利用该模型构建视频分析工具、视觉购物助手等应用,实现通过拍照或描述场景进行智能搜索的功能。模型现可通过Gemini API或Gemini Enterprise Agent平台使用。
Google 第一个原生多模态嵌入模型,把文本、图像、视频拉到同一个向量空间,做跨模态搜索的开发者可以不用再手动打标签了,但离「无感理解」还有距离。
Last week, we made Gemini Embedding 2, our first natively multimodal embedding model, available to the general public. Since then, developers have used it to build video analysis tools, visual shopping assistants, and more.
But you might be wondering... what is an embedding model? 🤔 Let’s break it down!
1. What is it?
Think of an embedding model as a "universal translator." It takes text, images, video, and audio data and turns them into a long string of numbers, like a unique digital fingerprint.
2. How does it work?
Historically, search has been text only. Now, instead of just matching data by keyword, Gemini Embedding 2 maps multiple modalities in the same space based on meaning. It "feels" the connection between a video of a soccer goal and the words "game-winning shot" without needing tags.
For example, "ocean" and "waves" are placed close together, but "ocean" and "toaster" are miles apart.
3. How can you use it?
Developers have been using it to incorporate smarter search functionality into their builds. This means creating tools where you can snap a photo of a product and type "find this in yellow," or search through thousands of hours of video by describing what happens in a scene.
4. Ready to try it out for yourself?
You can start using it today via the Gemini API or the Gemini Enterprise Agent Platform.
来源:Google AI · x.com