EmbeddingGemma: Powerful and Lightweight Text Representations
Authors:
Henrique Schechter Vera,
Sahil Dua,
Biao Zhang,
Daniel Salz,
Ryan Mullins,
Sindhu Raghuram Panyam,
Sara Smoot,
Iftekhar Naim,
Joe Zou,
Feiyang Chen,
Daniel Cer,
Alice Lisak,
Min Choi,
Lucas Gonzalez,
Omar Sanseviero,
Glenn Cameron,
Ian Ballantyne,
Kat Black,
Kaifeng Chen,
Weiyi Wang,
Zhe Li,
Gus Martins,
Jinhyuk Lee,
Mark Sherwood,
Juyeong Ji
, et al. (64 additional authors not shown)
Abstract:
We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoin…
▽ More
We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.
△ Less
Submitted 28 September, 2025; v1 submitted 24 September, 2025;
originally announced September 2025.
InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs
Authors:
Zhongyi Zhou,
Jing Jin,
Vrushank Phadnis,
Xiuxiu Yuan,
Jun Jiang,
Xun Qian,
Kristen Wright,
Mark Sherwood,
Jason Mayes,
Jingtao Zhou,
Yiyi Huang,
Zheng Xu,
Yinda Zhang,
Johnny Lee,
Alex Olwal,
David Kim,
Ram Iyengar,
Na Li,
Ruofei Du
Abstract:
Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In this paper, we introduce InstructPipe, an AI assistant for prototyping machine learn…
▽ More
Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In this paper, we introduce InstructPipe, an AI assistant for prototyping machine learning (ML) pipelines with text instructions. We contribute two large language model (LLM) modules and a code interpreter as part of our framework. The LLM modules generate pseudocode for a target pipeline, and the interpreter renders the pipeline in the node-graph editor for further human-AI collaboration. Both technical and user evaluation (N=16) shows that InstructPipe empowers users to streamline their ML pipeline workflow, reduce their learning curve, and leverage open-ended commands to spark innovative ideas.
△ Less
Submitted 13 March, 2025; v1 submitted 15 December, 2023;
originally announced December 2023.
