Google has introduced Gemini Embedding, an AI-powered text-processing model now integrated into the Gemini API.
The model has claimed the top spot on the Massive Text Embedding Benchmark (MTEB), surpassing rivals such as Mistral, Cohere, and Qwen in various natural language processing (NLP) tasks.
Gemini Embedding and other embedding models convert text into numerical representations (vectors) to enable semantic search, recommendation systems, and document retrieval. They allow for smarter search rankings (like Google Search), AI-powered customer support retrieval, document clustering, and recommendation engines.
According to Google, “The Gemini embedding model family achieves state-of-the-art results on the Massive Text Embedding Benchmark (MTEB), outperforming all available text embedding models across retrieval, clustering, classification, and reranking tasks.”
MTEB Benchmark Relevance
As AI-powered search and NLP technologies become increasingly sophisticated, benchmarks like MTEB serve as critical evaluation tools. Created by Hugging Face, MTEB tests AI models on more than 50 datasets, assessing their ability to rank, categorize, and retrieve textual data.
The MTEB leaderboard, an industry-standard ranking for AI embedding models, evaluates performance in retrieval, classification, clustering, and reranking tasks. Gemini Embedding achieved a mean task score of 68.32, outperforming Linq-Embed-Mistral and gte-Qwen2-7B-instruct, both of which scored in the low 60s.
Key results included an 85.13 in pair classification, 67.71 in retrieval, and 65.58 in reranking, making it the highest-performing text embedding model currently available.
Higher scores on this benchmark indicate improved performance in real-world applications such as AI-powered search engines, document analysis, and chatbot optimization.
Companies looking to integrate AI into their platforms often rely on these scores to determine which model best suits their needs. Google’s current leadership in this space signals its push to make Gemini Embedding a preferred solution for AI-driven text processing.
How Gemini Embedding Could Reshape AI Search and Business Applications
Google’s success in MTEB rankings signals broader implications for AI-powered search and enterprise solutions. Embedding models serve as the foundation for search ranking algorithms, recommendation engines, and chatbot responses.
A model with high retrieval and classification scores enhances AI’s ability to generate more relevant search results, making its impact especially valuable for Google Search and other AI-driven services.
Google’s advancements will be key for providing relevant search results as the company is expanding the use of AI powered search results. The company is currently testing a new AI mode for Google Search, which provides purely AI-driven search results that replace traditional links with AI-generated answers.
Beyond search, Gemini Embedding’s multilingual proficiency positions it as a tool for improving cross-language applications. AI models that perform well in retrieval tasks are crucial for businesses that operate in multiple languages, as they help enhance translation accuracy, customer service automation, and content ranking.
This makes Gemini Embedding a potentially useful asset for industries such as e-commerce, legal documentation, and technical support.
Enterprise clients using Google Cloud AI solutions may see improvements in AI-powered analytics, semantic search within databases, and automated data retrieval for research and business intelligence.
The model’s ability to outperform competitors in ranking and clustering tasks suggests that businesses relying on AI-driven content organization could benefit from its integration into cloud-based AI services.
Google’s AI Strategy: Competing Against Open-Source Alternatives
Google has been refining its text embedding models for years, but previous iterations, including text-multilingual-embedding-002, struggled to maintain dominance over emerging open-source alternatives.
Unlike open-source models, which offer greater customization and transparency, Google’s proprietary solution is integrated directly into the Gemini API, making it a seamless option for enterprises already using its cloud-based AI tools. However, the rapid advancements from competitors suggest that future MTEB benchmarks may become even more competitive.
Although Google currently leads in MTEB rankings, the AI text embedding space remains competitive, particularly with open-source alternatives challenging proprietary models. Companies like Cohere and Mistral have rapidly gained traction, offering transparency and flexibility that some enterprises prefer over closed-source solutions.
The main advantage of proprietary models like Gemini Embedding lies in their deep integration with Google’s broader AI ecosystem. However, open-source models provide greater adaptability for businesses that require specialized implementations. The question moving forward is whether Google can sustain its leadership in AI text processing as competition intensifies.
AI Model Benchmarks – LLM Leaderboard
Last updated: Mar 16, 2025Benchmark stats come from the model providers, if available. For models with optional advanced reasoning, we provide the highest benchmark score achieved.
| Organization | Model | Context | Parameters (B) | Input $/M | Output $/M | License | GPQA | MMLU | MMLU Pro | DROP | HumanEval | AIME’24 | SimpleBench | Model |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| openai | o3 | 128,000 | – | – | – | Proprietary | 87.70% | – | – | – | – | o3 | ||
| anthropic | Claude 3.7 Sonnet | 200,000 | – | $3.00 | $15.00 | Proprietary | 84.80% | 86.10% | – | – | – | 80.00% | 46.4% | Claude 3.7 Sonnet |
| xai | Grok-3 | 128,000 | – | – | – | Proprietary | 84.60% | – | 79.90% | – | – | 93.30% | Grok-3 | |
| xai | Grok-3 Mini | 128,000 | – | – | – | Proprietary | 84.60% | – | 78.90% | – | – | 90.80% | Grok-3 Mini | |
| openai | o3-mini | 200,000 | – | $1.10 | $4.40 | Proprietary | 79.70% | 86.90% | – | – | – | 86.50% | 22.8% | o3-mini |
| openai | o1-pro | 128,000 | – | – | – | Proprietary | 79.00% | – | – | – | – | 86.00% | o1-pro | |
| openai | o1 | 200,000 | – | $15.00 | $60.00 | Proprietary | 78.00% | 91.80% | – | – | 88.10% | 83.30% | 40.1% | o1 |
| Gemini 2.0 Flash Thinking | 1,000,000 | – | – | – | Proprietary | 74.20% | – | – | – | – | 73.30% | 30.7% | Gemini 2.0 Flash Thinking | |
| openai | o1-preview | 128,000 | – | $15.00 | $60.00 | Proprietary | 73.30% | 90.80% | – | – | – | 44.60% | 41.7% | o1-preview |
| deepseek | DeepSeek-R1 | 131,072 | 671 | $0.55 | $2.19 | Open | 71.50% | 90.80% | 84.00% | 92.20% | – | 79.80% | 30.9% | DeepSeek-R1 |
| openai | GPT-4.5 | 128,000 | – | – | – | Proprietary | 71.4% | 90.0% | – | – | 88.0% | 36.7% | 34.5% | GPT-4.5 |
| anthropic | Claude 3.5 Sonnet | 200,000 | – | $3.00 | $15.00 | Proprietary | 67.20% | 90.40% | 77.60% | 87.10% | 93.70% | 16.00% | 41.4% | Claude 3.5 Sonnet |
| qwen | QwQ-32B-Preview | 32,768 | 32.5 | $0.15 | $0.20 | Open | 65.20% | – | 70.97% | – | – | 50.00% | QwQ-32B-Preview | |
| Gemini 2.0 Flash | 1,048,576 | – | – | – | Proprietary | 62.10% | – | 76.40% | – | – | 35.5% | 18.9% | Gemini 2.0 Flash | |
| openai | o1-mini | 128,000 | – | $3.00 | $12.00 | Proprietary | 60.00% | 85.20% | 80.30% | – | 92.40% | 70.00% | 18.1% | o1-mini |
| deepseek | DeepSeek-V3 | 131,072 | 671 | $0.27 | $1.10 | Open | 59.10% | 88.50% | 75.90% | 91.60% | – | 39.2% | 18.9% | DeepSeek-V3 |
| Gemini 1.5 Pro | 2,097,152 | – | $2.50 | $10.00 | Proprietary | 59.10% | 85.90% | 75.80% | 74.90% | 84.10% | 19.3% | 27.1% | Gemini 1.5 Pro | |
| microsoft | Phi-4 | 16,000 | 14.7 | $0.07 | $0.14 | Open | 56.10% | 84.80% | 70.40% | 75.50% | 82.60% | Phi-4 | ||
| xai | Grok-2 | 128,000 | – | $2.00 | $10.00 | Proprietary | 56.00% | 87.50% | 75.50% | – | 88.40% | 22.7% | Grok-2 | |
| openai | GPT-4o | 128,000 | – | $2.50 | $10.00 | Proprietary | 53.60% | 88.00% | 74.70% | – | – | 17.8% | GPT-4o | |
| Gemini 1.5 Flash | 1,048,576 | – | $0.15 | $0.60 | Proprietary | 51.00% | 78.90% | 67.30% | – | 74.30% | Gemini 1.5 Flash | |||
| xai | Grok-2 mini | 128,000 | – | – | – | Proprietary | 51.00% | 86.20% | 72.00% | – | 85.70% | Grok-2 mini | ||
| meta | Llama 3.1 405B Instruct | 128,000 | 405 | $0.90 | $0.90 | Open | 50.70% | 87.30% | 73.30% | 84.80% | 89.00% | 23.0% | Llama 3.1 405B Instruct | |
| meta | Llama 3.3 70B Instruct | 128,000 | 70 | $0.20 | $0.20 | Open | 50.50% | 86.00% | 68.90% | – | 88.40% | 19.9% | Llama 3.3 70B Instruct | |
| anthropic | Claude 3 Opus | 200,000 | – | $15.00 | $75.00 | Proprietary | 50.40% | 86.80% | 68.50% | 83.10% | 84.90% | 23.5% | Claude 3 Opus | |
| qwen | Qwen2.5 32B Instruct | 131,072 | 32.5 | – | – | Open | 49.50% | 83.30% | 69.00% | – | 88.40% | Qwen2.5 32B Instruct | ||
| qwen | Qwen2.5 72B Instruct | 131,072 | 72.7 | $0.35 | $0.40 | Open | 49.00% | – | 71.10% | – | 86.60% | 23.30% | Qwen2.5 72B Instruct | |
| openai | GPT-4 Turbo | 128,000 | – | $10.00 | $30.00 | Proprietary | 48.00% | 86.50% | – | 86.00% | 87.10% | GPT-4 Turbo | ||
| amazon | Nova Pro | 300,000 | – | $0.80 | $3.20 | Proprietary | 46.90% | 85.90% | – | 85.40% | 89.00% | Nova Pro | ||
| meta | Llama 3.2 90B Instruct | 128,000 | 90 | $0.35 | $0.40 | Open | 46.70% | 86.00% | – | – | – | Llama 3.2 90B Instruct | ||
| qwen | Qwen2.5 14B Instruct | 131,072 | 14.7 | – | – | Open | 45.50% | 79.70% | 63.70% | – | 83.50% | Qwen2.5 14B Instruct | ||
| mistral | Mistral Small 3 | 32,000 | 24 | $0.07 | $0.14 | Open | 45.30% | – | 66.30% | – | 84.80% | Mistral Small 3 | ||
| qwen | Qwen2 72B Instruct | 131,072 | 72 | – | – | Open | 42.40% | 82.30% | 64.40% | – | 86.00% | Qwen2 72B Instruct | ||
| amazon | Nova Lite | 300,000 | – | $0.06 | $0.24 | Proprietary | 42.00% | 80.50% | – | 80.20% | 85.40% | Nova Lite | ||
| meta | Llama 3.1 70B Instruct | 128,000 | 70 | $0.20 | $0.20 | Open | 41.70% | 83.60% | 66.40% | 79.60% | 80.50% | Llama 3.1 70B Instruct | ||
| anthropic | Claude 3.5 Haiku | 200,000 | – | $0.10 | $0.50 | Proprietary | 41.60% | – | 65.00% | 83.10% | 88.10% | Claude 3.5 Haiku | ||
| anthropic | Claude 3 Sonnet | 200,000 | – | $3.00 | $15.00 | Proprietary | 40.40% | 79.00% | 56.80% | 78.90% | 73.00% | Claude 3 Sonnet | ||
| openai | GPT-4o mini | 128,000 | – | $0.15 | $0.60 | Proprietary | 40.20% | 82.00% | – | 79.70% | 87.20% | 10.7% | GPT-4o mini | |
| amazon | Nova Micro | 128,000 | – | $0.04 | $0.14 | Proprietary | 40.00% | 77.60% | – | 79.30% | 81.10% | Nova Micro | ||
| Gemini 1.5 Flash 8B | 1,048,576 | 8 | $0.07 | $0.30 | Proprietary | 38.40% | – | 58.70% | – | – | Gemini 1.5 Flash 8B | |||
| ai21 | Jamba 1.5 Large | 256,000 | 398 | $2.00 | $8.00 | Open | 36.90% | 81.20% | 53.50% | – | – | Jamba 1.5 Large | ||
| microsoft | Phi-3.5-MoE-instruct | 128,000 | 60 | – | – | Open | 36.80% | 78.90% | 54.30% | – | 70.70% | Phi-3.5-MoE-instruct | ||
| qwen | Qwen2.5 7B Instruct | 131,072 | 7.6 | $0.30 | $0.30 | Open | 36.40% | – | 56.30% | – | 84.80% | Qwen2.5 7B Instruct | ||
| xai | Grok-1.5 | 128,000 | – | – | – | Proprietary | 35.90% | 81.30% | 51.00% | – | 74.10% | Grok-1.5 | ||
| openai | GPT-4 | 32,768 | – | $30.00 | $60.00 | Proprietary | 35.70% | 86.40% | – | 80.90% | 67.00% | 25.1% | GPT-4 | |
| anthropic | Claude 3 Haiku | 200,000 | – | $0.25 | $1.25 | Proprietary | 33.30% | 75.20% | – | 78.40% | 75.90% | Claude 3 Haiku | ||
| meta | Llama 3.2 11B Instruct | 128,000 | 10.6 | $0.06 | $0.06 | Open | 32.80% | 73.00% | – | – | – | Llama 3.2 11B Instruct | ||
| meta | Llama 3.2 3B Instruct | 128,000 | 3.2 | $0.01 | $0.02 | Open | 32.80% | 63.40% | – | – | – | Llama 3.2 3B Instruct | ||
| ai21 | Jamba 1.5 Mini | 256,144 | 52 | $0.20 | $0.40 | Open | 32.30% | 69.70% | 42.50% | – | – | Jamba 1.5 Mini | ||
| openai | GPT-3.5 Turbo | 16,385 | – | $0.50 | $1.50 | Proprietary | 30.80% | 69.80% | – | 70.20% | 68.00% | GPT-3.5 Turbo | ||
| meta | Llama 3.1 8B Instruct | 131,072 | 8 | $0.03 | $0.03 | Open | 30.40% | 69.40% | 48.30% | 59.50% | 72.60% | Llama 3.1 8B Instruct | ||
| microsoft | Phi-3.5-mini-instruct | 128,000 | 3.8 | $0.10 | $0.10 | Open | 30.40% | 69.00% | 47.40% | – | 62.80% | Phi-3.5-mini-instruct | ||
| Gemini 1.0 Pro | 32,760 | – | $0.50 | $1.50 | Proprietary | 27.90% | 71.80% | – | – | – | Gemini 1.0 Pro | |||
| qwen | Qwen2 7B Instruct | 131,072 | 7.6 | – | – | Open | 25.30% | 70.50% | 44.10% | – | – | Qwen2 7B Instruct | ||
| mistral | Codestral-22B | 32,768 | 22.2 | $0.20 | $0.60 | Open | – | – | – | – | 81.10% | Codestral-22B | ||
| cohere | Command R+ | 128,000 | 104 | $0.25 | $1.00 | Open | – | 75.70% | – | – | – | 17.4% | Command R+ | |
| deepseek | DeepSeek-V2.5 | 8,192 | 236 | $0.14 | $0.28 | Open | – | 80.40% | – | – | 89.00% | DeepSeek-V2.5 | ||
| Gemma 2 27B | 8,192 | 27.2 | – | – | Open | – | 75.20% | – | – | 51.80% | Gemma 2 27B | |||
| Gemma 2 9B | 8,192 | 9.2 | – | – | Open | – | 71.30% | – | – | 40.20% | Gemma 2 9B | |||
| xai | Grok-1.5V | 128,000 | – | – | – | Proprietary | – | – | – | – | – | Grok-1.5V | ||
| moonshotai | Kimi-k1.5 | 128,000 | – | – | – | Proprietary | – | 87.40% | – | – | – | Kimi-k1.5 | ||
| nvidia | Llama 3.1 Nemotron 70B Instruct | 128,000 | 70 | – | – | Open | – | 80.20% | – | – | – | Llama 3.1 Nemotron 70B Instruct | ||
| mistral | Ministral 8B Instruct | 128,000 | 8 | $0.10 | $0.10 | Open | – | 65.00% | – | – | 34.80% | Ministral 8B Instruct | ||
| mistral | Mistral Large 2 | 128,000 | 123 | $2.00 | $6.00 | Open | – | 84.00% | – | – | 92.00% | 22.5% | Mistral Large 2 | |
| mistral | Mistral NeMo Instruct | 128,000 | 12 | $0.15 | $0.15 | Open | – | 68.00% | – | – | – | Mistral NeMo Instruct | ||
| mistral | Mistral Small | 32,768 | 22 | $0.20 | $0.60 | Open | – | – | – | – | – | Mistral Small | ||
| microsoft | Phi-3.5-vision-instruct | 128,000 | 4.2 | – | – | Open | – | – | – | – | – | Phi-3.5-vision-instruct | ||
| mistral | Pixtral-12B | 128,000 | 12.4 | $0.15 | $0.15 | Open | – | 69.20% | – | – | 72.00% | Pixtral-12B | ||
| mistral | Pixtral Large | 128,000 | 124 | $2.00 | $6.00 | Open | – | – | – | – | – | Pixtral Large | ||
| qwen | QvQ-72B-Preview | 32,768 | 73.4 | – | – | Open | – | – | – | – | – | QvQ-72B-Preview | ||
| qwen | Qwen2.5-Coder 32B Instruct | 128,000 | 32 | $0.09 | $0.09 | Open | – | 75.10% | 50.40% | – | 92.70% | Qwen2.5-Coder 32B Instruct | ||
| qwen | Qwen2.5-Coder 7B Instruct | 128,000 | 7 | – | – | Open | – | 67.60% | 40.10% | – | 88.40% | Qwen2.5-Coder 7B Instruct | ||
| qwen | Qwen2-VL-72B-Instruct | 32,768 | 73.4 | – | – | Open | – | – | – | – | – | Qwen2-VL-72B-Instruct | ||
| cohere | Command A | 256,000 | 111 | $2.50 | $10.00 | Open | – | 85.00% | – | – | – | – | – | Command A |
| baidu | ERNIE 4.5 | – | – | – | – | – | 75.00% | – | 79.00% | 87.00% | 85.00% | ERNIE 4.5 | ||
| Gemma 3 1B | 128,000 | 1 | – | – | Open | 19.20% | 29.90% | 14.70% | – | 32.00% | – | – | Gemma 3 1B | |
| Gemma 3 4B | 128,000 | 4 | – | – | Open | 30.80% | 46.90% | 43.60% | – | – | – | – | Gemma 3 4B | |
| Gemma 3 12B | 128,000 | 12 | – | – | Open | 40.90% | 65.20% | 60.60% | – | – | – | – | Gemma 3 12B | |
| Gemma 3 27B | 128,000 | 27 | – | – | Open | 42.40% | 72.1% | 67.50% | – | 89.00% | – | – | Gemma 3 27B | |
| qwen | Qwen2.5 Max | 32,768 | – | 59.00% | – | 76.00% | – | 93.00% | 23.00% | – | Qwen2.5 Max | |||
| qwen | QwQ 32B | 131,000 | 32.8 | Open | 59.00% | – | 76.00% | 98.00% | 78.00% | – | QwQ 32B |
