Agentset

1.9k

Gemini text-embedding-004 vs Gemini Embedding 2

Detailed comparison between Gemini text-embedding-004 and Gemini Embedding 2. See which embedding best meets your accuracy and performance needs. If you want to compare these models on your data, try Agentset.

Explore Gemini text-embedding-004Explore Gemini Embedding 2

Model Comparison

Gemini Embedding 2 takes the lead.

Both Gemini text-embedding-004 and Gemini Embedding 2 are powerful embedding models designed to improve retrieval quality in RAG applications. However, their performance characteristics differ in important ways.

Why Gemini Embedding 2:

  • Gemini Embedding 2 has 239 higher ELO rating
  • Gemini Embedding 2 delivers better accuracy (nDCG@10: 0.628 vs 0.538)
  • Gemini text-embedding-004 is 419ms faster on average
  • Gemini Embedding 2 has a 31.1% higher win rate

Overview

Key metrics

ELO Rating

Overall ranking quality

Gemini text-embedding-004

1366

Gemini Embedding 2

1605

Win Rate

Head-to-head performance

Gemini text-embedding-004

28.4%

Gemini Embedding 2

59.5%

Accuracy (nDCG@10)

Ranking quality metric

Gemini text-embedding-004

0.538

Gemini Embedding 2

0.628

Average Latency

Response time

Gemini text-embedding-004

16ms

Gemini Embedding 2

435ms

Embedding Models Are Just One Piece of RAG

Agentset gives you a managed RAG pipeline with the top-ranked models and best practices baked in. No infrastructure to maintain, no embeddings to manage.

Get started freeSchedule demo

Trusted by teams building production RAG applications

5M+
Documents
1,500+
Teams
99.9%
Uptime

Visual Performance Analysis

Performance

ELO Rating Comparison

Win/Loss/Tie Breakdown

Accuracy Across Datasets (nDCG@10)

Latency Distribution (ms)

Breakdown

How the models stack up

MetricGemini text-embedding-004Gemini Embedding 2Description
Overall Performance
ELO Rating
1366
1605
Overall ranking quality based on pairwise comparisons
Win Rate
28.4%
59.5%
Percentage of comparisons won against other models
Pricing & Availability
Price per 1M tokens
$0.020
$0.000
Cost per million tokens processed
Dimensions
768
3072
Vector embedding dimensions (lower is more efficient)
Release Date
2024-05-14
2026-03-10
Model release date
Accuracy Metrics
Avg nDCG@10
0.538
0.628
Normalized discounted cumulative gain at position 10
Performance Metrics
Avg Latency
16ms
435ms
Average response time across all datasets

Build RAG in Minutes, Not Months

Agentset gives you a complete RAG API with top-ranked embedding models and smart retrieval built in. Upload your data, call the API, and get accurate results from day one.

Get started freeRead the docs 
import { Agentset } from "agentset";

const agentset = new Agentset();
const ns = agentset.namespace("ns_1234");

const results = await ns.search(
  "What is multi-head attention?"
);

for (const result of results) {
  console.log(result.text);
}

Dataset Performance

By field

Comprehensive comparison of accuracy metrics (nDCG, Recall) and latency percentiles for each benchmark dataset.

business reports

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.000
0.091
Ranking quality at top 5 results
nDCG@10
0.000
0.084
Ranking quality at top 10 results
Recall@5
0.000
0.012
% of relevant docs in top 5
Recall@10
0.000
0.020
% of relevant docs in top 10
Latency Metrics
Mean
15ms
439ms
Average response time
P50
15ms
431ms
50th percentile (median)
P90
15ms
603ms
90th percentile

DBPedia

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.747
0.788
Ranking quality at top 5 results
nDCG@10
0.737
0.792
Ranking quality at top 10 results
Recall@5
0.057
0.061
% of relevant docs in top 5
Recall@10
0.108
0.120
% of relevant docs in top 10
Latency Metrics
Mean
14ms
436ms
Average response time
P50
14ms
432ms
50th percentile (median)
P90
14ms
592ms
90th percentile

FiQa

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.744
0.843
Ranking quality at top 5 results
nDCG@10
0.730
0.835
Ranking quality at top 10 results
Recall@5
0.647
0.763
% of relevant docs in top 5
Recall@10
0.752
0.816
% of relevant docs in top 10
Latency Metrics
Mean
16ms
466ms
Average response time
P50
16ms
454ms
50th percentile (median)
P90
16ms
605ms
90th percentile

SciFact

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.728
0.871
Ranking quality at top 5 results
nDCG@10
0.729
0.871
Ranking quality at top 10 results
Recall@5
0.813
0.959
% of relevant docs in top 5
Recall@10
0.857
0.959
% of relevant docs in top 10
Latency Metrics
Mean
15ms
404ms
Average response time
P50
15ms
360ms
50th percentile (median)
P90
15ms
537ms
90th percentile

MSMARCO

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.932
0.956
Ranking quality at top 5 results
nDCG@10
0.918
0.939
Ranking quality at top 10 results
Recall@5
0.117
0.122
% of relevant docs in top 5
Recall@10
0.208
0.221
% of relevant docs in top 10
Latency Metrics
Mean
18ms
441ms
Average response time
P50
18ms
446ms
50th percentile (median)
P90
18ms
584ms
90th percentile

ARCD

MetricGemini text-embedding-004Gemini Embedding 2Description
Accuracy Metrics
nDCG@5
0.021
0.868
Ranking quality at top 5 results
nDCG@10
0.027
0.875
Ranking quality at top 10 results
Recall@5
0.040
0.940
% of relevant docs in top 5
Recall@10
0.060
0.960
% of relevant docs in top 10
Latency Metrics
Mean
15ms
410ms
Average response time
P50
15ms
359ms
50th percentile (median)
P90
15ms
586ms
90th percentile

Explore More

Compare more embeddings

See how all embedding models stack up. Compare OpenAI, Cohere, Jina AI, Voyage, and more. View comprehensive benchmarks, compare performance metrics, and find the perfect embedding for your RAG application.

View Full LeaderboardAll Leaderboards