Citation and evidence

Dense retrieval

Report a problem with this article

Edit article
More

Cite this article

Dense retrieval is an approach to information retrieval in which queries and documents are encoded by a neural network into low-dimensional, real-valued vectors (embeddings), and documents are ranked by the similarity of their vectors to the query vector, usually a dot product or cosine similarity [1][2]. Because document vectors can be computed ahead of time and stored in an index, retrieval reduces to finding the nearest neighbors of the query vector, which libraries such as FAISS do efficiently over millions or billions of vectors [1][28]. The approach is usually contrasted with sparse retrieval, the term-matching tradition of TF-IDF and BM25, which represents text as high-dimensional, mostly zero vectors over a vocabulary and scores documents through an inverted index [1][3].

Dense representations for retrieval go back to latent semantic analysis in 1990, but until pretrained transformer encoders a dense representation used alone was typically inferior to a sparse one [1]. The 2020 paper on Dense Passage Retrieval (DPR) from Facebook AI showed that a pair of BERT encoders trained on a modest number of question-passage pairs could beat a strong Lucene BM25 system by 9 to 19 percentage points in top-20 passage retrieval accuracy on most of the open-domain question answering benchmarks it tested, the exception being SQuAD [1]. Later work concentrated on how to train such encoders (hard negatives, distillation, retrieval-oriented pretraining, large-scale contrastive learning) and on why they often generalize worse than BM25 to new domains [2][13]. A dense retriever supplied the documents in the 2020 paper that introduced retrieval-augmented generation (RAG), and in RAG practice dense retrieval is often combined with BM25 in hybrid search [30][31].

Definition and terminology

A 2022 survey by Wayne Xin Zhao and colleagues describes dense retrieval as using pretrained language models to learn representations of queries and texts in a latent space and to build a semantic matching function between the resulting dense vectors; the name comes from the use of dense vectors (embeddings) to represent texts [2]. The survey organizes the field around four aspects: architecture, training, indexing, and integration with the rest of a retrieval pipeline [2].

Jimmy Lin proposed in 2021 a framework that places dense and sparse methods in one picture. A retrieval method has a logical scoring model, made of encoders that map queries and documents into a representation space plus a comparison function that scores a query-document pair, and a physical retrieval model that returns the top-k documents from a large corpus. Scoring models then vary along two dimensions: dense versus sparse representations, and supervised (learned) versus unsupervised [3]. In that view BM25 is an unsupervised sparse model, DPR is a supervised dense model, and learned sparse models such as SPLADE are supervised sparse models [3][25].

Several overlapping terms are in use. "Dual encoder", "bi-encoder" and "BERT-Siamese" model all describe the common architecture in which query and document are encoded separately [1][8][15][30]. "Embedding-based retrieval" and "vector search" describe the same idea from the angle of the embeddings and the index [22][29].

How dense retrieval works

Encoders and scoring

DPR, the reference design, uses two independent BERT-base encoders: a question encoder $E_Q$ and a passage encoder $E_P$, each producing the vector at the [CLS] token, so each embedding has $d = 768$ dimensions [1]. Similarity is the inner product

sim(q,p)=EQ(q)⊤EP(p)\mathrm{sim}(q, p) = E_Q(q)^{\top} E_P(p)

where $q$ is a question and $p$ a passage [1]. The DPR authors note that more expressive scoring functions exist, such as networks with several layers of cross-attention, but that the function must be decomposable for passage vectors to be precomputed. In their ablations, L2 distance performed comparably to the dot product and both did better than cosine similarity [1].

This separation is what distinguishes a dense retriever from a cross-encoder, which reads the query and document together. DPR's own reader uses cross-attention to select among retrieved passages and the authors describe it as having more capacity than the dual encoder, but cross-attention is not feasible for retrieving from a large corpus because it is not decomposable. Applied to a small number of retrieved candidates, it works well [1]. Pipelines therefore pair a fast retriever with a cross-encoder that re-ranks a short list, as in the BM25 plus cross-encoder system that BEIR evaluated [1][13].

At indexing time every passage is passed through the document encoder once and the vectors are stored in a nearest-neighbor index. DPR indexed about 21 million Wikipedia passages with FAISS, using an HNSW index on CPU [1]. At query time only the query is encoded, and the index returns the $k$ passages whose vectors score highest, a maximum inner product search [1]. FAISS, described in a 2017 paper by Jeff Johnson, Matthijs Douze and Hervé Jégou, supports exact search, approximate nearest-neighbor search and compressed search based on product quantization, and its authors reported building a k-nearest-neighbor graph over 1 billion vectors in under 12 hours on four GPUs [28][29]. Vector similarity search of this kind is the core function of vector databases [29].

The cost profile differs from sparse retrieval. In the DPR paper, the dense index answered 995 questions per second (top 100 passages each) on a server CPU, while BM25 in Lucene handled 23.7 questions per second per CPU thread. Building the dense index was much slower: encoding 21 million passages took about 8.8 hours on 8 GPUs and building the FAISS index took 8.5 hours on one server, against about 30 minutes for a Lucene inverted index [1]. In the BEIR benchmark's measurements on 1 million DBPedia documents, the 768-dimensional dense models needed about 3 GB of index against 0.4 GB for BM25, and ColBERT, which stores many vectors per document, needed 20 GB [13].

Training objective

Training a dense retriever is a metric learning problem: the encoders are adjusted so that relevant query-passage pairs score higher than irrelevant ones [1]. For a question $q_i$ with one relevant passage $p_i^{+}$ and $n$ irrelevant passages $p_{i,1}^{-}, \dots, p_{i,n}^{-}$, DPR minimizes the negative log-likelihood of the positive passage:

L=−log⁡esim(qi,pi+)esim(qi,pi+)+∑j=1nesim(qi,pi,j−)L = -\log \frac{e^{\mathrm{sim}(q_i, p_i^{+})}}{e^{\mathrm{sim}(q_i, p_i^{+})} + \sum_{j=1}^{n} e^{\mathrm{sim}(q_i, p_{i,j}^{-})}}

This is a softmax over one positive and $n$ negatives, the same form as the InfoNCE loss used in contrastive learning; Contriever's version also divides each score by a temperature parameter $\tau$ [1][14].

Positive passages are usually given by the dataset; the choice of negatives is the main design decision [1]. DPR compared random passages, BM25 passages that match the question's words but do not contain the answer, and "gold" passages that are positives for other questions [1]. It used in-batch negatives, a trick earlier applied to dual encoders in full-batch and mini-batch training: with $B$ questions in a mini-batch, the $B \times B$ matrix of question-passage scores gives each question $B - 1$ negatives at almost no extra cost [1]. DPR's best model combined in-batch negatives with one BM25 hard negative per question [1].

Dense and sparse retrieval compared

AspectSparse retrieval (e.g. BM25)Dense retrieval (e.g. DPR)
RepresentationHigh-dimensional, mostly zero term-weight vectorsLow-dimensional real vectors, 768 dimensions in DPR
MatchingExact term overlapLearned similarity; can match synonyms and paraphrases
TrainingNone (BM25)Needs labeled or weakly supervised pairs
IndexInverted indexNearest-neighbor vector index
Index build (DPR paper, 21M passages)About 30 minutesAbout 8.8 h encoding on 8 GPUs plus 8.5 h FAISS build
Index size (BEIR, 1M documents)0.4 GBAbout 3 GB
Out-of-domain robustness (BEIR, 2021)Strong baselineOften below BM25

Expanded article table

Sources: [1][13].

The DPR paper gives an example of the difference. For the question "Who is the bad guy in lord of the rings?", the answer passage reads "Sala Baker is best known for portraying the villain Sauron in the Lord of the Rings trilogy." A term-based system would have difficulty retrieving it, while a dense model can learn that "bad guy" and "villain" mean the same thing [1]. In the other direction, the authors found that BM25 is sensitive to highly selective keywords and phrases, and that DPR did worse than BM25 on SQuAD, where annotators wrote questions after reading the passage and so copied many of its words [1]. A 2024 Anthropic engineering post made the same point for RAG: embedding models capture semantic relationships but "can miss crucial exact matches" [31].

The two signals are often combined. DPR's own hybrid took the top 2,000 passages from each of BM25 and DPR and re-ranked their union by $\mathrm{BM25}(q, p) + \lambda \cdot \mathrm{sim}(q, p)$ with $\lambda = 1.1$, which improved results on some datasets [1]. Yi Luan and colleagues also studied sparse-dense hybrids as a way to recover the precision of sparse retrieval [24].

History

Before transformers

Dense vectors for retrieval date at least to latent semantic analysis (Deerwester and colleagues, 1990), and discriminatively trained dense encoders using labeled query-document pairs appeared in the early 2010s [1]. The Deep Structured Semantic Model (DSSM) from Microsoft Research, presented at CIKM 2013, mapped queries and documents into a shared low-dimensional space with deep networks trained on search-engine clickthrough data, and used a "word hashing" technique to cope with large web vocabularies [4].

Learned retrieval for question answering (2019-2020)

The Open Retrieval Question Answering (ORQA) model of Kenton Lee, Ming-Wei Chang and Kristina Toutanova (ACL 2019) trained a retriever and a reader jointly from question-answer pairs alone, treating the evidence passage from all of Wikipedia as a latent variable. Because that was impractical to learn from scratch, the retriever was first pretrained with an Inverse Cloze Task (ICT). ORQA beat BM25 by up to 19 points in exact match on datasets where the asker genuinely sought an answer, while BM25 sufficed where the questioner already knew it [5]. REALM (Guu and colleagues, February 2020) extended the idea to language model pretraining, backpropagating through a retrieval step over millions of documents with masked language modeling as the training signal [6].

DPR, posted in April 2020 and published at EMNLP 2020, asked whether a better dense model could be trained from question-passage pairs alone, without ICT pretraining, which its authors called computationally intensive. Fine-tuning two BERT encoders was enough to beat BM25 by a wide margin (65.2% against 42.9% top-5 accuracy on Natural Questions) and raised end-to-end exact match on open Natural Questions to 41.5%, against 33.3% for ORQA; the authors concluded that additional pretraining may not be needed [1]. The same month, Omar Khattab and Matei Zaharia introduced ColBERT (SIGIR 2020), which keeps one vector per token and scores with a "late interaction" step; they reported quality competitive with BERT re-rankers while running two orders of magnitude faster [7].

Training improvements (2020-2021)

MethodFirst postedMain ideaSource
ANCE2020-07-01Hard negatives from an asynchronously refreshed ANN index of the corpus[8]
RocketQA2020-10-16Cross-batch negatives, denoised hard negatives, data augmentation[9]
TAS-Balanced2021-04-14Topic-aware batch sampling and distillation from two teachers[10]
Condenser2021-04-16Pretraining architecture that conditions LM prediction on the dense vector[11]
coCondenser2021-08-12Adds an unsupervised corpus-level contrastive loss to Condenser[12]
Contriever2021-12-16Unsupervised contrastive training with random cropping and MoCo[14]
GTR2021-12-15Scaling T5 dual encoders while keeping 768-dimension embeddings[15]

Expanded article table

ANCE, from Microsoft (ICLR 2021), argued that the main bottleneck was training: negatives sampled within a batch are mostly uninformative, which leads to diminishing gradient norms and slow convergence. It instead drew negatives from an approximate nearest-neighbor index of the whole corpus, refreshed asynchronously as the model trained, and reported nearly matching a BM25 plus BERT re-ranking pipeline with about a 100-fold speed-up. The authors also tested it in the first-stage retrieval of a commercial search engine [8]. RocketQA, from Baidu and Renmin University (NAACL 2021), named three obstacles (the gap between training and inference, unlabeled positives that end up sampled as negatives, and limited training data) and answered them with cross-batch negatives, denoised hard negatives and data augmentation [9].

TAS-Balanced (SIGIR 2021) targeted compute cost. It clustered queries by topic before training, sampled each batch from one cluster, and trained a 6-layer model with supervision from both a pairwise teacher and an in-batch negative teacher, a form of knowledge distillation. It could be trained on one consumer-grade GPU in under 48 hours, against a common configuration of eight V100s, and beat BM25 by 44% in nDCG@10 on the TREC Deep Learning query sets [10]. Luyu Gao and Jamie Callan's Condenser (EMNLP 2021) argued that a standard language model's attention structure is not ready to aggregate a text into one dense vector and proposed a pretraining architecture for that purpose [11]. Their coCondenser reached results comparable to RocketQA with simple small-batch fine-tuning, without heavy data engineering [12].

Generalization and benchmarks

BEIR

The BEIR benchmark (Thakur and colleagues, NeurIPS 2021 Datasets and Benchmarks track) gathered 18 public datasets across diverse retrieval tasks and domains to test zero-shot, out-of-distribution performance [13]. Its central finding was that BM25 is a robust baseline. Dense retrievers trained on MS MARCO, such as ANCE and TAS-B, did well on some datasets but much worse than BM25 on others, especially under domain shift (BioASQ) or task shift (Touché-2020); DPR, trained on Natural Questions rather than MS MARCO, generalized worst. Re-ranking with a cross-encoder and the late-interaction ColBERT generalized best, at much higher computational cost [13]. Among dense models TAS-B did best, which the authors speculated was due to its training setup combining in-batch negatives with distillation from strong teachers [13].

BEIR also measured a bias against dense models in the evaluation data. Many datasets were annotated from candidates found by lexical systems, so a relevant document with no word overlap may never have been judged. On TREC-COVID, 31.8% of TAS-B's top 10 results and 14.4% of ANCE's had not been seen by annotators, against 6.4% for BM25; after the authors judged the missing pairs (980 in total), the dense models' scores rose much more than the lexical ones [13].

Closing the gap

Later work used BEIR as the target. Contriever (Izacard and colleagues, published in TMLR in 2022) trained a retriever without any labels, using random spans cropped from the same document as positive pairs and MoCo to keep a queue of negatives. The unsupervised model outperformed BM25 on 11 of 15 BEIR datasets in Recall@100, and it also supported cross-lingual retrieval, such as English documents from Arabic queries, which term matching cannot do [14]. GTR, from Google Research, challenged the belief that the single-vector bottleneck limits out-of-domain generalization: it scaled T5-based dual encoders up to 4.8 billion parameters while keeping the embedding at 768 dimensions, pretrained on roughly 2 billion community question-answer pairs. Larger encoders brought significant improvements, especially out of domain, and the GTR models outperformed existing sparse and dense retrievers on BEIR [15]. E5 (Microsoft, December 2022) trained on CCPairs, a set of about 270 million text pairs filtered from about 1.3 billion web pairs, and was described by its authors as the first model to beat BM25 on BEIR zero-shot without labeled data [16].

The Massive Text Embedding Benchmark (MTEB), introduced in October 2022 with 8 task types, 58 datasets and 112 languages, broadened evaluation from retrieval to embeddings in general. Its authors found that no single embedding method dominated across all tasks [17].

LLM-based embedders

From 2023, embedding models were increasingly built on decoder-only large language models. A Microsoft team (ACL 2024) fine-tuned open LLMs with a standard contrastive loss on synthetic data that proprietary LLMs generated for hundreds of thousands of embedding tasks in 93 languages, using fewer than 1,000 training steps, and reported state-of-the-art BEIR and MTEB results when mixing synthetic and labeled data [19]. Google's Gemini Embedding (March 2025) was derived from the Gemini model and evaluated on the multilingual MMTEB benchmark [20]. Alibaba's Qwen3 Embedding series from Tongyi Lab (June 2025), in 0.6B, 4B and 8B sizes, used Qwen3 LLMs both as backbones and to synthesize training data [21].

  • Multi-vector (late interaction). ColBERT keeps an embedding for every token; each query token embedding is compared with all document token embeddings through a MaxSim operator, and the results are summed across query terms, keeping finer-grained matching while documents stay precomputable [7]. ColBERTv2 (NAACL 2022) used residual compression to cut the storage footprint of late interaction by 6 to 10 times [26]. Luan and colleagues' ME-BERT also represents a document with several vectors [24].
  • Learned sparse retrieval. SPLADE (SIGIR 2021) learns sparse term weights, including expansion terms, so that an inverted index can be used, aiming to keep the exact matching and efficiency of bag-of-words models [25].
  • Query-side generation. HyDE prompts an instruction-following LLM to write a hypothetical answer document and retrieves with that document's embedding from an unsupervised encoder such as Contriever, so no relevance labels are needed [18].
  • Variable-size embeddings. Matryoshka Representation Learning trains one embedding that encodes information at several granularities, so a single vector can be adapted to different computational budgets; its authors reported up to 14-fold speed-ups for large-scale retrieval on ImageNet [27].

Limitations

Out-of-domain generalization. BEIR showed that early dense retrievers often fell below BM25 on unseen domains and tasks [13]. Later models narrowed this gap, and E5 was reported as the first to beat BM25 on BEIR without labeled data [16]; in practice, embeddings are still often combined with BM25, which Anthropic found worked better than embeddings alone [31].

Index size and dimension. Nils Reimers and Iryna Gurevych (ACL 2021) showed theoretically and empirically that dense retrieval quality falls faster than sparse retrieval quality as the index grows, and that lower dimensionality raises the chance of false positives; in extreme cases sparse retrieval overtakes dense retrieval past some index size [23]. Luan and colleagues linked the encoding dimension, the margin between relevant and lower-ranked documents, and document length, suggesting that fixed-length encodings have limited capacity for precise retrieval of long documents [24].

A dimension bound on what can be retrieved. A 2025 paper by Orion Weller and colleagues at Google DeepMind and Johns Hopkins University (ICLR 2026) argued that the number of distinct top-k sets a single-vector model can return is limited by its embedding dimension [22]. Their LIMIT dataset uses very simple queries of the form "who likes X" over 50,000 short documents. Strong embedding models struggled to reach 20% Recall@100, while BM25 reached 93.6% and the multi-vector GTE-ModernColBERT reached 54.8% [22]. The authors caution that BM25's advantage comes from its very high dimensionality and exact word matching: when items were replaced with synonyms, BM25 fell further than the neural models. They present multi-vector, sparse and cross-encoder models as alternatives with trade-offs rather than a solution [22].

Exact terms and cost. Embeddings can miss exact matches such as identifiers [31], and indexing requires encoding the whole corpus with a neural network and storing a larger index than an inverted index [1][13].

Use in retrieval-augmented generation

The 2020 paper that named retrieval-augmented generation (Lewis and colleagues, NeurIPS 2020) used DPR as its retriever. The non-parametric memory was a dense vector index of 21 million 100-word Wikipedia chunks in FAISS, and only the query encoder was fine-tuned with the generator, because updating the document encoder would require periodically rebuilding the index, as REALM does [30].

Practitioners often pair dense retrieval with BM25 and a re-ranker. Anthropic's September 2024 "Contextual Retrieval" write-up, which retrieves documents split into chunks and was tested on codebases, fiction, arXiv papers and science papers, reported that embeddings plus BM25 did better than embeddings alone. Prepending a short, model-generated, chunk-specific context to each chunk before embedding it and before building the BM25 index cut the top-20 retrieval failure rate by 49%, and by 67% when a re-ranker was added [31]. Techniques such as HyDE target the case where no relevance labels exist for the target corpus [18].

Continue reading

More from See also

More to read

Topics:Information RetrievalMachine LearningNatural Language Processing

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih. "Dense Passage Retrieval for Open-Domain Question Answering". EMNLP 2020 (arXiv:2004.04906), 2020. arxiv.org/...2004.04906
  2. ^1 ^2 ^3 ^4Wayne Xin Zhao, Jing Liu, Ruiyang Ren, Ji-Rong Wen. "Dense Text Retrieval based on Pretrained Language Models: A Survey". arXiv:2211.14876, 2022. arxiv.org/...2211.14876
  3. ^1 ^2 ^3Jimmy Lin. "A Proposed Conceptual Framework for a Representational Approach to Information Retrieval". SIGIR Forum 55(2) (arXiv:2110.01529), 2021. arxiv.org/...2110.01529
  4. ^Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck. "Learning Deep Structured Semantic Models for Web Search using Clickthrough Data". CIKM 2013, Microsoft Research, 2013. microsoft.com/...eb-search-using-clickthrough-data
  5. ^Kenton Lee, Ming-Wei Chang, Kristina Toutanova. "Latent Retrieval for Weakly Supervised Open Domain Question Answering". ACL 2019 (arXiv:1906.00300), 2019. arxiv.org/...1906.00300
  6. ^Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang. "REALM: Retrieval-Augmented Language Model Pre-Training". arXiv:2002.08909, 2020. arxiv.org/...2002.08909
  7. ^1 ^2Omar Khattab, Matei Zaharia. "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT". SIGIR 2020 (arXiv:2004.12832), 2020. arxiv.org/...2004.12832
  8. ^1 ^2 ^3Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, Arnold Overwijk. "Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval". ICLR 2021 (arXiv:2007.00808), 2020. openreview.net/forum
  9. ^1 ^2Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang. "RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering". NAACL 2021 (arXiv:2010.08191), 2020. arxiv.org/...2010.08191
  10. ^1 ^2Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, Allan Hanbury. "Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling". SIGIR 2021 (arXiv:2104.06967), 2021. arxiv.org/...2104.06967
  11. ^1 ^2Luyu Gao, Jamie Callan. "Condenser: a Pre-training Architecture for Dense Retrieval". EMNLP 2021 (arXiv:2104.08253), 2021. arxiv.org/...2104.08253
  12. ^1 ^2Luyu Gao, Jamie Callan. "Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval". arXiv:2108.05540, 2021. arxiv.org/...2108.05540
  13. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, Iryna Gurevych. "BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models". NeurIPS 2021 Datasets and Benchmarks Track (arXiv:2104.08663), 2021. arxiv.org/...2104.08663
  14. ^1 ^2 ^3Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, Edouard Grave. "Unsupervised Dense Information Retrieval with Contrastive Learning". Transactions on Machine Learning Research (arXiv:2112.09118), 2022. arxiv.org/...2112.09118
  15. ^1 ^2 ^3Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming-Wei Chang, Yinfei Yang. "Large Dual Encoders Are Generalizable Retrievers". arXiv:2112.07899, 2021. arxiv.org/...2112.07899
  16. ^1 ^2Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei. "Text Embeddings by Weakly-Supervised Contrastive Pre-training". arXiv:2212.03533, 2022. arxiv.org/...2212.03533
  17. ^Niklas Muennighoff, Nouamane Tazi, Loïc Magne, Nils Reimers. "MTEB: Massive Text Embedding Benchmark". arXiv:2210.07316, 2022. arxiv.org/...2210.07316
  18. ^1 ^2Luyu Gao, Xueguang Ma, Jimmy Lin, Jamie Callan. "Precise Zero-Shot Dense Retrieval without Relevance Labels". arXiv:2212.10496, 2022. arxiv.org/...2212.10496
  19. ^Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei. "Improving Text Embeddings with Large Language Models". ACL 2024 (arXiv:2401.00368), 2023. arxiv.org/...2401.00368
  20. ^Jinhyuk Lee, Feiyang Chen, Sahil Dua, Daniel Cer, Madhuri Shanbhogue, Iftekhar Naim, Gustavo Hernández Ábrego, Zhe Li, Kaifeng Chen, Henrique Schechter Vera, Xiaoqi Ren, Shanfeng Zhang, et al. "Gemini Embedding: Generalizable Embeddings from Gemini". arXiv:2503.07891, 2025. arxiv.org/...2503.07891
  21. ^Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, Jingren Zhou. "Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models". arXiv:2506.05176, 2025. arxiv.org/...2506.05176
  22. ^1 ^2 ^3 ^4Orion Weller, Michael Boratko, Iftekhar Naim, Jinhyuk Lee. "On the Theoretical Limitations of Embedding-Based Retrieval". ICLR 2026 (arXiv:2508.21038), 2025. arxiv.org/...2508.21038
  23. ^Nils Reimers, Iryna Gurevych. "The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes". ACL 2021 (arXiv:2012.14210), 2020. arxiv.org/...2012.14210
  24. ^1 ^2 ^3Yi Luan, Jacob Eisenstein, Kristina Toutanova, Michael Collins. "Sparse, Dense, and Attentional Representations for Text Retrieval". Transactions of the Association for Computational Linguistics (arXiv:2005.00181), 2020. arxiv.org/...2005.00181
  25. ^1 ^2Thibault Formal, Benjamin Piwowarski, Stéphane Clinchant. "SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking". SIGIR 2021 (arXiv:2107.05720), 2021. arxiv.org/...2107.05720
  26. ^Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, Matei Zaharia. "ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction". NAACL 2022 (arXiv:2112.01488), 2021. arxiv.org/...2112.01488
  27. ^Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, Ali Farhadi. "Matryoshka Representation Learning". arXiv:2205.13147, 2022. arxiv.org/...2205.13147
  28. ^1 ^2Jeff Johnson, Matthijs Douze, Hervé Jégou. "Billion-scale similarity search with GPUs". arXiv:1702.08734, 2017. arxiv.org/...1702.08734
  29. ^1 ^2 ^3Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, Hervé Jégou. "The Faiss library". arXiv:2401.08281, 2024. arxiv.org/...2401.08281
  30. ^1 ^2 ^3Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks". NeurIPS 2020 (arXiv:2005.11401), 2020. arxiv.org/...2005.11401
  31. ^1 ^2 ^3 ^4 ^5Anthropic. "Introducing Contextual Retrieval". Anthropic, 2024-09-19. anthropic.com/...contextual-retrieval

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

1 revision · v1 · 4,171 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Cite this page: AI Wiki. "Dense retrieval." aiwiki.ai, updated 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/dense_retrieval

Edit article