Embeddings

In machine learning, an embedding is a learned representation that maps an input into a vector, usually so that a downstream model or comparison rule can use relationships encoded in the vector's geometry. If the input domain is X, an encoder can be written as:

$$f_{\theta}: X \rightarrow \mathbb{R}^{d}.$$

The output f_\theta(x) is an embedding vector, and the set of possible outputs is an embedding space. The map may encode a token, sentence, document, image, audio segment, graph node, user, item, or another object. What proximity means depends on the training objective, data, encoder, pooling procedure, normalization, and comparison function. An embedding is therefore not automatically semantic, interpretable, lower-dimensional than its input, or suitable for every task.

This article concerns embeddings as a general technique in representation learning. It does not treat word embeddings, sentence and document encoders, contextual token states, multimodal representations, and hosted embedding APIs as interchangeable. Those systems differ in what they encode, how many vectors they return, which similarity rule they expect, and whether their vector spaces are compatible.

Definition and scope

The term "embedding" is used at several levels of precision. In a strict mathematical sense, an embedding is an injective structure-preserving map. Machine-learning usage is looser: a learned vector representation is often called an embedding even when distinct inputs can map to the same vector, when information is deliberately discarded, or when the output dimension is larger than a simple input feature count. The relevant structure is the one induced by the model and its objective, not a guarantee that every property of the input is preserved.

For a fixed encoder and configuration, a single-vector embedding system typically returns an element of R^d. The coordinates may be stored as floating-point values or as a quantized representation. Individual coordinates do not generally have stable human-readable meanings. If every vector is multiplied by the same orthogonal matrix Q, then:

$$(Qx)^\mathsf{T}(Qy)=x^\mathsf{T}Q^\mathsf{T}Qy=x^\mathsf{T}y,$$

and Euclidean distances are also preserved. Consequently, many geometrically equivalent coordinate systems can support the same comparisons. An axis should not be labeled as a concept unless that interpretation has been established for the particular model and analysis.

An embedding layer is one way to obtain vectors, but it is not the whole category. A lookup layer associates a learned vector with each discrete identifier. An encoder network instead computes a representation from an input and can produce different vectors for the same token in different contexts. Some systems return one vector per input, some return one per token or patch, and some retain multiple vectors for late interaction. Calling all of these outputs "embeddings" does not make their shapes or uses identical.

Representation types

Scroll sideways for more →

The page Vector embeddings discusses vector-valued representations as a broad data type. The present page focuses on how learned embeddings are constructed, compared, evaluated, and deployed.

How embeddings are learned

An embedding objective defines which distinctions should be retained. There is no task-independent rule requiring nearby vectors to mean "similar" in an ordinary-language sense.

Objectives define the geometry

Predictive objectives make a representation useful for predicting some target. In the skip-gram formulation, for example, a center word is used to predict words that occur in its local context.^[2] A language model instead uses a representation to predict tokens under its sequence objective.^[1][7] A recommender may optimize observed user-item interactions.^[15] The resulting vectors need retain only the information useful for that training signal.

Contrastive objectives start from designated positive and negative pairs. One common per-example form is:

$$\mathcal{L}_i = -\log \frac{\exp(s(z_i,z_i^+)/\tau)} {\sum_{j}\exp(s(z_i,z_j)/\tau)},$$

where z_i^+ is a positive for anchor z_i, s is the chosen score, and tau is a temperature. The denominator's contents depend on the method: other batch elements, sampled negatives, or a memory structure may be used. SimCLR applied this family of objectives to augmented views of images, while SimCSE constructed sentence pairs from dropout views or labeled natural-language-inference examples.^[9][10]

Positive and negative construction is part of the model specification. An image crop teaches invariance to the crop only when the training setup treats it as another view of the same item. A query paired with a relevant passage teaches a retrieval relationship. A randomly sampled "negative" may actually be relevant, producing a false-negative training signal. Hard-negative mining can make training more discriminating, but its sampling source and relevance assumptions need to be reported.

Supervised objectives can also shape an embedding without using an explicit contrastive loss. A classifier's hidden layer, a matrix-factorization model's latent factors, or a token lookup table trained through a larger network may all be called embeddings. Their geometry is an outcome of the full objective and regularization, not a separately guaranteed semantic map.

Jointly learned lookup tables

For discrete inputs, a model can learn a matrix E in R^(V times d), where row E_i represents identifier i. The embedding receives gradients through the task that uses it. In a language model, for example, word representations can be learned jointly with the probability model rather than supplied as fixed features. Bengio and colleagues' 2003 neural probabilistic language model learned a distributed representation for each word together with a model of word-sequence probability, allowing related representations to share statistical strength.^[1]

The same mechanism appears in recommenders and other systems with categorical IDs. A matrix-factorization recommender represents users and items by latent-factor vectors, then predicts interactions from those factors. The factors are useful because they optimize a specified prediction problem, not because each coordinate has an inherent universal meaning.^[15]

Distributional word objectives

word2vec introduced efficient continuous bag-of-words and skip-gram architectures for learning word vectors from local contexts.^[2] A follow-up described subsampling and negative sampling and examined compositional phrase representations.^[3] These models helped popularize vector regularities, but their static word-type vectors are only one branch of embedding research.

GloVe trains a weighted least-squares model on global word-word co-occurrence counts, connecting corpus-level statistics with vector-space structure.^[4] fastText represents a word as a sum of character n-gram vectors, allowing it to construct vectors for some words absent from the training vocabulary and to use subword morphology.^[5] These objectives preserve different evidence, so matching dimensions do not make their vectors interchangeable.

Contextual representations

ELMo made each token representation a function of the entire input sentence through a deep bidirectional language model.^[6] BERT learned deep bidirectional contextual representations with masked-language-model pretraining.^[7] In both cases, the output for a token depends on context. A raw token-state sequence is not automatically a strong single-vector representation of the whole sentence.

Sentence-BERT addressed that distinction by adapting BERT with siamese and triplet structures and explicit pooling to produce fixed-size sentence vectors for similarity and retrieval.^[8] SimCSE used contrastive objectives for sentence embeddings, with dropout-based positive pairs in its unsupervised variant and natural-language-inference pairs in its supervised variant.^[9] Research on unadapted BERT sentence vectors found anisotropic geometry and poor direct semantic-similarity behavior under the tested pooling setup, which is one reason pooling and objective choice must be reported rather than hidden behind the word "embedding."^[24]

Contrastive and multimodal objectives

Contrastive learning trains representations by increasing agreement for designated positive pairs relative to negatives. The choice of views and augmentations defines what the representation should treat as invariant. SimCLR showed, for its visual self-supervised setting, that augmentation composition and a learned projection head materially affected the learned representation.^[10]

CLIP trained separate image and text encoders on image-text pairs so corresponding images and descriptions could be compared in a joint space.^[11] ImageBind aligned six modalities using images as a binding modality, showing in its experiments that every pair of modalities did not need direct paired training data.^[12] These results demonstrate learned cross-modal alignment. They do not imply that vectors from arbitrary image, text, or audio models share a space.

Graph and relational objectives

Graph embeddings encode selected structural relationships. TransE modeled a relation as a translation between entity vectors for multi-relational data.^[13] node2vec learned node representations by optimizing a neighborhood-preservation objective over biased random walks.^[14] The two objectives are not equivalent: one targets labeled relational triples, while the other samples network neighborhoods. Applications involving a knowledge graph should identify the relation model and negative-sampling procedure rather than referring only to "graph embeddings."

Geometry and comparison

An embedding model should be paired with the comparison function used during training or specified by its documentation. Common choices include dot product, cosine similarity, and Euclidean distance.

For nonzero vectors x and y, cosine similarity is:

$$\operatorname{cos}(x,y) = \frac{x^\mathsf{T}y}{\lVert x\rVert_2\lVert y\rVert_2}.$$

Euclidean distance is:

$$d_2(x,y)=\lVert x-y\rVert_2.$$

If both vectors have unit length, then:

$$\lVert x-y\rVert_2^2 = \lVert x\rVert_2^2+\lVert y\rVert_2^2-2x^\mathsf{T}y = 2-2x^\mathsf{T}y.$$

Thus, for unit-normalized vectors, maximizing dot product, maximizing cosine similarity, and minimizing Euclidean distance produce the same ranking. OpenAI's current embedding FAQ states that its API embeddings are L2-normalized by default, including after shortening, and therefore have this ranking equivalence.^[29] The result should not be generalized to every model: some models use vector magnitude, require normalization after truncation, or specify another score.

A similarity score has no universal interpretation across models, versions, tasks, or corpora. A threshold that works for duplicate detection in one distribution may fail for classification or retrieval in another. Thresholds should be selected on held-out data that reflects the intended operating conditions, and reports should include both the score rule and any normalization.

Vector dimensions are also not portable identifiers. Two encoders can both return 768 numbers while assigning unrelated geometry to them. Even versions from the same provider may be incompatible. Google's current documentation explicitly states that gemini-embedding-001 and gemini-embedding-2 occupy incompatible spaces and require existing data to be re-embedded during migration.^[30] Compatibility should be assumed only when a model publisher documents it or when an alignment procedure has been validated.

Pooling, context, and instructions

Turning variable-length input into one vector requires a design choice. Text encoders may use a designated token, mean pooling over token states, weighted pooling, or a learned aggregation head. Different layers can also produce different representations. Sentence-BERT reported that directly using common BERT pooling choices yielded weak sentence embeddings in its comparison, then trained a siamese architecture for efficient sentence-level comparison.^[8] The more general lesson is not that one pooling rule always wins, but that a token encoder and a sentence encoder solve different problems.

Tokenization and context limits affect what is represented. If an input exceeds a model's limit, an interface may reject it, truncate it, or require chunking. Chunk boundaries can separate a statement from its qualifier, a definition from its term, or a table row from its header. For retrieval, useful chunking is therefore an empirical document-processing decision, not a fixed character count that transfers across domains.

Some retrieval encoders are asymmetric. Dense Passage Retrieval used separate question and passage encoders, trained with question-passage pairs, to support maximum-inner-product retrieval.^[16] Hosted services may expose this distinction through query and document input types or through text instructions. Voyage's current interface, for example, can prepend different prompts when input_type is query or document.^[32] Omitting or changing such instructions can change vectors even when the underlying text is unchanged.

Multi-vector systems are another distinct case. ColBERT independently encodes query and document tokens and applies a late-interaction rule instead of reducing each document to one vector before comparison.^[17] A storage or search component built for one vector per item cannot reproduce that scoring rule without retaining the required local representations.

Choosing the encoded unit

The encoded unit controls what the index can retrieve. A document-level vector is economical but can blur several unrelated topics. A sentence-level vector is more specific but can lose surrounding definitions, qualifications, and references. Token-level or multi-vector systems retain finer interactions at greater storage and scoring cost.^[17] There is no generally correct chunk size.

For structured material, boundaries should follow the material when possible. Headings, paragraphs, table rows with their headers, code blocks with nearby explanation, and speaker turns can carry relationships that fixed-length slicing destroys. Overlap can preserve some boundary context, but also creates near-duplicate candidates and changes retrieval metrics. The chosen unit should be evaluated with real queries and relevance judgments.

Long-input support does not remove this decision. A model that accepts an entire document can produce one vector for it, but that vector may not expose a small fact strongly enough for nearest-neighbor retrieval. Conversely, splitting every sentence can deprive the generator or reranker of necessary context. Systems often retrieve compact units and then expand to a parent section, or use a second-stage reranker, but those are application designs rather than properties of the embedding alone.

Retrieval and indexing

Dense embeddings are widely used in information retrieval. A dual encoder can precompute document vectors, embed a query, and retrieve items with high similarity. This architecture makes query-time comparison cheaper than applying a full cross-encoder to every query-document pair, though a cross-encoder can still rerank a smaller candidate set.

For N stored vectors of dimension d, an exact linear scan evaluates N similarities and has work proportional to Nd. Approximate nearest-neighbor indexes trade exactness, memory, build time, and update cost for lower query latency. HNSW organizes proximity graphs in layers and performs approximate graph search.^[19] Product quantization divides vectors into subspaces and represents them with short codes so distances can be approximated with reduced storage.^[20] Neither method guarantees a fixed speedup or accuracy loss across data sets and parameter choices.

A vector database stores vectors and associated records and may provide one or more indexing methods. It does not create a meaningful embedding space by itself. The encoder, preprocessing, distance rule, index configuration, filter behavior, and evaluation corpus all contribute to system quality.

Dense retrieval is also not uniformly superior to lexical retrieval. BEIR evaluated 18 public data sets spanning diverse retrieval tasks and found substantial variation across lexical, sparse, dense, late-interaction, and reranking systems.^[23] Hybrid retrieval can be useful because token matching and learned similarity fail in different ways. Retrieval should be evaluated with task-appropriate measures such as recall at k, precision at k, mean reciprocal rank, or normalized discounted cumulative gain, followed by end-to-end evaluation of the consuming application.

Dense and lexical retrieval expose different failure modes. Dense encoders may connect paraphrases that share few tokens, as demonstrated in the Dense Passage Retrieval setting, but can miss exact identifiers, rare names, numbers, or out-of-distribution terminology.^[16][23] Lexical systems preserve exact token evidence but may miss paraphrases. Filters can also change the candidate set before or during vector search, so filtered recall should be measured rather than inferred from unfiltered tests.

Approximate search adds a second notion of recall: whether the index returned the neighbors that an exact scan under the same vector score would have returned. That ANN recall is distinct from relevance recall, which asks whether the results satisfy a human or task-specific judgment. A high ANN recall cannot repair a weak encoder, and a strong encoder can still be undermined by an overly aggressive index configuration.

In retrieval-augmented generation, embeddings may help select non-parametric context for a generative model. The original RAG work combined a sequence-to-sequence model with a dense vector index of Wikipedia.^[18] Retrieval does not guarantee that a source is correct, that the generator uses it faithfully, or that the answer is entailed by it. Grounding and citation quality require separate checks.

Dimensionality, storage, and compression

Embedding dimension d is a capacity and systems parameter, not a universal quality scale. A higher-dimensional vector uses more storage and comparison work, but may or may not improve a particular task. No evidence supports a universal claim that one range, such as 384 to 768 dimensions, is optimal for all applications.

For N uncompressed float32 vectors, raw vector storage is:

$$\text{bytes}=N \times d \times 4.$$

One million 768-dimensional float32 vectors therefore require 3,072,000,000 bytes, approximately 2.861 GiB, before index structures, identifiers, metadata, replicas, or allocator overhead. This arithmetic is a capacity estimate, not a quote for any database or hosted service.

Dimensionality reduction, including Principal Component Analysis, can compress existing vectors, but the transformed representation must be evaluated for the target task. Matryoshka Representation Learning instead trains coarse-to-fine prefix representations so selected leading dimensions can be used at multiple lengths. The original MRL paper reported large storage and retrieval gains in its own evaluated settings, but those experimental figures are not a guarantee for arbitrary models or data.^[21] Truncating an ordinary embedding that was not trained for prefix use can have different effects.

Quantization reduces the precision used to store each component. Int8 values require one byte per component, while bit-packed binary representations require one bit per component, before auxiliary data. These widths imply 4-fold and 32-fold raw-value reductions relative to float32, respectively. Retrieval quality after quantization is model-, corpus-, score-, and index-dependent. It should be measured end to end rather than represented by a universal degradation percentage.

Applications

Embeddings are useful when a downstream operation can exploit learned geometry:

• Search and retrieval: compare queries with documents, passages, images, or other indexed objects.
• Recommendation: represent users, items, or interactions in a latent space. Matrix-factorization systems are an early and still influential example.^[15]
• Classification: train a classifier on frozen or adapted representations.
• Clustering: group vectors, while remembering that clusters depend on the metric, scaling, and algorithm.
• Near-duplicate and anomaly detection: compare vectors against examples or a reference distribution using a calibrated rule.
• Cross-modal matching: retrieve an image from text or compare other modalities when the encoders were trained into a joint space.^[11][12]
• Graph prediction: use node, entity, or relation vectors for link prediction or node-level tasks.^[13][14]

An embedding is an intermediate representation, not a complete application. A search system also needs a corpus, relevance definition, index, filters, and evaluation. A recommender needs an interaction objective and a policy for new users and items. A classifier needs labels and a decision rule. Performance cannot be inferred from an embedding model's name or dimension alone.

Evaluation and model selection

Model selection should follow the intended task and deployment constraints. The original MTEB benchmark covered eight text-embedding tasks, 58 data sets, and 112 languages and reported that no evaluated method dominated every task.^[22] BEIR focused on zero-shot information retrieval across 18 heterogeneous data sets.^[23] Both results argue against choosing a model solely from one aggregate leaderboard score.

Useful evaluation dimensions include:

Scroll sideways for more →

An end-to-end comparison should hold preprocessing, chunking, task instructions, output dimension, normalization, distance, quantization, index parameters, and reranker policy constant unless a change is itself under test. It should also distinguish model inference time from index-search time and include the cost of re-embedding a corpus when a model changes.

Reproducibility record

A reproducible embedding experiment should identify:

• the exact model identifier, revision, weights, tokenizer, and inference library;
• the encoded unit and any parsing, cleaning, truncation, chunking, or overlap;
• the task instruction, input type, pooling rule, selected layer, and output dimension;
• whether vectors are normalized, shortened, transformed, or quantized;
• the similarity rule and any score threshold;
• the index algorithm, build parameters, query parameters, filters, and reranker;
• the evaluation data, relevance judgments, metrics, random seeds, and hardware;
• the treatment of failures, empty inputs, duplicates, and unsupported languages or modalities.

For an API, the request parameters and evaluation date matter because an unpinned identifier may change behavior. For a local model, a repository name without a commit or immutable revision may also be insufficient. Recording the configuration with each vector collection helps prevent accidental mixing and makes later re-embedding auditable.

Hosted interfaces snapshot

The following small snapshot was checked against official documentation on July 28, 2026. It is not an exhaustive catalog, and model identifiers, limits, and defaults are mutable product facts.

Scroll sideways for more →

These rows describe interfaces, not a controlled quality comparison. Provider benchmark claims use different models, data sets, and conditions. Pricing is intentionally omitted because it changes independently of the underlying representation.

Limitations and risks

Geometry is not truth

High similarity means that a model placed vectors near each other under its learned objective. It does not establish factual truth, logical implication, causation, or identity. A model can place two common misconceptions near each other, separate rare paraphrases, or retrieve topical but non-answering passages. Downstream systems must check the property they actually need.

Bias and uneven performance

Embeddings can reflect associations in training data. Bolukbasi and colleagues demonstrated gender stereotypes in word embeddings trained on Google News and proposed a post-processing method for the measured bias.^[25] Gonen and Goldberg later showed, for two debiasing methods, that neighborhood structure could retain recoverable gender information after the targeted metric was reduced.^[26] These studies do not show that every embedding has the same bias, but they do show why a single intrinsic score is not proof of fairness.

Evaluation should include relevant populations, languages, domains, and failure costs. Mitigation claims should identify the measured attribute, threat model, data, and downstream effect.

Privacy and inversion

A vector is not automatically anonymized merely because it is not human-readable. Morris and colleagues trained an inversion model that exactly reconstructed 92 percent of tested 32-token inputs for two text-embedding models in their experimental setting and recovered full names from embedded clinical notes.^[27] The number is specific to their models, data, access assumptions, and attack. The broader conclusion is that embeddings can retain sensitive information and should be protected according to the sensitivity of their source data.

Access controls, retention rules, encryption, logging, and deletion procedures should cover both source records and derived vectors. Releasing vectors can also expose membership, attributes, or relationships even when exact input recovery is unsuccessful.

Drift and incompatibility

Vectors can change when the model, revision, tokenizer, pooling, task instruction, dimension, normalization, or preprocessing changes. Mixing incompatible vectors can silently degrade retrieval because a database may accept equal-length arrays without knowing whether their coordinate systems match. A production index should record the full embedding configuration and use a migration plan that rebuilds or deliberately aligns vectors.

Compression and approximate search

Shortening, quantization, and approximate indexing alter the representation or the search procedure. Their errors can interact: a model may remain accurate after one change but not after several. Reported quality should describe the complete deployed pipeline, including any reranker, rather than only the encoder before compression.

Practical checklist

Before deploying an embedding pipeline:

1. Define the encoded unit and target decision. Specify whether an item is a token, span, document, media segment, entity, user, or product.
2. Pin the encoder, revision, tokenizer, pooling rule, task instruction, dimension, output type, normalization, and distance function.
3. Build evaluation data from the intended domain, language mix, temporal range, and hard negatives.
4. Compare lexical, dense, hybrid, and reranked retrieval where relevant rather than presuming one architecture wins.
5. Measure end-to-end quality and systems behavior at the planned index size.
6. Test truncation, quantization, and ANN settings together with the final score rule.
7. Evaluate sensitive-data exposure, access control, deletion, and known attack assumptions.
8. Audit performance and harmful associations across relevant slices.
9. Version stored vectors and reject incompatible writes instead of relying only on array length.
10. Plan corpus re-embedding and rollback before changing a model or configuration.

See also

• Artificial Intelligence
• Deep Learning
• Machine Learning
• Neural Network
• Large Language Model

References

1. ^Bengio, Yoshua, Rejean Ducharme, Pascal Vincent, and Christian Jauvin. "A Neural Probabilistic Language Model." *Journal of Machine Learning Research* 3, 1137-1155 (2003). jmlr.org/...bengio03a
2. ^Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean. "Efficient Estimation of Word Representations in Vector Space." arXiv:1301.3781 (2013). arxiv.org/...1301.3781
3. ^Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. "Distributed Representations of Words and Phrases and their Compositionality." *Advances in Neural Information Processing Systems* 26 (2013). proceedings.neurips.cc/...65f3c4923ce901b-Abstract
4. ^Pennington, Jeffrey, Richard Socher, and Christopher D. Manning. "GloVe: Global Vectors for Word Representation." *Proceedings of EMNLP 2014*, 1532-1543. aclanthology.org/D14-1162
5. ^Bojanowski, Piotr, Edouard Grave, Armand Joulin, and Tomas Mikolov. "Enriching Word Vectors with Subword Information." *Transactions of the Association for Computational Linguistics* 5, 135-146 (2017). aclanthology.org/Q17-1010
6. ^Peters, Matthew E., et al. "Deep Contextualized Word Representations." *Proceedings of NAACL-HLT 2018*, 2227-2237. aclanthology.org/N18-1202
7. ^Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." *Proceedings of NAACL-HLT 2019*, 4171-4186. aclanthology.org/N19-1423
8. ^Reimers, Nils, and Iryna Gurevych. "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks." *Proceedings of EMNLP-IJCNLP 2019*, 3982-3992. aclanthology.org/D19-1410
9. ^Gao, Tianyu, Xingcheng Yao, and Danqi Chen. "SimCSE: Simple Contrastive Learning of Sentence Embeddings." *Proceedings of EMNLP 2021*, 6894-6910. aclanthology.org/2021.emnlp-main.552
10. ^Chen, Ting, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. "A Simple Framework for Contrastive Learning of Visual Representations." *Proceedings of the 37th International Conference on Machine Learning*, PMLR 119, 1597-1607 (2020). proceedings.mlr.press/...chen20j
11. ^Radford, Alec, et al. "Learning Transferable Visual Models From Natural Language Supervision." *Proceedings of the 38th International Conference on Machine Learning*, PMLR 139, 8748-8763 (2021). proceedings.mlr.press/...radford21a
12. ^Girdhar, Rohit, et al. "ImageBind: One Embedding Space To Bind Them All." *Proceedings of CVPR 2023*, 15180-15190. openaccess.thecvf.com/..._Them_All_CVPR_2023_paper
13. ^Bordes, Antoine, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. "Translating Embeddings for Modeling Multi-relational Data." *Advances in Neural Information Processing Systems* 26 (2013). proceedings.neurips.cc/...3fa24680a88d2f9-Abstract
14. ^Grover, Aditya, and Jure Leskovec. "node2vec: Scalable Feature Learning for Networks." *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, 855-864 (2016). doi.org/...2939672.2939754
15. ^Koren, Yehuda, Robert Bell, and Chris Volinsky. "Matrix Factorization Techniques for Recommender Systems." *Computer* 42(8), 30-37 (2009). doi.org/...MC.2009.263
16. ^Karpukhin, Vladimir, et al. "Dense Passage Retrieval for Open-Domain Question Answering." *Proceedings of EMNLP 2020*, 6769-6781. aclanthology.org/2020.emnlp-main.550
17. ^Khattab, Omar, and Matei Zaharia. "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT." *Proceedings of SIGIR 2020*, 39-48. doi.org/...3397271.3401075
18. ^Lewis, Patrick, et al. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." *Advances in Neural Information Processing Systems* 33 (2020). proceedings.neurips.cc/...bc26945df7481e5-Abstract
19. ^Malkov, Yu A., and Dmitry A. Yashunin. "Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs." *IEEE Transactions on Pattern Analysis and Machine Intelligence* 42(4), 824-836 (2020). arxiv.org/...1603.09320
20. ^Jegou, Herve, Matthijs Douze, and Cordelia Schmid. "Product Quantization for Nearest Neighbor Search." *IEEE Transactions on Pattern Analysis and Machine Intelligence* 33(1), 117-128 (2011). doi.org/...TPAMI.2010.57
21. ^Kusupati, Aditya, et al. "Matryoshka Representation Learning." *Advances in Neural Information Processing Systems* 35 (2022). proceedings.neurips.cc/...0e42-Abstract-Conference
22. ^Muennighoff, Niklas, Nouamane Tazi, Loic Magne, and Nils Reimers. "MTEB: Massive Text Embedding Benchmark." *Proceedings of EACL 2023*, 2014-2037. aclanthology.org/2023.eacl-main.148
23. ^Thakur, Nandan, et al. "BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models." *Proceedings of NeurIPS Datasets and Benchmarks* 2021. openreview.net/forum
24. ^Li, Bohan, et al. "On the Sentence Embeddings from Pre-trained Language Models." *Proceedings of EMNLP 2020*, 9119-9130. aclanthology.org/2020.emnlp-main.733
25. ^Bolukbasi, Tolga, et al. "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." *Advances in Neural Information Processing Systems* 29 (2016). proceedings.neurips.cc/...571622f4f316ec5-Abstract
26. ^Gonen, Hila, and Yoav Goldberg. "Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them." *Proceedings of NAACL-HLT 2019*, 609-614. aclanthology.org/N19-1061
27. ^Morris, John X., Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M. Rush. "Text Embeddings Reveal (Almost) As Much As Text." *Proceedings of EMNLP 2023*, 12448-12460. aclanthology.org/2023.emnlp-main.765
28. ^OpenAI. "New embedding models and API updates." January 25, 2024. Accessed July 28, 2026. openai.com/...new-embedding-models-and-api-updates
29. ^OpenAI Help Center. "Embeddings FAQ." Accessed July 28, 2026. help.openai.com/...6824809-embeddings-faq
30. ^Google AI for Developers. "Embeddings: Gemini API." Accessed July 28, 2026. ai.google.dev/...embeddings
31. ^Cohere. "Cohere's Embed Models." Accessed July 28, 2026. docs.cohere.com/...cohere-embed
32. ^Voyage AI. "Text Embeddings." Accessed July 28, 2026. docs.voyageai.com/...embeddings