One-Shot Learning

One-shot learning is a machine learning approach in which a model learns to recognize or classify new categories from only a single labeled example per class. Unlike conventional supervised learning methods that typically require hundreds or thousands of labeled training examples, one-shot learning systems aim to replicate the human ability to generalize from very limited experience. The term was introduced by Li Fei-Fei, Rob Fergus, and Pietro Perona in their 2006 IEEE TPAMI paper on object category recognition, which demonstrated a Bayesian framework for learning visual categories from one or a few examples by leveraging prior knowledge from previously learned categories ^[1]. As that paper observed, "learning visual models of object categories notoriously requires hundreds or thousands of training examples," yet "it is possible to learn much information about a category from just one, or a handful, of images" ^[1].

One-shot learning occupies a specific position within the broader spectrum of N-shot learning paradigms. Zero-shot learning requires no examples of target classes at all, instead relying on auxiliary information such as semantic attributes or textual descriptions. Few-shot learning generalizes to settings with a small number of examples (typically 2 to 20 per class). One-shot learning is the special case where exactly one example is available per novel class, making it one of the most challenging data-efficient learning problems. It remains difficult even for modern methods: on the standard 5-way 1-shot miniImageNet benchmark, leading metric-learning approaches reach only about 49 to 50 percent accuracy, compared with roughly 20 percent for random guessing among five classes ^[5][7].

Explain like I'm 5 (ELI5)

Imagine someone shows you a single photo of a platypus. You have never seen one before. But from that one picture, you notice key features: a duck-like bill, a beaver tail, and brown fur. Later at the zoo, you spot a platypus right away because you remember those features from the one photo. That is basically what one-shot learning does for computers. It teaches a computer to recognize something new after seeing it just one time, by paying attention to the features that make it different from everything else.

Why is one-shot learning needed?

Traditional deep learning models, particularly convolutional neural networks for image classification, achieve high accuracy only when trained on large labeled datasets. ImageNet, for example, contains over 14 million labeled images across more than 20,000 categories. Collecting and annotating datasets of this scale is time-consuming, expensive, and sometimes impossible for specialized domains.

Several real-world scenarios make large-scale data collection impractical:

• Rare categories. In medical imaging, examples of rare diseases may be extremely scarce. A hospital might have only a handful of confirmed cases for a particular condition.
• Privacy-sensitive data. Face verification systems must enroll new users from a single photo. Collecting hundreds of images per person raises privacy concerns.
• Rapidly changing classes. Product recognition systems must accommodate new items that appear daily, making it infeasible to retrain from scratch each time.
• Cost of expert annotation. Labeling satellite imagery, biological specimens, or industrial defects often requires domain experts whose time is limited and costly.

Humans, by contrast, can learn to identify a new animal, recognize a new friend's face, or understand a new word after encountering it just once or twice. The Fei-Fei team framed this as taking "advantage of knowledge coming from previously learned categories, no matter how different these categories might be" ^[1]. This cognitive ability inspired researchers to develop machine learning methods that can similarly generalize from minimal data ^[1][14].

How is one-shot learning formally defined?

One-shot learning is most commonly formulated as an N-way K-shot classification problem. In this setting:

• A support set S contains K labeled examples from each of N novel classes. For one-shot learning, K = 1, so the support set has exactly N examples total.
• A query set Q contains unlabeled examples that must be classified into one of the N classes.
• The model must assign each query example to the correct class, using only the support set for guidance.

This formulation is distinct from standard classification because the classes in the support set are typically unseen during training. The model must generalize to new classes at test time without any fine-tuning.

Episodic training

Most modern one-shot learning methods use episodic training (also called meta-training), a protocol popularized by Matching Networks that matches training conditions to test conditions ^[3]. During each training iteration, the algorithm samples a random N-way K-shot task (called an episode) from the training set:

1. Select N classes at random from the training classes.
2. For each class, sample K examples for the support set and a separate set of query examples.
3. Train the model to classify the query examples correctly, given the support set.
4. Update model parameters based on the classification loss.

By training on thousands of such episodes, the model learns a general strategy for classifying novel classes from few examples, rather than memorizing specific categories.

Benchmark datasets

Several standard benchmarks are used to evaluate one-shot learning methods:

Scroll sideways for more →

Omniglot, sometimes called the "transpose of MNIST", was introduced by Brenden Lake and colleagues in 2015 ^[14] and is considered an easier benchmark. It contains 1,623 handwritten characters drawn from 50 alphabets, with 20 examples of each character produced by 20 different people ^[14]. miniImageNet, proposed by Vinyals et al. (2016) ^[3] and later standardized by Ravi and Larochelle (2017) ^[15], has become the most widely used benchmark for comparing few-shot learning methods.

How does one-shot learning work?

Research on one-shot learning can be broadly organized into four families of approaches: metric-based methods, optimization-based (meta-learning) methods, memory-augmented methods, and data augmentation-based methods.

Metric-based methods

Metric-based approaches learn an embedding space where examples from the same class are close together and examples from different classes are far apart. Classification is then performed by comparing the distances between query embeddings and support set embeddings.

Siamese networks

Siamese networks, applied to one-shot image recognition by Koch, Zemel, and Salakhutdinov (2015) ^[2], consist of two identical neural network branches that share the same weights. Each branch processes one input image and produces a feature vector. The network then computes a distance between the two feature vectors (typically using a weighted L1 distance followed by a sigmoid function) to output a similarity score.

During training, the network is presented with pairs of images that are either from the same class (positive pairs) or different classes (negative pairs). It learns to output high similarity for same-class pairs and low similarity for different-class pairs. At test time, a query image is compared against each example in the support set, and the class of the most similar support example is assigned.

Strengths: Siamese networks are conceptually simple and can generalize to new classes without retraining. Once the embedding function is learned, new classes can be added by providing a single reference image.

Limitations: Pairwise comparison scales quadratically with the number of support examples and classes, and the fixed distance function may not capture complex class boundaries.

Matching networks

Matching Networks, introduced by Vinyals, Blundell, Lillicrap, Kavukcuoglu, and Wierstra (2016), extend the Siamese approach by framing one-shot learning as a differentiable nearest-neighbor problem. The authors describe a framework that "maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types" ^[3]. The model uses an attention mechanism over the support set embeddings to produce a predicted label.

A key innovation of Matching Networks is the use of Full Context Embeddings (FCE), where the embedding of each support example is conditioned on the entire support set through a bidirectional LSTM. This allows the model to produce embeddings that are adapted to the specific task at hand. On Omniglot, Matching Networks reached 98.1% accuracy on the 5-way 1-shot task and 93.8% on the harder 20-way 1-shot task, and the method improved one-shot accuracy on ImageNet from 87.6% to 93.2% over prior approaches ^[3].

Prototypical networks

Prototypical Networks, proposed by Snell, Swersky, and Zemel (2017) ^[5], simplify the metric-based approach by computing a single prototype representation for each class. The prototype is defined as the mean of the embedded support examples for that class. As the authors put it, prototypical networks "learn a metric space in which classification can be performed by computing distances to prototype representations of each class" ^[5]. Classification is performed by computing the distance from a query embedding to each class prototype and assigning the nearest class.

Formally, given an embedding function f (typically a CNN), the prototype for class c is:

p_c = (1/|S_c|) * sum of f(x) for all x in S_c

where S_c is the set of support examples for class c. For one-shot learning, each prototype is simply the embedding of the single support example.

Prototypical Networks use squared Euclidean distance as the distance metric. The authors showed that this choice is equivalent to a linear classifier in the embedding space, providing a simple inductive bias well suited to the limited-data regime ^[5]. The method achieves strong results while being simpler than Matching Networks and faster to train.

Relation networks

Relation Networks, introduced by Sung, Yang, Zhang, Xiang, Torr, and Hospedales (2018) ^[7], replace the fixed distance function with a learned relation module. Instead of computing Euclidean or cosine distance between embeddings, the model concatenates the query and support embeddings and passes them through a small neural network that outputs a relation score between 0 and 1.

This approach allows the model to learn a non-linear similarity function that may capture more complex relationships than a fixed metric. Relation Networks were evaluated on both Omniglot and miniImageNet and showed competitive performance with other metric-based methods ^[7].

Comparison of metric-based methods

Scroll sideways for more →

The miniImageNet figures above use the standardized evaluation adopted by later work; the original Matching Networks paper reported a somewhat higher 46.6% on 5-way 1-shot miniImageNet using Full Context Embeddings ^[3][5].

Optimization-based methods (meta-learning)

Optimization-based methods take the approach of "learning to learn" by training model initialization or update rules that enable rapid adaptation to new tasks from minimal data.

Model-Agnostic Meta-Learning (MAML)

MAML, proposed by Finn, Abbeel, and Levine (2017), is a meta-learning algorithm that seeks an initialization of model parameters from which a few gradient steps on a small support set can produce good performance on the corresponding query set. In the authors' framing, "the parameters of the model are explicitly trained such that a small number of gradient steps with a small amount of training data from a new task will produce good generalization performance on that task" ^[6]. The key insight is that some parameter initializations are better suited for fast adaptation than others.

The MAML training procedure works in two loops:

1. Inner loop. For each task (episode), start from the current meta-parameters and perform one or a few gradient descent steps on the support set loss to get task-specific parameters.
2. Outer loop. Evaluate the task-specific parameters on the query set and compute the meta-loss. Update the meta-parameters by backpropagating through the inner loop optimization.

This requires computing second-order gradients (gradients of gradients), which is computationally expensive. A first-order approximation called FOMAML drops these second-order terms with minimal performance loss.

MAML is "model-agnostic" in the sense that it can be applied to any model trained with gradient descent, including classification, regression, and reinforcement learning tasks. On 5-way 1-shot miniImageNet, MAML achieves approximately 48.70% accuracy ^[6].

Reptile

Reptile, proposed by Nichol, Achiam, and Schulman (2018), is a simpler alternative to MAML that avoids computing second-order gradients ^[16]. Instead of differentiating through the optimization process, Reptile repeatedly performs several steps of SGD on a task and then moves the initialization toward the resulting parameters. Despite its simplicity, Reptile achieves comparable performance to MAML on standard benchmarks ^[16].

Meta-SGD

Meta-SGD (Li, Zhou, Chen, and Li, 2017) extends MAML by also meta-learning the learning rate and update direction for each parameter, rather than using a fixed learning rate ^[17]. This provides more flexibility in the inner-loop optimization and can lead to faster adaptation, and the authors report that Meta-SGD can adapt "any differentiable learner in just one step" ^[17].

Memory-augmented methods

Memory-augmented neural networks (MANNs) extend standard neural networks with an external memory module that allows them to store and retrieve information about previously seen examples.

Santoro, Bartunov, Botvinick, Wierstra, and Lillicrap (2016) demonstrated that a neural network augmented with an external memory, similar to a Neural Turing Machine, could perform one-shot learning by quickly encoding new examples into memory and using content-based addressing to retrieve relevant memories at test time ^[4]. Their approach uses a Least Recently Used Access (LRUA) memory writing strategy that encourages the network to write new information to either the least recently used memory slot or the most recently used slot, balancing novelty and familiarity.

The advantage of memory-augmented approaches is that they can rapidly incorporate new information without changing the model weights, making them well suited for scenarios where new classes appear continuously.

Data augmentation-based methods

Data augmentation-based methods address the scarcity of training examples by generating additional synthetic examples for novel classes.

Hariharan and Girshick (2017) proposed learning a "hallucinator" network that generates additional feature vectors for underrepresented classes ^[9]. Given one real example from a novel class, the hallucinator produces multiple synthetic feature vectors by learning transformations from base classes that have abundant data. This approach improved one-shot accuracy on ImageNet by 2.3x ^[9]. A follow-up paper by Wang, Girshick, Hebert, and Hariharan (2018) jointly optimized the hallucinator with a meta-learner, yielding up to a 6-point improvement in classification accuracy when only a single training example is available ^[10].

Other data augmentation strategies for one-shot learning include geometric transformations (rotation, scaling, flipping), color jittering, and more sophisticated methods such as using generative adversarial networks to synthesize training examples.

Graph-based methods

Garcia and Bruna (2018) proposed using graph neural networks for few-shot learning by constructing a graph where nodes represent both support and query examples and edges encode similarity relationships ^[8]. Message passing on this graph allows information to propagate between examples, enabling the model to make predictions by aggregating neighborhood information. This framework generalizes several earlier few-shot learning methods and can be extended to semi-supervised and active learning settings.

Comparison of approach families

Scroll sideways for more →

How does one-shot learning relate to other paradigms?

One-shot learning is closely connected to several related learning paradigms.

How does one-shot learning differ from zero-shot and few-shot learning?

Scroll sideways for more →

The three paradigms differ chiefly in how much labeled evidence about a novel class is available at test time. Zero-shot learning sees no example of the target class and must bridge the gap with side information such as textual descriptions or attribute vectors. One-shot learning sees exactly one labeled example and typically needs no auxiliary information. Few-shot learning sees a handful (commonly 2 to 20) and, all else equal, is easier than one-shot because averaging over several support examples reduces the effect of an atypical or noisy sample.

Transfer learning

Transfer learning involves reusing a model trained on one task (the source) for a different but related task (the target). One-shot learning can be seen as an extreme form of transfer learning, where the "transfer" happens from base classes with many examples to novel classes with only one example. Many one-shot learning methods use a backbone network pretrained on base classes and then apply metric learning or meta-learning strategies to generalize to novel classes.

Meta-learning

Meta-learning, or "learning to learn," is the broader framework within which many one-shot learning methods operate. While meta-learning encompasses any approach that improves learning efficiency across tasks, one-shot learning focuses specifically on the extreme low-data regime ^[13]. MAML, Matching Networks, and Prototypical Networks are all instances of meta-learning applied to the one-shot setting.

One-shot learning and in-context learning

With the rise of large language models, the term "one-shot" took on a second, related meaning. In in-context learning, a pretrained model performs a new task from demonstrations placed directly in its prompt, with no gradient updates or parameter changes. The GPT-3 paper (Brown et al., 2020) formalized this usage, defining one-shot as "the same as few-shot except that only one demonstration is allowed, in addition to a natural language description of the task" ^[18]. The authors distinguish one-shot from few-shot and zero-shot because it "most closely matches the way in which some tasks are communicated to humans" ^[18].

This differs from classical one-shot learning in an important way. Classical methods learn a reusable embedding or initialization during a training phase and then classify a novel class from one labeled support example. In-context one-shot learning instead uses a single input-output demonstration to condition a frozen model (GPT-3 has 175 billion parameters) at inference time, exploiting patterns absorbed during large-scale pretraining rather than any dedicated few-shot training objective ^[18].

Less-than-one-shot learning

Sucholutsky and Schonlau (2021) introduced the concept of "less than one"-shot learning, where a model must learn to distinguish N classes given fewer than N examples total ^[12]. This is achieved through the use of soft labels (probability distributions over classes rather than hard class assignments), which encode information about multiple classes in a single example. While still primarily theoretical and experimental on simple datasets, this work pushes the boundaries of data-efficient learning beyond what was previously considered possible.

What is one-shot learning used for?

One-shot learning has found practical applications across many domains.

Computer vision

• Face recognition and verification. Systems like Apple's Face ID enroll a new user from a single set of reference images. The system must verify identity from one example, making one-shot learning techniques directly applicable.
• Object recognition in robotics. Robots deployed in warehouses or homes may encounter novel objects. One-shot learning allows them to identify new items after a single demonstration.
• Handwriting recognition. The Omniglot benchmark was originally designed to test character recognition across diverse writing systems, a task where collecting large datasets per character is impractical ^[14].

Medical imaging

Medical imaging datasets are often limited because of privacy constraints, the rarity of certain conditions, and the high cost of expert annotation. One-shot and few-shot learning methods enable models to classify medical images (such as X-rays, MRI scans, or histopathology slides) with minimal labeled examples, which is especially valuable for rare diseases.

Drug discovery

Altae-Tran, Ramsundar, Pappu, and Pande (2017) demonstrated that one-shot learning can be applied to molecular property prediction, enabling models to predict chemical properties like toxicity from very few examples of a compound class ^[11]. Their iterative refinement LSTM combined with graph neural networks achieved strong results on molecular datasets with limited training data.

Natural language processing

In NLP, one-shot and few-shot learning have been applied to tasks such as text classification, named entity recognition, and relation extraction in low-resource settings. Large language models like GPT-3 have demonstrated remarkable few-shot and one-shot abilities through in-context learning, where examples are provided directly in the input prompt rather than used to update model weights ^[18].

Other applications

• Signature verification. Verifying whether a signature is authentic by comparing it to a single reference sample.
• Audio classification. Identifying new sound categories from a single recording, useful for environmental monitoring and wildlife tracking.
• Autonomous navigation. Drones and self-driving vehicles may encounter novel obstacles or road signs that were not in the training data.

What are the main challenges in one-shot learning?

Despite significant progress, one-shot learning still faces several open challenges.

Domain shift. Most methods are evaluated on benchmarks where training and test classes come from similar distributions (for example, different subsets of ImageNet). Performance degrades significantly when there is a large domain gap between base and novel classes, such as transferring from natural images to medical scans.

Scalability. Many one-shot learning methods are evaluated on small-scale benchmarks with a limited number of classes. Scaling to thousands or millions of classes, as would be needed in real-world product recognition or biodiversity monitoring, remains difficult.

Cross-modal generalization. Current methods typically operate within a single modality (images, text, or audio). Developing one-shot learning systems that can transfer knowledge across modalities is an active research area.

Noisy and ambiguous support examples. In practical settings, the single available example may be noisy, atypical, or poorly representative of the class. Robust one-shot learning methods must handle such cases gracefully.

Evaluation standardization. Different papers use different dataset splits, backbone architectures, and training procedures, making fair comparison of methods challenging ^[13]. Efforts to standardize evaluation protocols are ongoing.

Open-world recognition. Most one-shot learning methods assume a closed-set scenario where every query belongs to one of the N support classes. In the real world, models must also detect when a query does not belong to any known class (out-of-distribution detection).

Historical timeline

Scroll sideways for more →

See also

• Few-shot learning
• Zero-shot learning
• Meta-learning
• Transfer learning
• In-context learning
• Siamese network
• Data augmentation
• Embedding
• MAML

References

1. Fei-Fei, L., Fergus, R., & Perona, P. (2006). "One-shot learning of object categories." *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 28(4), 594-611. ↩
2. Koch, G., Zemel, R., & Salakhutdinov, R. (2015). "Siamese neural networks for one-shot image recognition." *ICML Deep Learning Workshop*. ↩
3. Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra, D. (2016). "Matching networks for one shot learning." *Advances in Neural Information Processing Systems*, 29. ↩
4. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016). "Meta-learning with memory-augmented neural networks." *Proceedings of the 33rd International Conference on Machine Learning*. ↩
5. Snell, J., Swersky, K., & Zemel, R. (2017). "Prototypical networks for few-shot learning." *Advances in Neural Information Processing Systems*, 30. ↩
6. Finn, C., Abbeel, P., & Levine, S. (2017). "Model-agnostic meta-learning for fast adaptation of deep networks." *Proceedings of the 34th International Conference on Machine Learning*, 70, 1126-1135. ↩
7. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., & Hospedales, T. M. (2018). "Learning to compare: Relation network for few-shot learning." *IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 1199-1208. ↩
8. Garcia, V. & Bruna, J. (2018). "Few-shot learning with graph neural networks." *International Conference on Learning Representations (ICLR)*. ↩
9. Hariharan, B. & Girshick, R. (2017). "Low-shot visual recognition by shrinking and hallucinating features." *IEEE International Conference on Computer Vision (ICCV)*. ↩
10. Wang, Y., Girshick, R., Hebert, M., & Hariharan, B. (2018). "Low-shot learning from imaginary data." *IEEE/CVF Conference on Computer Vision and Pattern Recognition*. ↩
11. Altae-Tran, H., Ramsundar, B., Pappu, A. S., & Pande, V. (2017). "Low data drug discovery with one-shot learning." *ACS Central Science*, 3(4), 283-293. ↩
12. Sucholutsky, I. & Schonlau, M. (2021). "'Less than one'-shot learning: Learning N classes from M < N samples." *Proceedings of the AAAI Conference on Artificial Intelligence*, 35(11), 9739-9746. ↩
13. Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). "Generalizing from a few examples: A survey on few-shot learning." *ACM Computing Surveys*, 53(3), 1-34. ↩
14. Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). "Human-level concept learning through probabilistic program induction." *Science*, 350(6266), 1332-1338. ↩
15. Ravi, S. & Larochelle, H. (2017). "Optimization as a model for few-shot learning." *International Conference on Learning Representations (ICLR)*. ↩
16. Nichol, A., Achiam, J., & Schulman, J. (2018). "On first-order meta-learning algorithms." *arXiv preprint arXiv:1803.02999*. ↩
17. Li, Z., Zhou, F., Chen, F., & Li, H. (2017). "Meta-SGD: Learning to learn quickly for few-shot learning." *arXiv preprint arXiv:1707.09835*. ↩
18. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). "Language models are few-shot learners." *Advances in Neural Information Processing Systems*, 33, 1877-1901. ↩