Multi-task learning

Multi-task learning (MTL) is a machine learning approach in which one model is trained on several tasks at the same time, so that what it learns for one task helps it learn the others. Rich Caruana, in the 1997 journal article that remains the field's standard reference, defined it as "an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias," achieved "by learning tasks in parallel while using a shared representation" ^[1]. In a neural network this typically means a common set of layers feeding several task-specific output heads, with gradients from every task shaping the shared weights.

The idea has moved from a niche trick to a default assumption of modern AI. Instruction tuning of large language models is multi-task learning at the scale of nearly two thousand tasks, and researchers at Google showed in 2022 that finetuning on roughly 1,800 tasks lifted a 540-billion-parameter model's average performance by 9.4 percent ^[17]. At the same time, the field's central difficulty has not gone away: tasks can also interfere with each other, a failure mode known as negative transfer, and a substantial body of work exists on deciding which tasks to train together and how to keep their gradients from fighting ^[8][9][12].

Origins

Caruana introduced multitask learning in a 1993 ICML paper, "Multitask Learning: A Knowledge-Based Source of Inductive Bias" ^[2], and developed it fully in the 1997 article "Multitask Learning" in the journal Machine Learning (volume 28, pages 41-75) ^[1]. The core observation is that the standard methodology of learning one task at a time throws away information. Training signals for related tasks, even tasks whose predictions will never be used, act as an inductive bias: they push the shared hidden layer toward internal features that are useful across tasks, and a hypothesis that explains several tasks at once is more likely to generalize than one fit to a single task ^[1].

The 1997 paper demonstrated this in three domains. In 1D-ALVINN, a simulated road-following problem derived from Pomerleau's ALVINN work, a network trained to predict steering direction alongside eight extra tasks (such as locations of road edges and the centerline) reduced steering error by 15 to 30 percent compared with single-task networks trained on identical data ^[1]. In 1D-DOORS, an object recognition problem using images collected by a robot-mounted camera, joint training on ten door-related tasks improved the two main tasks, doorknob location and door type, by 20 to 30 percent ^[1]. The third domain was pneumonia risk prediction using the Medis Pneumonia Database of 14,199 hospitalized cases collected from 78 hospitals in 1989. Here Caruana used 35 lab results that only become available after a patient is admitted as extra training-time outputs. The trick, which he called "using the future to predict the present," means the model never needs those values at prediction time, yet the extra training signals bias the shared representation toward features that matter for risk ^[1].

The paper also identified mechanisms that explain why the benefit occurs, including statistical data amplification (related tasks effectively enlarge the sample for shared features), attribute selection (extra tasks help identify which inputs are relevant), eavesdropping (a feature that is hard to learn from one task can be picked up from another that learns it easily), and representation bias (the search over weights is pulled toward regions that serve multiple tasks) ^[1]. Although backpropagation networks with multiple outputs were the primary vehicle, Caruana also presented multitask variants of k-nearest neighbor and kernel regression and sketched one for decision trees ^[1]. A later survey by Yu Zhang and Qiang Yang organized the large non-deep MTL literature that followed into feature learning, low-rank, task clustering, task relation learning, and decomposition approaches ^[24].

Hard and soft parameter sharing

Deep learning architectures for MTL are conventionally divided into two families, a taxonomy popularized by Sebastian Ruder's 2017 overview ^[3][4].

Hard parameter sharing is the design Caruana used and is still the most common: hidden layers are shared between all tasks, with task-specific output layers on top ^[4]. Its appeal is statistical as well as computational. Ruder, citing theoretical work by Jonathan Baxter from 1997, notes that sharing parameters across N tasks reduces the risk of overfitting those shared parameters by roughly a factor of N relative to task-specific parameters, since the representation must work for every task at once ^[4].

In soft parameter sharing, each task keeps its own model and parameters, and the distance between the models' parameters is regularized to encourage them to stay similar ^[4]. Architectures in this family learn how much to share rather than fixing it by hand. Cross-stitch networks (Misra et al., CVPR 2016) place trainable "cross-stitch units" between parallel task networks; the units learn linear combinations of the activations of both networks at each layer, so the model discovers per-layer how much sharing helps, with the largest gains reported on categories with few training examples ^[5]. The Multi-gate Mixture-of-Experts (MMoE) model (Ma et al., KDD 2018) shares a pool of expert subnetworks across all tasks but gives each task its own gating network, letting the model learn task relationships from data; Google reported that it outperformed shared-bottom baselines on a large-scale recommender system, particularly when the tasks were less related ^[6].

Auxiliary tasks

Not every task in a multi-task model needs to matter on its own. Auxiliary tasks are added purely to improve a main task, and choosing them well is one of the practical arts of MTL. Ruder's overview catalogs common types, including hints (explicitly training a feature known to be useful), predicting inputs, adversarial objectives, and Caruana's using-the-future scheme ^[4].

The best-known demonstration in reinforcement learning is the UNREAL agent (Jaderberg et al., 2016), which augmented an actor-critic agent with unsupervised auxiliary control and prediction tasks: the agent "maximises many other pseudo-reward functions simultaneously," all sharing one representation that keeps developing even when extrinsic rewards are sparse. UNREAL averaged 880 percent of expert human performance on Atari and 87 percent on the 3D Labyrinth suite, where the auxiliary tasks produced a mean 10x speedup in learning ^[7].

Negative transfer and task interference

Jointly trained tasks do not always help each other. Negative transfer, the phenomenon where knowledge from a poorly related source task hurts target performance, was long discussed informally; Wang et al. (CVPR 2019) gave it a formal definition and an adversarial filtering method for avoiding it ^[8]. In multi-task networks the same problem appears as task interference: objectives compete for shared capacity, and adding tasks can make everything worse.

One response is to choose groupings. Standley et al. (ICML 2020) studied which computer vision tasks should be learned together and proposed assigning cooperating tasks to the same network while giving competing tasks separate networks; the resulting task groupings achieved better accuracy with less inference time than either one large multi-task network or a full set of single-task networks ^[9].

The other response is to fix the optimization. Yu et al. (NeurIPS 2020) diagnosed what they called the "tragic triad" of multi-task optimization: conflicting gradients (task gradients pointing in opposing directions), dominating gradients (large differences in gradient magnitude across tasks), and high positive curvature. When all three occur, gradient descent on the summed loss function makes poor progress ^[12]. Their remedy, gradient surgery via projecting conflicting gradients (PCGrad), projects each task's gradient onto the normal plane of any other task's gradient it conflicts with, removing the destructive component. PCGrad is model-agnostic and improved both supervised benchmarks such as NYUv2 and multi-task RL on Meta-World MT10 and MT50 ^[12].

Balancing losses and gradients

Even without outright conflict, tasks with different units, scales, or noise levels can drown each other out when their losses are simply summed. Several widely used methods address the weighting problem directly.

Scroll sideways for more →

Kendall et al. demonstrated uncertainty weighting on a model that performs per-pixel depth regression, semantic segmentation, and instance segmentation from a single monocular image, and reported that the jointly trained model outperformed separate models trained individually on each task ^[10].

Multi-task learning in NLP

Natural language processing turned multi-task learning from a training trick into a unification strategy. The Natural Language Decathlon (decaNLP, 2018) posed ten tasks, from question answering and translation to semantic parsing, all cast as question answering over a context, and trained a single Multitask Question Answering Network on all of them without task-specific modules ^[13]. T5 (Raffel et al., 2019) went further by converting every text problem into the same text-to-text format, so one encoder-decoder model with one objective could handle summarization, classification, translation, and question answering ^[14].

Instruction tuning turned this into the dominant recipe for making language models useful. FLAN (Wei et al., 2021) finetuned a 137-billion-parameter model on more than 60 NLP tasks verbalized as natural-language instructions and found that the tuned model beat zero-shot 175-billion-parameter GPT-3 on 20 of 25 evaluated datasets ^[15]. T0 (Sanh et al., ICLR 2022) made the multi-task framing explicit, asking whether zero-shot generalization can be "directly induced by explicit multitask learning" on a large mixture of prompted datasets; the resulting model often outperformed models up to 16 times its size ^[16]. Chung et al. then scaled instruction finetuning to roughly 1,800 tasks: Flan-PaLM 540B outperformed PaLM 540B by 9.4 percent on average and reached 75.2 percent on five-shot MMLU ^[17]. Seen through this lens, every instruction-tuned chat model is a multi-task learner, and pre-training itself is often described as implicit multi-task learning over the objectives hidden in web text ^[16].

Multi-task reinforcement learning

Reinforcement learning has proved to be one of the hardest settings for MTL, because tasks differ not just in labels but in reward scales and dynamics. Distral (Teh et al., 2017) trains task-specific policies that are all regularized toward a shared "distilled" policy, the centroid of the task policies, which captures common behavior and transfers it across tasks while limiting destructive gradient interference ^[18]. The Meta-World benchmark (Yu et al., 2019) provides 50 simulated robotic manipulation tasks for evaluating multi-task and meta-learning algorithms; its authors found that although each individual task is learnable, contemporary algorithms "struggle to learn with multiple tasks at the same time, even with as few as ten distinct training tasks" ^[19]. DeepMind's Gato (2022) pushed generalist ambitions further: a single network with a single set of weights that can play Atari, caption images, chat, and stack blocks with a real robot arm, deciding from context whether to emit text, joint torques, or button presses ^[20].

Mixture of experts and multi-task models

The mixture of experts architecture is closely tied to multi-task learning, because routing gives a network a way to allocate different parameters to different inputs or tasks instead of forcing everything through one shared trunk. The sparsely gated MoE layer of Shazeer et al. (2017), with up to thousands of expert subnetworks and a trainable gating network selecting a sparse few per example, scaled models to 137 billion parameters and improved language modeling and machine translation ^[21]. MMoE adapted the idea to multi-task learning directly by giving each task its own gate over shared experts ^[6]. In modern LLMs, sparse MoE is mainstream: Mixtral 8x7B, described in a January 2024 paper by Mistral AI, uses eight experts per layer with a router that picks two per token, so a 47-billion-parameter model activates only about 13 billion parameters per inference step ^[22]. Because experts specialize, MoE models are frequently discussed as a structural mitigation for task interference, echoing MMoE's finding that per-task gating helps most when tasks are weakly related ^[6].

Model merging and related paradigms

Multi-task learning sits in a family of transfer methods that are easy to confuse. Transfer learning usually means sequential transfer: pre-train on one task, then fine-tune on another. Meta-learning optimizes for fast adaptation to new tasks rather than joint performance on a fixed set. Continual learning handles tasks arriving over time. MTL proper trains on all tasks simultaneously with shared parameters ^[3][24].

A newer alternative to joint training is model merging. Ilharco et al. (ICLR 2023) showed that subtracting a pre-trained model's weights from a fine-tuned model's weights yields a "task vector," and that adding several task vectors to the base model improves performance on all the corresponding tasks at once, without any joint training run ^[23]. Merging effectively performs multi-task composition in weight space, and it has become an active research direction for combining specialist models cheaply. Together with instruction tuning and sparse expert architectures, it means the questions Caruana raised in 1997 (what should be shared, between which tasks, and how to stop them from hurting each other) are now being asked about the largest models in deployment.

See also

• Transfer learning
• Instruction tuning
• Mixture of experts
• Meta-learning
• Continual learning
• Model merging

References

1. ^Caruana, R. "Multitask Learning." Machine Learning, vol. 28, pp. 41-75, 1997. link.springer.com/...A:1007379606734
2. ^Caruana, R. "Multitask Learning: A Knowledge-Based Source of Inductive Bias." Proceedings of the 10th International Conference on Machine Learning (ICML), pp. 41-48, 1993. dblp.org/...Caruana93
3. ^Ruder, S. "An Overview of Multi-Task Learning in Deep Neural Networks." arXiv, June 15, 2017. arxiv.org/...1706.05098
4. ^Ruder, S. "An Overview of Multi-Task Learning in Deep Neural Networks." ruder.io. ruder.io/multi-task
5. ^Misra, I.; Shrivastava, A.; Gupta, A.; Hebert, M. "Cross-stitch Networks for Multi-task Learning." CVPR 2016. arxiv.org/...1604.03539
6. ^Ma, J.; Zhao, Z.; Yi, X.; Chen, J.; Hong, L.; Chi, E. H. "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts." KDD 2018. dl.acm.org/...3219819.3220007
7. ^Jaderberg, M.; Mnih, V.; Czarnecki, W. M.; Schaul, T.; Leibo, J. Z.; Silver, D.; Kavukcuoglu, K. "Reinforcement Learning with Unsupervised Auxiliary Tasks." arXiv, November 16, 2016. arxiv.org/...1611.05397
8. ^Wang, Z.; Dai, Z.; Poczos, B.; Carbonell, J. "Characterizing and Avoiding Negative Transfer." CVPR 2019. arxiv.org/...1811.09751
9. ^Standley, T.; Zamir, A. R.; Chen, D.; Guibas, L.; Malik, J.; Savarese, S. "Which Tasks Should Be Learned Together in Multi-task Learning?" ICML 2020. arxiv.org/...1905.07553
10. ^Kendall, A.; Gal, Y.; Cipolla, R. "Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics." CVPR 2018. arxiv.org/...1705.07115
11. ^Chen, Z.; Badrinarayanan, V.; Lee, C.-Y.; Rabinovich, A. "GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks." ICML 2018. arxiv.org/...1711.02257
12. ^Yu, T.; Kumar, S.; Gupta, A.; Levine, S.; Hausman, K.; Finn, C. "Gradient Surgery for Multi-Task Learning." NeurIPS 2020. arxiv.org/...2001.06782
13. ^McCann, B.; Keskar, N. S.; Xiong, C.; Socher, R. "The Natural Language Decathlon: Multitask Learning as Question Answering." arXiv, June 20, 2018. arxiv.org/...1806.08730
14. ^Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P. J. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer." arXiv, October 23, 2019. arxiv.org/...1910.10683
15. ^Wei, J.; Bosma, M.; Zhao, V. Y.; Guu, K.; Yu, A. W.; Lester, B.; Du, N.; Dai, A. M.; Le, Q. V. "Finetuned Language Models Are Zero-Shot Learners." ICLR 2022. arxiv.org/...2109.01652
16. ^Sanh, V.; Webson, A.; Raffel, C.; Bach, S. H.; et al. "Multitask Prompted Training Enables Zero-Shot Task Generalization." ICLR 2022. arxiv.org/...2110.08207
17. ^Chung, H. W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; et al. "Scaling Instruction-Finetuned Language Models." arXiv, October 20, 2022. arxiv.org/...2210.11416
18. ^Teh, Y. W.; Bapst, V.; Czarnecki, W. M.; Quan, J.; Kirkpatrick, J.; Hadsell, R.; Heess, N.; Pascanu, R. "Distral: Robust Multitask Reinforcement Learning." arXiv, July 13, 2017. arxiv.org/...1707.04175
19. ^Yu, T.; Quillen, D.; He, Z.; Julian, R.; Narayan, A.; Shively, H.; Bellathur, A.; Hausman, K.; Finn, C.; Levine, S. "Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning." arXiv, October 24, 2019. arxiv.org/...1910.10897
20. ^Reed, S.; Zolna, K.; Parisotto, E.; et al. "A Generalist Agent." arXiv, May 12, 2022. arxiv.org/...2205.06175
21. ^Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q.; Hinton, G.; Dean, J. "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer." arXiv, January 23, 2017. arxiv.org/...1701.06538
22. ^Jiang, A. Q.; Sablayrolles, A.; Roux, A.; Mensch, A.; et al. "Mixtral of Experts." arXiv, January 8, 2024. arxiv.org/...2401.04088
23. ^Ilharco, G.; Ribeiro, M. T.; Wortsman, M.; Gururangan, S.; Schmidt, L.; Hajishirzi, H.; Farhadi, A. "Editing Models with Task Arithmetic." ICLR 2023. arxiv.org/...2212.04089
24. ^Zhang, Y.; Yang, Q. "A Survey on Multi-Task Learning." IEEE Transactions on Knowledge and Data Engineering (arXiv preprint July 25, 2017). arxiv.org/...1707.08114