DeBERTa
DeBERTa (Decoding-enhanced BERT with Disentangled Attention) is a family of pre-trained language models developed by Microsoft Research that improves BERT and RoBERTa with two innovations: a disentangled…
Microsoft
Explore Microsoft through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Deep Learning
Articles that also belong to these categories. Counts cover all of Microsoft.
All articles (45)Large Language Models (9)Open Source AI (9)Developer Tools (7)AI Agents (6)AI Models (6)Model Architecture (6)AI Hardware (5)AI Infrastructure (4)AI Tools & Products (4)People (4)AI Code Generation (3)Small Language Models (3)Transformer Models (3)AI History (2)Conversational AI (2)Data Centers (2)Deep Learning (2)AI Benchmarks (1)AI Companies (1)AI Ethics (1)AI Incidents & Controversies (1)AI Inference (1)Artificial Intelligence (1)Computer Vision (1)Generative AI (1)Google DeepMind (1)Healthcare AI (1)Image Generation (1)Information Retrieval (1)Natural Language Processing (1)Programming Languages (1)Reasoning Models (1)Software Development (1)Speech & Audio AI (1)Training & Optimization (1)
Showing 1-2 of 2 articles
ZeRO (Zero Redundancy Optimizer)
ZeRO (Zero Redundancy Optimizer) is a family of memory-optimization techniques for training large neural networks introduced by Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He at microsoft…