DeBERTa
DeBERTa (Decoding-enhanced BERT with Disentangled Attention) is a family of pre-trained language models developed by Microsoft Research that improves BERT and RoBERTa with two innovations: a disentangled…
Microsoft
Explore Microsoft through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Transformer Models
Articles that also belong to these categories. Counts cover all of Microsoft.
All articles (45)Large Language Models (9)Open Source AI (9)Developer Tools (7)AI Agents (6)AI Models (6)Model Architecture (6)AI Hardware (5)AI Infrastructure (4)AI Tools & Products (4)People (4)AI Code Generation (3)Small Language Models (3)Transformer Models (3)AI History (2)Conversational AI (2)Data Centers (2)Deep Learning (2)AI Benchmarks (1)AI Companies (1)AI Ethics (1)AI Incidents & Controversies (1)AI Inference (1)Artificial Intelligence (1)Computer Vision (1)Generative AI (1)Google DeepMind (1)Healthcare AI (1)Image Generation (1)Information Retrieval (1)Natural Language Processing (1)Programming Languages (1)Reasoning Models (1)Software Development (1)Speech & Audio AI (1)Training & Optimization (1)
Showing 1-3 of 3 articles
Differential Transformer
The Differential Transformer (often shortened to Diff Transformer or DIFF Transformer) is a decoder-only neural sequence architecture introduced by researchers at Microsoft Research and Tsinghua University in…
LongNet
LongNet is a transformer variant introduced by Microsoft Research in July 2023 that is designed to scale attention to sequences exceeding one billion tokens while preserving performance on shorter inputs.