ALBERT
ALBERT (A Lite BERT) is a parameter-efficient variant of the BERT language model developed by researchers at Google Research and the Toyota Technological Institute at Chicago (TTIC).
Transformer Models
Explore Transformer Models through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Natural Language Processing
Articles that also belong to these categories. Counts cover all of Transformer Models.
All articles (46)Deep Learning (18)Model Architecture (16)Machine Learning (11)Large Language Models (9)Natural Language Processing (9)Computer Vision (5)Neural Networks (4)AI Models (3)Microsoft (3)Mixture of Experts (3)Chinese AI (2)Google (2)Interpretability (2)Meta AI (2)Open Source AI (2)People (2)Training & Optimization (2)AI Companies (1)AI Hardware (1)AI Inference (1)Algorithms (1)Artificial Intelligence (1)Diffusion Models (1)Generative AI (1)Google DeepMind (1)Healthcare AI (1)Image Generation (1)Multimodal AI (1)Robotics (1)
Showing 1-9 of 9 articles
DeBERTa
DeBERTa (Decoding-enhanced BERT with Disentangled Attention) is a family of pre-trained language models developed by Microsoft Research that improves BERT and RoBERTa with two innovations: a disentangled…
DistilBERT
DistilBERT is a compressed version of BERT released by Hugging Face in October 2019 that is 40% smaller and 60% faster than BERT-base while retaining 97% of its language-understanding performance on the GLUE…
ELECTRA
ELECTRA, which stands for Efficiently Learning an Encoder that Classifies Token Replacements Accurately
Longformer
Longformer is a transformer architecture for processing long documents, introduced by Iz Beltagy, Matthew E. Peters
PaLM
PaLM (Pathways Language Model) is a family of dense, decoder-only large language models developed by Google Research. Google announced the original family on April 4, 2022.
Positional encoding
Positional encoding is a technique used to inject information about token order into transformer models.
RoBERTa
RoBERTa (Robustly Optimized BERT Pretraining Approach) is an open-source natural language processing model released in July 2019 by researchers at Facebook AI (now Meta AI) and the University of Washington…
XLNet
XLNet is a generalized autoregressive pretraining method for natural language processing that combines the strengths of autoregressive and autoencoding language models.