ALBERT
ALBERT (A Lite BERT) is a parameter-efficient variant of the BERT language model developed by researchers at Google Research and the Toyota Technological Institute at Chicago (TTIC).
Natural Language Processing
Explore Natural Language Processing through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Transformer Models
Articles that also belong to these categories. Counts cover all of Natural Language Processing.
All articles (239)Machine Learning (92)Large Language Models (58)AI Benchmarks (42)Deep Learning (37)Information Retrieval (29)AI Models (24)Artificial Intelligence (24)Open Source AI (23)Data & Datasets (15)AI Companies (9)Model Evaluation (9)Transformer Models (9)Meta AI (8)Speech & Audio AI (8)OpenAI (7)Training & Optimization (7)AI Ethics (6)AI Safety (6)AI Tools & Products (6)Computer Vision (6)Generative AI (6)People (6)AI Agents (5)Prompt Engineering (5)Developer Tools (4)Neural Networks (4)AI Code Generation (3)Algorithms (3)Anthropic (3)Mathematics (3)Reasoning Models (3)Statistics (3)AI Alignment (2)AI History (2)AI Research (2)Chinese AI (2)Diffusion Models (2)Google DeepMind (2)Healthcare AI (2)Model Architecture (2)Multimodal AI (2)AI Inference (1)Computer Science (1)Education AI (1)Enterprise AI (1)Legal AI (1)Microsoft (1)Mixture of Experts (1)NVIDIA (1)Research Organizations (1)Voice AI (1)
Showing 1-9 of 9 articles
DeBERTa
DeBERTa (Decoding-enhanced BERT with Disentangled Attention) is a family of pre-trained language models developed by Microsoft Research that improves BERT and RoBERTa with two innovations: a disentangled…
DistilBERT
DistilBERT is a compressed version of BERT released by Hugging Face in October 2019 that is 40% smaller and 60% faster than BERT-base while retaining 97% of its language-understanding performance on the GLUE…
ELECTRA
ELECTRA, which stands for Efficiently Learning an Encoder that Classifies Token Replacements Accurately
Longformer
Longformer is a transformer architecture for processing long documents, introduced by Iz Beltagy, Matthew E. Peters
PaLM
PaLM (Pathways Language Model) is a family of dense, decoder-only large language models developed by Google Research. Google announced the original family on April 4, 2022.
Positional encoding
Positional encoding is a technique used to inject information about token order into transformer models.
RoBERTa
RoBERTa (Robustly Optimized BERT Pretraining Approach) is an open-source natural language processing model released in July 2019 by researchers at Facebook AI (now Meta AI) and the University of Washington…
XLNet
XLNet is a generalized autoregressive pretraining method for natural language processing that combines the strengths of autoregressive and autoencoding language models.