AWQ (Activation-aware Weight Quantization)
Activation-aware Weight Quantization (AWQ) is a post-training quantization method for large language models that compresses weights to 4-bit (and optionally 3-bit) integers while keeping near-FP16 task…
Deep Learning
Explore Deep Learning through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Inference
Articles that also belong to these categories. Counts cover all of Deep Learning.
All articles (333)Machine Learning (187)Neural Networks (81)Computer Vision (53)Training & Optimization (41)Natural Language Processing (37)Generative AI (31)Model Architecture (21)Transformer Models (18)Artificial Intelligence (16)Large Language Models (12)People (12)Reinforcement Learning (12)Developer Tools (11)Mathematics (11)AI Inference (9)Interpretability (8)AI Infrastructure (7)Speech & Audio AI (7)AI Hardware (6)AI Companies (5)AI History (5)Google DeepMind (5)Open Source AI (5)AI Research (4)AI for Science (4)Algorithms (4)Model Evaluation (4)Robotics (4)AI Models (3)Google (3)Meta AI (3)OpenAI (3)AI Alignment (2)AI Tools & Products (2)Chinese AI (2)Data & Datasets (2)Healthcare AI (2)MLOps (2)Microsoft (2)Multimodal AI (2)NVIDIA (2)Software Development (2)Statistics (2)AI Benchmarks (1)AI Ethics (1)AI Safety (1)Data Science (1)Diffusion Models (1)Embodied AI (1)Image Generation (1)Information Retrieval (1)Programming Languages (1)Prompt Engineering (1)Video Generation (1)
Showing 1-9 of 9 articles
GPTQ
GPTQ (Generative Pre-trained Transformer Quantization) is a one-shot post-training quantization method that compresses the weights of large language models to 3 or 4 bits using approximate second-order…
KV Cache
KV cache, short for key-value cache, is transient model state used during Transformer generation.
Knowledge Distillation
Knowledge distillation is a training method in which a student model learns from signals produced by a teacher model.
NVIDIA Triton Inference Server
NVIDIA Triton Inference Server is open-source model deployment software that lets teams run trained models from any machine learning or deep learning framework on any processor (GPU, CPU, or other accelerator)…
QLoRA
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method that finetunes a 65-billion-parameter large language model on a single 48 GB GPU while matching the quality of full 16-bit…
Quantization
Quantization is the representation of values from a large or continuous set by values from a smaller, usually finite set.
Speculative Decoding
Speculative decoding is a lossless inference acceleration technique for autoregressive transformer models in which a small, fast draft model proposes several future tokens at once and the larger target model…
YaRN
YaRN (Yet another RoPE extensioN) is a compute-efficient method for extending the context window of large language models that use Rotary Position Embeddings (RoPE).