DFlash
DFlash is a speculative decoding method in which the draft model is a small block-diffusion network rather than an autoregressive one.
Open Source AI
Explore Open Source AI through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Inference
Articles that also belong to these categories. Counts cover all of Open Source AI.
All articles (473)Large Language Models (173)AI Models (123)Developer Tools (117)Chinese AI (87)AI Agents (60)AI Companies (47)AI Infrastructure (43)Mixture of Experts (42)Robotics (31)Multimodal AI (29)NVIDIA (29)AI Code Generation (25)Machine Learning (25)Natural Language Processing (23)Generative AI (18)AI Tools & Products (17)Meta AI (17)Small Language Models (17)AI Inference (15)Data & Datasets (15)Diffusion Models (14)Image Generation (13)Information Retrieval (13)Video Generation (13)Computer Vision (12)Google (11)Reasoning Models (11)AI Research (10)Embodied AI (10)Research Organizations (10)AI Hardware (9)AI Safety (9)Artificial Intelligence (9)Microsoft (9)Speech & Audio AI (9)Google DeepMind (8)MLOps (7)Reinforcement Learning (7)Training & Optimization (7)AI for Science (6)OpenAI (6)Robot Hardware (6)Deep Learning (5)Drug Discovery (5)Voice AI (5)AI Benchmarks (4)Enterprise AI (4)Healthcare AI (4)Model Evaluation (4)Algorithms (3)Conversational AI (3)Humanoid Robots (3)Interpretability (3)People (3)AI Ethics (2)AI Policy & Regulation (2)Agentic Commerce (2)Anthropic (2)Mathematics (2)Model Architecture (2)Neural Networks (2)Physical AI (2)Programming Languages (2)Software Development (2)Transformer Models (2)AI Events (1)AI Incidents & Controversies (1)AI Policy (1)Data Science (1)Music & Audio Generation (1)Quantum Computing (1)Robotics Companies (1)World Models (1)
Showing 1-15 of 15 articles
Decision 2.0
Decision 2.0 is a family of open-weight models for structured decisions from the vLLM Semantic Router project.
DeepSelect
DeepSelect is an open-source library of top-k selection kernels published by the Chinese AI company DeepSeek.
ExLlamaV2 (EXL2)
EXL2 (ExLlamaV2 format) is an open-source, mixed-bit weight-quantization format for compressing large language models so they run fast on a single consumer-class NVIDIA GPU.
FlashInfer
FlashInfer is an open-source GPU kernel library and code-generation system for large language model inference.
FlashMLA
FlashMLA is an open-source library of attention kernels from DeepSeek, built around Multi-head Latent Attention (MLA), the attention variant DeepSeek uses to shrink the KV cache in its large language models.
GraphRAG
GraphRAG is a graph-based approach to retrieval-augmented generation developed by Microsoft Research, first described publicly on February 13, 2024 and formalized in the paper "From Local to Global: A Graph…
Inco AI
Inco AI is an AI inference company that builds speculative decoding draft models, quantized checkpoints, a hosted inference platform and an open-source local inference engine for Macs.
NVIDIA BioNeMo Inference Runtime
NVIDIA BioNeMo Inference Runtime (BioIR) is a GPU-accelerated Python library for biomolecular structure prediction inference.
NVIDIA Dynamo
NVIDIA Dynamo is an open-source, low-latency distributed inference serving framework designed to deploy and scale generative AI and reasoning models across large GPU clusters.
NVIDIA TensorRT-LLM
NVIDIA TensorRT-LLM is an open-source library developed by nvidia for high-performance inference of large language models on NVIDIA GPUs.
OpenVINO
OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit developed by Intel for optimizing and deploying deep learning inference across a heterogeneous set of Intel hardware…
Optimum-Quanto
Optimum Quanto, commonly referred to as Quanto, is a PyTorch-based quantization toolkit developed and maintained by Hugging Face that provides linear weight and activation quantization across multiple devices…
Splash (inference engine)
Splash is an open-source large language model inference engine for Macs with Apple silicon, released by the inference company Inco AI on 18 September 2026 under the Apache-2.0 license.
Text Generation Inference (TGI)
Text Generation Inference (TGI) is an open-source toolkit developed by Hugging Face for deploying and serving large language models in production.