Atlas (inference engine)
Atlas is an open-source large language model inference engine written in Rust and CUDA, built for desktop "unified memory" machines such as the NVIDIA DGX Spark (GB10) and AMD Strix Halo (Ryzen AI Max+ 395)…
Open Source AI
Explore Open Source AI through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Inference
Articles that also belong to these categories. Counts cover all of Open Source AI.
All articles (517)Large Language Models (184)AI Models (134)Developer Tools (129)Chinese AI (92)AI Agents (69)AI Companies (48)AI Infrastructure (46)Mixture of Experts (43)Multimodal AI (32)NVIDIA (31)Robotics (31)AI Code Generation (28)Machine Learning (26)AI Inference (23)Natural Language Processing (23)Meta AI (20)Small Language Models (19)AI Tools & Products (18)Generative AI (18)Data & Datasets (16)Diffusion Models (15)Video Generation (15)Speech & Audio AI (14)Computer Vision (13)Image Generation (13)Information Retrieval (13)Reasoning Models (13)AI Research (12)Google (11)Research Organizations (11)Training & Optimization (11)AI Safety (10)Embodied AI (10)Microsoft (10)Reinforcement Learning (10)AI for Science (9)Artificial Intelligence (9)Google DeepMind (8)MLOps (8)Robot Hardware (8)Voice AI (7)Enterprise AI (6)OpenAI (6)AI Benchmarks (5)AI Hardware (5)Deep Learning (5)Drug Discovery (5)Healthcare AI (5)Model Evaluation (5)AI Policy & Regulation (3)Algorithms (3)Anthropic (3)Conversational AI (3)Humanoid Robots (3)Interpretability (3)Mathematics (3)People (3)Software Development (3)World Models (3)AI Compute (2)AI Ethics (2)AI in Gaming (2)Agentic Commerce (2)Model Architecture (2)Neural Networks (2)Physical AI (2)Programming Languages (2)Transformer Models (2)AI Events (1)AI Incidents & Controversies (1)Autonomous Vehicles (1)Consumer AI Devices (1)Data Science (1)Finance AI (1)Music & Audio Generation (1)Quantum Computing (1)Robotics Companies (1)
Showing 1-23 of 23 articles
DFlash
DFlash is a speculative decoding method in which the draft model is a small block-diffusion network rather than an autoregressive one.
Decision 2.0
Decision 2.0 is a family of open-weight models for structured decisions from the vLLM Semantic Router project.
DeepInfer
DeepInfer is an open-source inference framework developed by DeepSeek for running its models across accelerator clusters. DeepSeek announced its public release on October 20, 2026.
DeepSelect
DeepSelect is an open-source library of top-k selection kernels published by the Chinese AI company DeepSeek.
ExLlamaV2 (EXL2)
EXL2 (ExLlamaV2 format) is an open-source, mixed-bit weight-quantization format for compressing large language models so they run fast on a single consumer-class NVIDIA GPU.
FlashInfer
FlashInfer is an open-source GPU kernel library and code-generation system for large language model inference.
FlashMLA
FlashMLA is an open-source library of attention kernels from DeepSeek, built around Multi-head Latent Attention (MLA), the attention variant DeepSeek uses to shrink the KV cache in its large language models.
GraphRAG
GraphRAG is a graph-based approach to retrieval-augmented generation developed by Microsoft Research, first described publicly on February 13, 2024 and formalized in the paper "From Local to Global: A Graph…
Inco AI
Inco AI is an AI inference company that builds speculative decoding draft models, quantized checkpoints, a hosted inference platform and an open-source local inference engine for Macs.
Magnitude (inference engine)
Magnitude is an open-source inference engine for running open-weight language models on local hardware, developed by Magnitude AI Inc., a San Francisco startup from the Y Combinator Summer 2025 batch.
NVIDIA BioNeMo Inference Runtime
NVIDIA BioNeMo Inference Runtime (BioIR) is a GPU-accelerated Python library for biomolecular structure prediction inference.
NVIDIA Dynamo
NVIDIA Dynamo is an open-source, low-latency distributed inference serving framework designed to deploy and scale generative AI and reasoning models across large GPU clusters.
NVIDIA TensorRT-LLM
NVIDIA TensorRT-LLM is an open-source library developed by nvidia for high-performance inference of large language models on NVIDIA GPUs.
Neywa Labs
Neywa Labs is a startup that builds software for running artificial intelligence models locally on personal devices, with a focus on Macs and other Apple silicon hardware.
OpenVINO
OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit developed by Intel for optimizing and deploying deep learning inference across a heterogeneous set of Intel hardware…
Optimum-Quanto
Optimum Quanto, commonly referred to as Quanto, is a PyTorch-based quantization toolkit developed and maintained by Hugging Face that provides linear weight and activation quantization across multiple devices…
PrismML
PrismML (also styled Prism ML; legal name Prism ML, Inc.) is an American artificial intelligence company, based in Pasadena, California
Project Maya
Project Maya is open-source software for running large mixture-of-experts language models on local hardware.
Splash (inference engine)
Splash is an open-source large language model inference engine for Macs with Apple silicon, released by the inference company Inco AI on 18 September 2026 under the Apache-2.0 license.
Text Generation Inference (TGI)
Text Generation Inference (TGI) is an open-source toolkit developed by Hugging Face for deploying and serving large language models in production.
Underdog Saluki 27B
Underdog Saluki 27B is a 2-bit quantized build of Alibaba's open-weight Qwen3.8-27B language model, released on 7 October 2026 by Underdog, the on-device AI assistant made by Conway Research.
d3-turbo
d3-turbo is a multimodal decision model in the Decision 3.0 family from the vLLM Semantic Router team. It produces structured decisions and probabilities rather than free-form generated text.