AI Inference

Explore AI Inference through related topics and the articles other pages reference most.

Explore articles

Reset filters
Browse subtopics: Open Source AI

Articles that also belong to these categories. Counts cover all of AI Inference.

Showing 1-23 of 23 articles

DeepInfer

DeepInfer is an open-source inference framework developed by DeepSeek for running its models across accelerator clusters. DeepSeek announced its public release on October 20, 2026.

Chinese AIOpen Source AI

FlashMLA

FlashMLA is an open-source library of attention kernels from DeepSeek, built around Multi-head Latent Attention (MLA), the attention variant DeepSeek uses to shrink the KV cache in its large language models.

AI InfrastructureDeveloper Tools

GraphRAG

GraphRAG is a graph-based approach to retrieval-augmented generation developed by Microsoft Research, first described publicly on February 13, 2024 and formalized in the paper "From Local to Global: A Graph…

Information RetrievalMicrosoft

Inco AI

Inco AI is an AI inference company that builds speculative decoding draft models, quantized checkpoints, a hosted inference platform and an open-source local inference engine for Macs.

AI CompaniesOpen Source AI

Neywa Labs

Neywa Labs is a startup that builds software for running artificial intelligence models locally on personal devices, with a focus on Macs and other Apple silicon hardware.

AI CompaniesDeveloper Tools

OpenVINO

OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit developed by Intel for optimizing and deploying deep learning inference across a heterogeneous set of Intel hardware…

Developer ToolsOpen Source AI

Optimum-Quanto

Optimum Quanto, commonly referred to as Quanto, is a PyTorch-based quantization toolkit developed and maintained by Hugging Face that provides linear weight and activation quantization across multiple devices…

Developer ToolsOpen Source AI

PrismML

PrismML (also styled Prism ML; legal name Prism ML, Inc.) is an American artificial intelligence company, based in Pasadena, California

AI CompaniesOpen Source AI

d3-turbo

d3-turbo is a multimodal decision model in the Decision 3.0 family from the vLLM Semantic Router team. It produces structured decisions and probabilities rather than free-form generated text.

AI ModelsMultimodal AI