ExLlamaV2 (EXL2)
EXL2 (ExLlamaV2 format) is an open-source, mixed-bit weight-quantization format for compressing large language models so they run fast on a single consumer-class NVIDIA GPU.
Open Source AI
Explore Open Source AI through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Inference
Articles that also belong to these categories. Counts cover all of Open Source AI.
All articles (444)Large Language Models (165)AI Models (120)Developer Tools (109)Chinese AI (77)AI Agents (53)AI Companies (42)Mixture of Experts (40)AI Infrastructure (37)Multimodal AI (28)NVIDIA (28)Robotics (28)Machine Learning (25)AI Code Generation (23)Natural Language Processing (23)AI Tools & Products (18)Generative AI (18)Meta AI (17)Small Language Models (16)Data & Datasets (15)Video Generation (13)Computer Vision (12)Diffusion Models (12)Image Generation (12)Information Retrieval (12)Google (11)Reasoning Models (11)AI Research (10)Artificial Intelligence (10)AI Hardware (9)AI Inference (9)Microsoft (9)Research Organizations (9)Speech & Audio AI (9)Embodied AI (8)Google DeepMind (8)MLOps (7)Training & Optimization (7)AI for Science (6)Reinforcement Learning (6)Robot Hardware (6)Deep Learning (5)Drug Discovery (5)OpenAI (5)Voice AI (5)AI Benchmarks (4)AI Safety (4)Healthcare AI (4)Model Evaluation (4)Algorithms (3)Conversational AI (3)AI Ethics (2)Agentic Commerce (2)Anthropic (2)Enterprise AI (2)Humanoid Robots (2)Interpretability (2)Mathematics (2)Model Architecture (2)Neural Networks (2)People (2)Physical AI (2)Programming Languages (2)Software Development (2)Transformer Models (2)AI Incidents & Controversies (1)AI Policy (1)AI Policy & Regulation (1)Data Science (1)Music & Audio Generation (1)Quantum Computing (1)Robotics Companies (1)World Models (1)
Showing 1-9 of 9 articles
FlashInfer
FlashInfer is an open-source GPU kernel library and code-generation system for large language model inference.
GraphRAG
GraphRAG is a graph-based approach to retrieval-augmented generation developed by Microsoft Research, first described publicly on February 13, 2024 and formalized in the paper "From Local to Global: A Graph…
NVIDIA BioNeMo Inference Runtime
NVIDIA BioNeMo Inference Runtime (BioIR) is a GPU-accelerated Python library for biomolecular structure prediction inference.
NVIDIA Dynamo
NVIDIA Dynamo is an open-source, low-latency distributed inference serving framework designed to deploy and scale generative AI and reasoning models across large GPU clusters.
NVIDIA TensorRT-LLM
NVIDIA TensorRT-LLM is an open-source library developed by nvidia for high-performance inference of large language models on NVIDIA GPUs.
OpenVINO
OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit developed by Intel for optimizing and deploying deep learning inference across a heterogeneous set of Intel hardware…
Optimum-Quanto
Optimum Quanto, commonly referred to as Quanto, is a PyTorch-based quantization toolkit developed and maintained by Hugging Face that provides linear weight and activation quantization across multiple devices…
Text Generation Inference (TGI)
Text Generation Inference (TGI) is an open-source toolkit developed by Hugging Face for deploying and serving large language models in production.