# vLLM

> Source: https://aiwiki.ai/wiki/vllm
> Updated: 2026-07-29
> Fact-checked: 2026-07-29
> Categories: AI Tools & Products, Large Language Models, Machine Learning
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "vLLM." aiwiki.ai, 29 Jul 2026. https://aiwiki.ai/wiki/vllm
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

vLLM is an open-source inference and serving engine for [large language models](https://aiwiki.ai/wiki/large_language_model). It is software for running already trained models, either through a Python interface for offline work or through a network server for online requests. It does not train a foundation model, provide model weights, or by itself supply the surrounding product features of a hosted model service. Its central systems contribution is PagedAttention, a method for managing the key-value state retained during autoregressive generation in fixed-size blocks rather than requiring one contiguous allocation for every request.[1]

The project was first announced publicly on June 20, 2023, by a team associated primarily with the University of California, Berkeley. The accompanying paper by Woosuk Kwon and colleagues was published at the 29th ACM Symposium on Operating Systems Principles in October 2023.[1][2] The original paper evaluated vLLM against FasterTransformer and research implementations of Orca. In those experiments, it delivered two to four times their throughput at a comparable latency level, with larger gains in the tested cases involving longer sequences, larger models, or more complex decoding. That result describes the paper's particular 2023 models, hardware, traces, implementations, and latency constraint; it is not a universal speedup over every later serving engine.[1]

vLLM has since expanded beyond its original memory manager into a broader [inference engine](https://aiwiki.ai/wiki/inference_engine) with scheduling, model execution, distributed parallelism, quantization, adapter serving, multiple API surfaces, and production telemetry. UC Berkeley contributed the project to the Linux Foundation in July 2024, and the PyTorch Foundation announced vLLM as a hosted project under its governance in May 2025.[3][4] The repository uses the Apache License 2.0.[5] Model weights, tokenizers, and other dependencies used with the engine can have separate licenses and usage conditions.

## Scope

vLLM occupies the execution layer between a model artifact and an application. For offline inference, an application imports the `LLM` class, loads a supported model, and submits a collection of prompts. For online serving, an operator starts a process with `vllm serve`, after which clients submit requests over HTTP. The current quickstart documents both modes and presents different installation paths for NVIDIA CUDA, AMD ROCm, Google TPU, Ascend NPU, and Apple Silicon environments.[6]

This scope creates several boundaries:

- vLLM schedules and executes inference. It is not a model architecture and does not determine what a model learned.
- API compatibility describes request and response protocols, not identical output behavior across providers.
- A listed model architecture or accelerator is not a guarantee that every combination of model, precision, kernel, feature, and device works.
- Performance depends on the model, prompt and output length distributions, request arrival pattern, hardware, precision, parallelism, and service-level objective.

These distinctions matter because a serving engine can be correct and useful without being the fastest choice for every workload. They also prevent model quality, model licensing, and service reliability from being attributed to the runtime alone.

## Inference workload

Autoregressive generation has two broad phases. During prefill, the model processes the input tokens and builds attention state. During decode, it repeatedly produces new tokens while consulting the state for the prompt and the tokens already generated. The retained attention keys and values are commonly called the [KV cache](https://aiwiki.ai/wiki/kv_cache). For a single request, that cache grows with sequence length. Across a server, requests arrive and finish at different times and have prompt and output lengths that are not generally known in advance.[1]

The model weights occupy a comparatively stable allocation while requests are active, but the KV cache is dynamic. A server that reserves a large contiguous region for each request can waste space in three ways: unused reservation, unused space within an allocation, and gaps that cannot satisfy later allocations. The original paper measured its particular Orca baselines and found that only 20.4 to 38.2 percent of their allocated KV-cache memory held token state in the reported experiment. This is a historical measurement of those baselines, not a standing claim that every non-vLLM system wastes the same fraction.[1]

Batching is important because one decode step often leaves accelerator compute capacity unused. A serving scheduler can combine work from multiple requests so that a model forward pass processes more tokens. Unlike static batching, [continuous batching](https://aiwiki.ai/wiki/continuous_batching) can change the active set between engine iterations as requests arrive, advance, or finish. Higher concurrency can improve aggregate throughput, but it also consumes more KV-cache space and can increase queueing or per-request latency. Memory management and scheduling therefore have to be considered together.[25]

## PagedAttention

[PagedAttention](https://aiwiki.ai/wiki/paged_attention) applies an operating-system paging analogy to attention state. Each sequence has logical KV blocks, and each block holds keys and values for a fixed number of tokens. A block table maps those logical blocks to physical blocks. The physical blocks assigned to adjacent parts of one sequence do not need to be adjacent in accelerator memory.[1]

The design changes allocation behavior in several ways:

1. Physical blocks are assigned as tokens require them instead of reserving space for the maximum possible sequence length at admission time.
2. Equal-size blocks remove external fragmentation among KV-cache allocations.
3. Internal waste is bounded mainly by unused slots in the last partially filled block of a sequence.
4. When a request finishes, its blocks can be returned independently to the free pool.

The mapping is part of the attention operation rather than a transparent virtual-memory system supplied by the operating system. The PagedAttention kernel uses the block table to locate the keys and values needed for attention. That indirection is not free: the paper notes extra address translation, branching, and non-contiguous access, and reports that the implementation fuses related memory access with attention work to reduce overhead.[1]

Block size is a tradeoff rather than an absolute optimum. Small blocks reduce unused capacity at sequence ends and provide finer allocation granularity, but they enlarge block tables and can reduce memory-access efficiency. Large blocks can improve parallel access while increasing internal fragmentation and reducing the chance that a reusable prefix ends on a full-block boundary. The best choice can therefore vary with model architecture, kernel backend, sequence distribution, and hardware.

### Sharing and copy-on-write

Several decoding patterns create sequences with a common prefix. Parallel sampling produces multiple continuations from one prompt; beam search creates branches that may share earlier tokens. PagedAttention can map logical blocks in different sequences to the same physical blocks. A reference count records how many mappings use each block. If a sequence needs to modify a shared block, vLLM allocates another physical block and copies the shared content before writing, a block-level copy-on-write operation.[1]

This sharing concerns cached intermediate state, not generated text or model weights. It can reduce duplicate KV-cache storage when prefixes truly match. It does not make unrelated requests share state, and its benefit depends on prefix length, block boundaries, cache retention, and the later divergence of the sequences.

## Scheduling and cache lifecycle

The scheduler decides which tokens to compute in each engine step under a token and sequence budget. Running decode work, partially processed prompts, and newly admitted requests compete for that budget. If demand for KV-cache blocks exceeds available capacity, work may have to wait or be preempted. The current V1 guide says V1 no longer requires GPU-to-CPU KV-cache swapping to handle request preemptions.[8]

Chunked prefill divides a long prompt into portions that can share engine steps with decode tokens. Current V1 guidance says chunked prefill is enabled whenever possible and that the scheduler gives decode work priority before using the remaining token budget for prompt chunks. This can improve inter-token latency and device utilization, but tuning the token budget changes the balance between prompt latency, decode latency, and throughput.[11]

Automatic prefix caching retains computed KV blocks from a prompt and reuses them when a later request has the same prefix. It can avoid repeated prefill work in use cases such as repeated questions over one document or later turns in one conversation. It does not accelerate generation of new output tokens, and it provides no reuse when requests lack a cached common prefix.[10]

These mechanisms solve different problems. PagedAttention manages where active KV state resides. Prefix caching decides whether previously computed prefix state can be reused. Chunked prefill controls how prompt computation is scheduled. Speculative decoding changes how candidate output tokens are proposed and verified. Enabling one does not imply that the others will improve a particular workload.

## V1 architecture

vLLM V1 reworked the scheduler, KV-cache manager, worker, sampler, and API server into a more unified architecture. At the research cutoff, the project documentation described V0 as fully deprecated and maintained a feature-status guide for V1 because support still changes over time.[8] Version-specific deployment decisions should therefore be checked against the documentation for the installed release rather than inferred from an older tutorial.

The current online architecture separates responsibilities across processes:[7]

| Process | Primary responsibility |
|---|---|
| API server | Accepts HTTP requests, performs input processing such as tokenization and media loading, and streams results |
| Engine core | Runs scheduling, manages KV-cache state, and coordinates model execution |
| GPU worker | Loads model weights, manages device memory, and executes forward passes |
| Data-parallel coordinator | Conditionally coordinates load distribution and synchronized work across data-parallel ranks |

For offline use, the `LLM` class is the main Python entry point and does not require a separate HTTP server. Online use normally starts through `vllm serve`; the documentation marks the older direct `python -m vllm.entrypoints.openai.api_server` invocation as deprecated.[7]

This process separation means accelerator count is not the only capacity consideration. Tokenization, media decoding, networking, request routing, and output streaming consume CPU time and memory. Multi-process or multi-node deployments also introduce interprocess communication, collective operations, topology constraints, and additional failure modes.

## Interfaces

The HTTP server implements a subset of protocols associated with the [OpenAI API](https://aiwiki.ai/wiki/openai_api). Current documentation lists Completions, Chat Completions, Responses, Embeddings, audio transcription, and audio translation endpoints, with each endpoint limited to applicable model types. It also documents intentional differences, such as an unsupported `suffix` field for Completions and an ignored `user` field for Chat Completions.[9]

Calling the server "OpenAI-compatible" therefore means that supported requests can use the documented protocol and clients. It does not mean that every OpenAI endpoint or parameter exists, that defaults are identical, or that the same prompt produces the same tokens. Model chat templates and a repository's `generation_config.json` can also affect how requests are rendered and which sampling defaults apply.[9]

The Python interface provides direct control over model loading and sampling for scripts and data pipelines. Both interfaces eventually use the engine's scheduling and model-execution components, but the operational concerns differ. Offline jobs can often prioritize total throughput and reproducible dataset processing. An online service usually has concurrent arrivals, cancellation, streaming, authentication, rate limits, and tail-latency targets.

## Execution features

### Prefix reuse and adapter serving

Automatic prefix caching is most useful when long prompt regions recur. Cache capacity remains finite, so admission, eviction, and routing affect the hit rate. In a replicated deployment, each replica can have a different local cache state unless an external KV-transfer or routing system coordinates them.

vLLM can serve supported [LoRA](https://aiwiki.ai/wiki/lora) adapters over a base model, including selection on a per-request basis. The current documentation limits this to model implementations that declare LoRA support. It also warns that dynamically loading adapters at runtime creates security risks and should not be enabled in an untrusted production environment.[12] Adapter rank, number of concurrently active adapters, model architecture, and kernel support all affect memory and performance.

### Quantization

Quantization represents weights, activations, or KV-cache values with lower-precision formats to reduce memory use and, on suitable hardware and kernels, improve execution efficiency. Lower precision can also change numerical behavior or model quality. vLLM supports multiple quantization implementations, but its own compatibility table varies by accelerator generation and backend and is explicitly described as subject to change.[14]

A format name is not enough to establish compatibility. Operators must match the checkpoint format, model architecture, accelerator, driver and software stack, kernel implementation, and requested features. A quantized checkpoint should also be evaluated on the application's quality and latency workload rather than assumed to preserve all behavior.

### Speculative decoding

[Speculative decoding](https://aiwiki.ai/wiki/speculative_decoding) proposes candidate tokens through a draft mechanism and verifies them with the target model. Current vLLM documentation includes model-based methods and methods based on prompt or suffix patterns. It describes the intended benefit as lower inter-token latency for suitable medium-to-low request-rate, memory-bound workloads and cautions that realized gains depend on the model family, traffic, hardware, and sampling configuration.[13]

Speculation consumes extra work and sometimes extra memory. If proposed tokens are rarely accepted or the target model is already compute-bound at high load, it can provide little benefit or reduce throughput. The engine's documentation also notes that floating-point and batching effects can cause practical output differences even when the sampling algorithm is designed to preserve the target distribution.[13]

### Model and modality support

The supported-model registry covers generative text models as well as selected multimodal, embedding, classification, scoring, and speech architectures. Some architectures use native vLLM implementations, some use a Transformers modeling backend, and some are supplied through plugins.[15] Feature support is not uniform across these paths.

A model being listed does not imply support for every combination of [Hugging Face](https://aiwiki.ai/wiki/hugging_face) revision, task, multimodal input, adapter, quantization method, speculative method, attention backend, or parallelism mode. The current support table and release notes are the appropriate source for a specific deployment. Pinning a model revision and testing the intended configuration is safer than relying on a family name.

### Compilation and graph execution

On supported execution paths, V1 uses `torch.compile` as part of model execution. The integration captures PyTorch graphs, can partition or specialize them, invokes a compiler backend for executable artifacts, and can use CUDA Graphs to reduce repeated launch overhead on compatible paths. The project documentation says compilation completes before the server accepts requests so that a request does not unexpectedly trigger a new compile.[23]

Compilation changes both startup and steady-state behavior. The first start for a model and configuration may spend time producing artifacts and capturing graphs. A later start can reuse a compilation cache when its cache key matches the relevant code and configuration. Changing a model, software version, backend, shape policy, or compilation option can invalidate or bypass that work. Warm-start timing should therefore be reported separately from cold-start timing.

Not every model path, operator, attention backend, batch shape, or accelerator supports the same graph mode. Eager execution remains useful for debugging and for combinations that cannot be compiled, but it can have different performance. A benchmark should state whether compilation and graph capture were enabled, whether caches were warm, and which shapes were captured. Describing only the command line can miss these material differences.

## Distributed execution

When a model does not fit on one accelerator or when a service needs more replicas, vLLM exposes several parallelism strategies. [Tensor parallelism](https://aiwiki.ai/wiki/tensor_parallelism) shards operations within model layers. Pipeline parallelism assigns different layer groups to different stages. Data parallelism replicates model execution so that ranks can process independent batches, although mixture-of-experts deployments can require synchronization across ranks.[16][17]

Tensor and pipeline parallelism make one model instance span multiple devices. Data parallelism increases the number of engine cores and replicas. Combining them multiplies the number of workers and changes communication patterns. More devices do not guarantee proportional throughput because collective communication, interconnect topology, pipeline bubbles, load imbalance, CPU service capacity, and request lengths can dominate.

The official scaling guide recommends avoiding distributed execution when a model fits and the workload does not need it, using tensor parallelism within a node when appropriate, and combining tensor and pipeline parallelism for models that exceed one node. It also requires consistent model paths and software environments across nodes.[16] These are starting points rather than universal tuning rules.

## Deployment workflow

A defensible deployment begins by fixing the artifact identities: vLLM release or commit, model revision, tokenizer revision, optional adapter revisions, container image digest, and accelerator software versions. It then verifies that the exact model-task-feature combination appears in the matching support documentation. This is more precise than selecting a recent image and allowing model repositories or dependencies to move independently.

The project publishes container images for several hardware paths. Its Docker guide documents separate images or instructions for NVIDIA CUDA, AMD ROCm, Intel XPU, and Apple Silicon.[24] For CUDA deployments, the tagged installation source notes that vLLM's PyTorch multiprocessing uses shared memory, particularly for tensor-parallel inference.[26] A container packages user-space dependencies but does not remove dependencies on host drivers, device access, network topology, model credentials, or adequate shared memory.

Before exposing a service, an operator can test the model in offline mode with a small, fixed prompt set. The same artifact can then be started on a non-public endpoint and checked for protocol behavior, chat-template rendering, stop conditions, maximum context handling, cancellation, streaming, malformed input, and out-of-memory recovery. Multimodal deployments additionally need limits on media source, size, count, and decoding work.

Load testing should use a distribution that resembles expected traffic rather than one constant prompt. Useful cases include cold and warm prefix caches, short and long prompts, short and long outputs, bursts, sustained arrival rates, and requests near context limits. The test should continue long enough to expose queue growth, cache churn, memory leaks, thermal or power behavior, and periodic background effects. The metrics and benchmark interfaces provide observability for this process, but the acceptance thresholds come from the application.[18][20]

An upgrade is a new configuration, even when the public API remains compatible. Model outputs, numerical behavior, scheduling, memory capacity, startup time, feature composition, and latency can all change. A staged rollout should compare quality and performance against the pinned prior version, preserve a rollback path, and keep the prior model and container artifacts available. Release notes are evidence about intended changes, not a substitute for testing the deployed combination.[21]

## Performance evaluation

The original paper's two-to-four-times result remains important evidence for the design that introduced vLLM, but it should not be carried forward as a timeless comparison. Later engines can adopt related memory techniques, kernels and hardware change, and the project itself changes quickly. A current comparison needs fixed versions and a reproducible experimental setup.[1]

Serving performance has several distinct measures:

| Measure | What it captures |
|---|---|
| Request throughput | Completed requests per unit time |
| Output-token throughput | Generated tokens per unit time |
| Time to first token | Delay from request submission to the first streamed token |
| Time per output token | Average time for output tokens after the first |
| Inter-token latency | Gaps between streamed output tokens |
| End-to-end latency | Total time until a request completes |
| Goodput | Work completed while satisfying defined latency objectives |

The vLLM benchmark tooling reports throughput together with distributions for time to first token, time per output token, inter-token latency, and end-to-end latency.[20] A comparison that reports only tokens per second can hide queueing or unacceptable tail latency. Conversely, a one-request latency test does not establish capacity under concurrent load.

A useful benchmark records at least the exact engine commit or release, model and model revision, tokenizer, precision and quantization, accelerator and interconnect, driver and library versions, prompt and output distributions, arrival process, concurrency, streaming mode, warmup, cache state, parallelism, and latency targets. The same prompts and stopping rules should be used across engines. Results should distinguish input, output, and total-token throughput and report failed or rejected requests.

Feature combinations also need separate testing. Prefix caching can transform a repeated-prefix workload while doing little for unrelated prompts. Speculative decoding may improve low-load decode latency but consume capacity under a different traffic regime. Chunked prefill changes the tradeoff between first-token and inter-token latency. Quantization may relieve a memory limit while introducing conversion work or quality changes. There is no single configuration that establishes an engine's performance for every deployment.

## Operations and security

The online server exposes Prometheus-format operational metrics at `/metrics`. Current metrics cover running and waiting requests, prompt and generation tokens, KV-cache use, prefix-cache activity, preemptions, and latency phases including queue time, time to first token, time per output token, and end-to-end latency.[18] Metric names and availability evolve, so dashboards should be validated when upgrading rather than assuming permanent names.

Operators also need accelerator-level telemetry, process health, logs, request outcomes, saturation indicators, and capacity limits. Queue growth together with high KV-cache use can point to a memory-bound configuration; rising first-token latency with stable decode latency can indicate prefill or queue pressure. These are diagnostic patterns, not proof of one cause without workload and system measurements.

The server should not be treated as a complete public security boundary. The project's security guide states that API-key protection does not cover every endpoint and recommends a reverse proxy that allowlists intended routes while adding authentication, rate limiting, and logging. It also warns against enabling development or profiler endpoints in production and calls for network isolation of internal distributed-communication ports.[19]

Remote model code, model artifacts, media URLs, plugins, and dynamically loaded adapters expand the trust boundary. Production deployments should pin reviewed versions, restrict network and file access, control which model code can execute, and test upgrade and rollback procedures. The Apache license of the engine does not supersede licenses or security obligations attached to models and dependencies.[5]

## Limitations and correctness

Paged allocation reduces KV-cache fragmentation, but it cannot eliminate the memory cost of model weights, activations, long contexts, or high concurrency. It also introduces metadata and address-translation work. Prefix caching requires repeated prefixes and retained blocks. Distributed execution adds communication. Quantization and speculative decoding have hardware, quality, and workload constraints. These are engineering tradeoffs rather than defects unique to vLLM.

Model outputs can vary across runs because request batching and floating-point execution can change numerical results. The current reproducibility guide says default operation does not guarantee reproducible results and that its reproducibility modes remain bounded to the same hardware and vLLM version.[22] Reproducibility, model quality, protocol compatibility, and numerical closeness should be tested separately.

Rapid development is another practical limitation. Defaults, feature support, API details, model coverage, hardware backends, and metric names can change between releases. Configuration copied from an old article may be invalid or may select a different execution path on a newer release. Operators should pin versions, read the matching documentation and release notes, and run acceptance and performance tests before an upgrade.

## Release status

As of the article's research cutoff, July 28, 2026 at 23:59:59 UTC+07:00, GitHub marked v0.26.0 as the latest vLLM release. It was published on July 27, 2026.[21] This statement is intentionally time-bounded. The project has an active release cadence, so the repository's releases page should be consulted for the current version.

The mature subject of the article is the engine's architecture and role, not a release-by-release feature catalog. Exact model additions, kernel changes, deprecations, accelerator support, and experimental features belong in versioned release notes and documentation. Keeping those volatile matrices out of the encyclopedic core avoids presenting a transient snapshot as a durable property of vLLM.

## See also

- [Attention](https://aiwiki.ai/wiki/attention)
- [PyTorch](https://aiwiki.ai/wiki/pytorch)
- [SGLang](https://aiwiki.ai/wiki/sglang)
- [NVIDIA TensorRT-LLM](https://aiwiki.ai/wiki/tensorrt_llm)

## References

1. Woosuk Kwon et al. "Efficient Memory Management for Large Language Model Serving with PagedAttention." SOSP 2023. https://arxiv.org/abs/2309.06180
2. vLLM Project. "vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention." June 20, 2023. https://vllm.ai/blog/2023-06-20-vllm
3. PyTorch Foundation. "vLLM." Project overview. https://pytorch.org/projects/vllm/
4. PyTorch Foundation. "PyTorch Foundation Welcomes vLLM as a Hosted Project." May 7, 2025. https://pytorch.org/blog/pytorch-foundation-welcomes-vllm/
5. vLLM Project. "Apache License, Version 2.0." v0.26.0 repository license. https://github.com/vllm-project/vllm/blob/v0.26.0/LICENSE
6. vLLM Project. "Quickstart." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/getting_started/quickstart.md
7. vLLM Project. "Architecture Overview." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/design/arch_overview.md
8. vLLM Project. "vLLM V1." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/usage/v1_guide.md
9. vLLM Project. "OpenAI-Compatible Server." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/serving/online_serving/openai_compatible_server.md
10. vLLM Project. "Automatic Prefix Caching." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/features/automatic_prefix_caching.md
11. vLLM Project. "Optimization and Tuning." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/configuration/optimization.md
12. vLLM Project. "LoRA Adapters." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/features/lora.md
13. vLLM Project. "Speculative Decoding." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/features/speculative_decoding/README.md
14. vLLM Project. "Quantization." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/features/quantization/README.md
15. vLLM Project. "Supported Models." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/models/supported_models.md
16. vLLM Project. "Parallelism and Scaling." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/serving/parallelism_scaling.md
17. vLLM Project. "Data Parallel Deployment." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/serving/data_parallel_deployment.md
18. vLLM Project. "Production Metrics." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/usage/metrics.md
19. vLLM Project. "Security." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/usage/security.md
20. vLLM Project. "Benchmark CLI." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/benchmarking/cli.md
21. vLLM Project. "vLLM v0.26.0 Release Notes." July 27, 2026. https://github.com/vllm-project/vllm/releases/tag/v0.26.0
22. vLLM Project. "Reproducibility." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/usage/reproducibility.md
23. vLLM Project. "torch.compile integration." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/design/torch_compile.md
24. vLLM Project. "Using Docker." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/deployment/docker.md
25. vLLM Project. "vLLM repository overview." v0.26.0. https://github.com/vllm-project/vllm/blob/v0.26.0/README.md
26. vLLM Project. "NVIDIA CUDA GPU installation source." v0.26.0 documentation source. https://github.com/vllm-project/vllm/blob/v0.26.0/docs/getting_started/installation/gpu.cuda.inc.md