NVIDIA BioNeMo Inference Runtime

NVIDIA BioNeMo Inference Runtime (BioIR) is a GPU-accelerated Python library for biomolecular structure prediction inference. NVIDIA announced it in public beta on 10 September 2026, alongside a developer blog post describing how to run it at proteome scale. ^[1][2] BioIR keeps folding models as ordinary PyTorch torch.nn.Module objects, with no engine build, export step or separate build artifact between a checkpoint and the forward pass, and accelerates them with fused CUDA kernels, CUDA Graph capture, and a pipeline that places one complete model replica on each visible GPU. ^[2][3]

The library sits at the model-execution layer of the structure prediction stack. It runs AlphaFold 2 lineage and all-atom cofolding models from prepared inputs and writes PDB or mmCIF structures with confidence scores, but it does not build multiple sequence alignments (MSAs) and does not run the HHsearch or HMMsearch template searches. Callers stage those themselves and hand BioIR an A3M alignment per protein chain, plus any template CIF files they want used. ^[1][4] The code is published under the Apache License 2.0 at github.com/NVIDIA-BioNeMo/BioNeMo-Inference-Runtime, with release wheels on PyPI as bionemo-ir. ^[5][6][7]

Infobox

Why a separate runtime for folding models

Structure prediction models in the AlphaFold family are not shaped like the transformer stacks that general-purpose inference servers are tuned for. Their cost concentrates in operations that barely appear elsewhere: Evoformer and Pairformer trunks, triangle attention and triangle multiplication over an N-by-N pair representation, attention with pair bias, and, in the AlphaFold 3 generation of models, an atom-level diffusion module run repeatedly over sampling steps. The pair tensor grows with the square of sequence length, so memory rather than arithmetic is often the binding constraint. ^[3][4]

NVIDIA's stated position is that general-purpose inference stacks do not fully optimize these operations, and that BioIR exists to supply biology-aware PyTorch modules, kernels and graph optimizations for them while leaving the development workflow in Python. ^[3] The target users named in the documentation are foundation-model developers and machine learning researchers who build or modify biomolecular architectures, rather than end users of a hosted prediction service. ^[3]

Three levels of acceleration

NVIDIA's documentation separates BioIR's optimizations into three layers that address different bottlenecks, and is explicit that they are not interchangeable. ^[1][9]

The optimize() mechanism selects nothing about kernels. It is the CUDA Graph path: each requested submodule is swapped for a tracker that keeps the original module as its eager fallback, and the captured graph is replayed across sampling steps. Targets are discovered from a decorator on the module tree rather than hand-listed, and a config key that matches nothing raises an error instead of silently doing nothing. ^[9] OpenFold2 and AlphaFold2 have no CUDA-graph module in BioIR; the gain from this path comes from capturing the diffusion token transformer on Boltz-1, Boltz-2, OpenFold3 and Protenix. ^[4][9]

The Ray layer is deliberately narrow. It places a complete model replica on each visible GPU on one node and feeds independent records to them, and NVIDIA's documentation states plainly that it does not split a single forward pass across GPUs and does not reduce the latency of one prediction. ^[1][3] Splitting one forward pass, by model or context parallelism of the kind in NVIDIA's separate Fold-CP work on Boltz-2, is described in the support matrix as planned rather than available. ^[4]

Lower GPU memory pressure is presented as a consequence of the same layers rather than a separate feature. Fused operations avoid materializing the intermediates their eager equivalents build, output-row chunking bounds peak activations on the O(N^2) pair tensors above a device-memory-dependent threshold (2,560 residues on an 80 GB GPU), and attention masks are precomputed once before the layer loop instead of rebuilt per layer. ^[9]

Two ways to use it

BioIR exposes two workflows that share the same compute path. ^[3][9]

The end-to-end processor runs a supported model from a JSON InputRequest to written structures. Five stages consume inputs in dependency order: parser, tokenizer, feature generator, folding engine, writer. The parser materializes FASTA and A3M files referenced by the request and deduplicates them by content hash; the tokenizer and feature generator assemble batch tensors; the folding engine builds the module, applies optimize() when an accelerated config is present, and runs the forward pass under torch.inference_mode(); the writer serializes each result to PDB, mmCIF or both in one pass, with scores returned as a JSON string. ^[1][9] Two executor backends drive those stages: a serial backend that completes one input at a time, which NVIDIA recommends for setup validation and per-request timing, and the Ray backend that maps them onto a Ray Data pipeline. ^[1][3]

Direct PyTorch integration skips the pipeline. A caller constructs a supported model as an nn.Module, or reuses individual optimized modules such as a Pairformer stack, an Evoformer, or a diffusion transformer inside an architecture of their own. NVIDIA states that a custom model does not need to belong to a supported end-to-end family; it only needs to contain a module BioIR optimizes. Kernel and module optimizations apply to both workflows because they are properties of the modules; pipeline scaling applies only to the processor path. ^[3][9]

Stage-level tuning is exposed through per-stage configs. Each stage takes a compute worker count, most take num_cpus, memory and batch_size, and the Ray engine stage adds num_gpus, max_concurrent_batches and accelerator_type. The capacity rule is that the engine stage's compute multiplied by its num_gpus must not exceed the visible GPU count. NVIDIA's guidance is to start from the default replica-mode config, add parser, tokenizer or feature workers when engines wait on input, and reduce concurrency or separate large inputs when GPU or object-store memory causes failures. ^[1]

Supported models and inputs

The model keys BioIR accepts, and whether each one can run through the end-to-end pipeline, are listed in the repository support matrix. ^[4]

Input coverage differs by family. AlphaFold2 and OpenFold2 fold protein chains only and require an unpaired A3M MSA per chain; Boltz-1, Boltz-2 and OpenFold3 additionally accept RNA, DNA and ligand chains given as CCD codes or SMILES strings, treat paired MSAs as optional, and still require an MSA on every protein chain. Templates are accepted only on protein chains and must be supplied by the caller as CIF or PDB hits. ^[4]

Hardware coverage

NVIDIA qualified six GPUs for the first release: H200, H100, A100, L40S, GB200 and GB300. The support matrix separately lists every architecture the backend covers, and is careful to mark that table as backend compatibility rather than release qualification. ^[2][4]

The fused kernels cover triangle attention, pairwise attention, dual GEMMs for triangle multiplication, pair-weighted averaging, outer-product mean, gated sigmoid, adaptive layer normalization, and two Triton fused ops for layer-norm-plus-projection and SwiGLU. ^[4]

An unusual detail of the release is what is and is not open. The repository and the wheels ship the Python callables plus precompiled CUBIN payloads for every CuTeDSL kernel, and NVIDIA's support matrix states that the CuTeDSL kernel source is not open-sourced and that there is no plan to publish it. Shipping the CUBINs avoids CuTeDSL just-in-time compilation, so those kernels run at full speed from the first iteration; where a CUBIN is missing for the current architecture or dtype, the operation falls back to PyTorch rather than compiling. The Triton fused ops remain ordinary Python source and still compile on first use. ^[4]

Performance reported by NVIDIA

All figures below are NVIDIA's own measurements published in the BioIR repository and developer blog. They are vendor benchmarks, not independent evaluations.

Model-forward speedups

The repository publishes a generated benchmark summary comparing BioIR against two baselines of the same reference implementations: the open-source code under torch.compile, and the same code in PyTorch eager mode. Speedup is defined as the OSS forward time divided by the BioIR forward time per sample, reported as a geometric mean of per-sample ratios rather than a ratio of totals. ^[2][11]

Source: repository benchmark summary. ^[2] Figure 2 of the launch blog post reports the same torch.compile comparison as 1.55x and 1.54x for OpenFold3, 1.78x and 1.75x for Boltz-2, and 2.56x and 2.61x for OpenFold2 monomer on H100 and H200, differing from the repository table by at most 0.01x. Those blog measurements used one discarded warmup call and one measurement call across 17 inputs spanning 29 to 1,734 residues, with the OSS baselines configured to use cuEquivariance and, for OpenFold3, DeepSpeed. ^[1]

NVIDIA's benchmark page is unusually direct about where the numbers do not hold. It reports that torch.compile is ahead of BioIR on 21% of individual measurements, that this is concentrated on longer inputs and only on Boltz-2 and OpenFold3, and that the ratio against torch.compile ranges from roughly 1.5x to 3.0x on Ampere, Hopper and Ada down to 1.0x to 2.1x on Blackwell and 0.8x to 1.7x on the GB10 desktop part. It also breaks every result out by residue count on the grounds that an overall geometric mean hides too much: Boltz-2 on A100, for instance, is reported at 1.99x below 512 residues and 4.23x above 1,024 against torch.compile. ^[11]

Memory and accuracy

For Boltz-2, the published peak allocated memory is 20.34 GiB for BioIR against 32.27 GiB for both OSS baselines on H100 and H200, a ratio of 0.63x. Accuracy is scored with OpenStructure lDDT over every sample and DockQ over the subset with a supported protein interface; on H100 the three implementations score 0.667, 0.665 and 0.664 lDDT and 0.611, 0.606 and 0.601 DockQ, which NVIDIA frames as the expected outcome on the grounds that a speedup which moved those scores would not be the same computation. ^[11]

The 1,000-target dimer benchmark

The launch blog post reports a matched end-to-end comparison on 1,000 human dimer targets with combined sequence lengths below 2,800 residues, run on 8 H100 GPUs with three recycles, 200 sampling steps and five diffusion samples per target. Both sides used the same targets, staged MSAs, inference recipe and GPU configuration. ^[1]

NVIDIA describes the result as a 2.90x improvement in residue-normalized throughput, and states that the throughput metrics are specific to that configuration and should not be generalized to other BioIR-supported models, datasets or hardware. The comparison measures folding only: it excludes MSA generation, preprocessing CPU allocations, storage, data transfer, retries and engineering overhead. ^[1]

Energy estimates

From the same 1,000-target run, NVIDIA linearly extrapolated to one million comparable targets using rated-power equivalents for an 8-way H100 80 GB HBM3 node. The estimate is 11 MWh for BioIR against 35 MWh for the public implementation using 8-GPU thermal design power equivalents, and 21 MWh against 64 MWh using full-node maximum power. NVIDIA labels these as folding-only estimates for IT equipment rather than metered energy measurements, and states that they exclude data center overhead such as power usage effectiveness. ^[1]

Use in the AlphaFold Database expansion

NVIDIA's blog post cites the recent expansion of the AlphaFold Protein Structure Database as real proteome-scale work in which BioIR was used, and states that an early version of the library contributed accelerated modules to an NVIDIA-internal version of OpenFold2-MM. ^[1]

The underlying project is a four-way collaboration between EMBL's European Bioinformatics Institute, Google DeepMind, NVIDIA and Seoul National University, announced by EMBL-EBI on 16 March 2026. The accompanying preprint reports predictions for 23,441,822 homodimers and 7,620,644 heterodimers drawn from 4,777 proteomes, about 31 million candidate complexes in total, and the release of 1,754,242 high-confidence homodimers through the database, selected with a combined threshold on ipSAE, average pLDDT and backbone clashes. A further 56,959 heterodimers passed the same filters and were analysed alongside them, giving the 1,811,201 structures used in the clustering analysis. The preprint is inconsistent on that heterodimer count: its results text gives 56,956 while its Figure 4 caption and its methods give 56,959, and only the latter is consistent with the 1,811,201 total. EMBL-EBI's announcement adds that roughly 18 million lower-confidence homodimers are available by bulk download from its FTP server, and a 19 May 2026 update to that announcement records almost 80,000 high-confidence heterodimer predictions added to the database plus 8.1 million lower-confidence heterodimers available for download. EMBL-EBI put the compute that the collaboration absorbed on behalf of the community at around 17 million GPU hours. ^[12][13]

NVIDIA's blog post puts the outcome at "1.81 million released as high-confidence predictions", which matches neither figure the preprint attaches to the release: the availability statement releases 1,754,242 high-confidence homodimers and the abstract rounds that to 1.8M, while 1,811,201 is the size of the combined homodimer and heterodimer set that the preprint clusters. The preprint itself describes the accelerated inference path as an implementation of OpenFold using NVIDIA TensorRT and cuEquivariance, run alongside ColabFold on a DGX H100 SuperPOD, with AlphaFold-Multimer model_1_multimer_v3 weights, four recycles with early stopping and no relaxation. The BioIR contribution to that internal implementation is NVIDIA's own account of it. ^[1][12]

Release and packaging

The GitHub repository was created on 30 July 2026 and the first tagged prerelease, 0.1.0rc1, was published on 9 September 2026 with wheels for x86_64 and aarch64 built against a CUDA 13.2 PyTorch container. The bionemo-ir project on PyPI carries 0.1.0rc0 and 0.1.0rc1 from 10 September 2026 and the 0.1.0 release the same day, roughly the same hour as NVIDIA Healthcare's announcement post. ^[6][7][1]

The wheels ship kernels precompiled, so installation builds no CUDA and running needs only the driver's libcuda.so.1; the documented prerequisites are Python 3.12, Linux on x86_64 or aarch64 with glibc 2.34 or newer, and driver 580 or newer. NVIDIA warns that running the CUDA 13.2 build through an older driver's forward-compatibility shim has been observed to hang or crash part-way through a run, and that results from such a configuration are discarded rather than corrected. Building from source additionally requires Docker and the NVIDIA Container Toolkit, and model checkpoints are fetched from their upstream publishers without NVIDIA credentials. ^[2]

NVIDIA's announcement post credited a set of external collaborators on the work, among them Apheris, Aureka Biotechnologies, Boltz, Dataiku, the Institute for Protein Design at the University of Washington, Latent Labs, MoleculeMind, Nebius, the Open Molecular Software Foundation, the OpenFold consortium, Proxima, SandboxAQ, Terray Therapeutics and Xaira Therapeutics. ^[14]

Relationship to the BioNeMo platform

BioIR is one component of NVIDIA BioNeMo, the company's platform for AI in digital biology and drug discovery, which also includes the open-source BioNeMo Framework, NIM inference microservices for models such as OpenFold2, OpenFold3, Boltz-2 and DiffDock, reference workflows NVIDIA calls BioNeMo Blueprints, and the BioNeMo Agent Toolkit. ^[10][15] Despite the shared naming, BioNeMo is a separate product line from NVIDIA NeMo, which addresses general-purpose language and agent models. NVIDIA's launch post points users who want agentic orchestration at the BioNeMo Agent Toolkit rather than at the runtime. ^[1]

See also

• Protein folding
• Boltz
• AlphaFold 3
• Inference optimization
• Ray Serve

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21Tretina, Kyle; Tal, Roy; Salehi, Mahan; Patel, Neel. "High-Throughput Structure Prediction with BioNeMo Inference Runtime", NVIDIA Technical Blog, 10 September 2026. developer.nvidia.com/...-bionemo-inference-runtime
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10NVIDIA. "BioNeMo Inference Runtime" README, GitHub, retrieved 11 September 2026. github.com/...BioNeMo-Inference-Runtime
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NVIDIA. "NVIDIA BioNeMo Inference Runtime" overview, BioIR documentation (`docs/fern/pages/overview.mdx`), retrieved 11 September 2026. github.com/...overview.mdx
4. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25NVIDIA. "BioIR Support Matrix", BioIR documentation (`docs/ref/support-matrix.md`), retrieved 11 September 2026. github.com/...support-matrix.md
5. ^1 ^2NVIDIA. LICENSE (Apache License 2.0), BioNeMo-Inference-Runtime repository, retrieved 11 September 2026. github.com/...LICENSE
6. ^1 ^2GitHub REST API, repository and releases for NVIDIA-BioNeMo/BioNeMo-Inference-Runtime, retrieved 11 September 2026. api.github.com/...releases
7. ^1 ^2 ^3Python Package Index, project `bionemo-ir`, retrieved 11 September 2026. pypi.org/...bionemo-ir
8. ^NVIDIA. CITATION.cff, BioNeMo-Inference-Runtime repository, retrieved 11 September 2026. github.com/...CITATION.cff
9. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NVIDIA. "Architecture", BioIR documentation (`docs/ref/architecture.md`), retrieved 11 September 2026. github.com/...architecture.md
10. ^1 ^2NVIDIA. "NVIDIA Opens BioNeMo to Scale Digital Biology for Global Biopharma and Scientific Industry", NVIDIA Newsroom, 18 November 2024. nvidianews.nvidia.com/...a-and-scientific-industry
11. ^1 ^2 ^3NVIDIA. "Benchmarks", BioIR documentation (`docs/ref/benchmark.md`), retrieved 11 September 2026. github.com/...benchmark.md
12. ^1 ^2Han, Yewon; Tsenkov, Maxim I.; Venanzi, Niccolo A. E.; et al. "AlphaFold Database expands to proteome-scale quaternary structures", preprint, NVIDIA Research. research.nvidia.com/...afdb.pdf
13. ^Stroe, Oana. "Millions of protein complexes added to AlphaFold Database shed light on how proteins interact", EMBL-EBI news, 16 March 2026, with update of 19 May 2026. ebi.ac.uk/...first-complexes-alphafold-database
14. ^NVIDIA Healthcare (@NVIDIAHealth), announcement post, 10 September 2026. x.com/...2098065212767084753
15. ^NVIDIA. "NVIDIA BioNeMo Agent Toolkit" README, GitHub, retrieved 11 September 2026. github.com/...bionemo-agent-toolkit