Luminal

Luminal (Luminal AI Inc.) is a San Francisco company that builds an open-source AI inference compiler and sells inference capacity and licensed deployments built on it. The compiler, also called Luminal, is written in Rust, represents a model as a graph of a small set of primitive operations, and chooses kernels by searching a space of logically equivalent implementations rather than by applying a fixed sequence of hand-written optimization passes.^[1][2] The company was founded by Joe Fioti (chief executive), Matthew Gunton (chief technology officer) and Jake Stevens, went through Y Combinator's Summer 2025 batch, and announced a $5.3 million seed round led by Felicis Ventures on November 17, 2025.^[3][4] The open-source project predates the company: its GitHub repository was created by Fioti in July 2023, while Y Combinator lists 2025 as the founding year.^[5][6]

Origins

The luminal repository was created on July 18, 2023, with an initial commit by Joe Fioti, under his personal GitHub account (jafioti/luminal); the same day a luminal crate appeared on crates.io with the description "Deep learning at the speed of light."^[5][7] The project's early documentation described it as "a deep learning library that uses composable compilers to achieve high performance" and contrasted its small core with PyTorch, which it said contained "hundreds of thousands of lines of code."^[8] The project website luminalai.com was live by April 2024 with the same tagline, and Fioti posted the project to Hacker News in March 2024 under the title "Luminal: Fast ML in Rust through graph compilation."^[9] The crates.io package has not been updated since version 0.2.0 in March 2024, and the repository's manifest still carries that version number; development has continued on GitHub rather than through crate releases.^[7]

A GitHub organization, luminal-ai, was created on February 3, 2025, and the repository moved under it.^[10] Fioti presented "Luminal - Search-Based Deep Learning Compilers" at the AI Engineer World's Fair in June 2025, and the founders published a Y Combinator launch post titled "Luminal - PyTorch for Production" on July 20, 2025.^[11][12] In that post the founders said the compiler was "already powering research at Yale, production workloads at VC-backed startups and several research labs," a company statement without named customers.^[12] A Show HN post on August 20, 2025 ("Luminal - Open-source, search-based GPU compiler") reached 153 points; in the thread Fioti described the approach as "very similar to superoptimisation" but tractable because it covers only linear algebra over a small set of primitive operations.^[13]

TechCrunch reported that Fioti had been working on chip design at Intel about three years before the November 2025 funding announcement when he concluded that "the more important bottleneck was in software."^[3] His Y Combinator profile says he worked on CPU microcode and ML accelerators at Intel; the launch post's version of the biography, that "every Intel chip sold has the AI accelerator Joe worked on," is the company's own wording.^[4][12]

Founders and team

Scroll sideways for more →

TechCrunch confirms the Apple and Amazon backgrounds of Stevens and Gunton.^[3] Y Combinator's company page listed a team size of seven and three open roles (compiler engineer, cloud inference engineer, senior compiler engineer) as of September 6, 2026, with Jared Friedman as the batch partner.^[4] The careers page says the team works in person from a San Francisco office.^[14]

The compiler

Primitive operations

Luminal's core design decision is a very small operation vocabulary. Every model, including transformers and convolutional networks, is lowered to a directed acyclic graph of primitive operations, and everything else (subtraction, division, matrix multiplication, convolution, attention) is expressed as compositions of those primitives; a matrix multiply, for example, becomes a broadcast multiply followed by a sum reduction.^[8][11] The set has grown since the project started:

Scroll sideways for more →

The current core specification describes the high-level intermediate representation (HLIR) as "a complete bit-level logical specification of a tensor program" and lists, in addition to the 15 computational primitives, Input, Output and Constant nodes.^[15] The README says the goal is that "it should be possible to understand the entire core library in an afternoon"; in the June 2025 talk Fioti put the implementation at under 5,000 lines of code, without saying what was counted.^[1][11] The full repository, including backends and examples, contained about 159,000 lines of Rust on September 6, 2026.^[5]

Search-based compilation

Instead of a pipeline of rewrite passes that each fire only when they are known to help, Luminal builds a search space of equivalent programs and measures candidates. The core spec's compile flow runs from a frontend program (PyTorch's Core ATen IR via torch.compile, or the project's own GraphTensor API in Rust) to an HLIR graph, a loop-rolled HLIR graph, and then "egglog saturation," which applies rewrite rules inside an e-graph, a data structure that stores many equivalent expressions compactly.^[15] The result is one saturated e-graph per bucket of dynamic dimensions (such as sequence length). A runtime then extracts candidate low-level graphs, compiles and profiles them on the target hardware, and keeps the fastest per bucket.^[15] The extraction step uses a genetic search in the shipped utilities; in 2025 Fioti described Monte Carlo tree search as the way to narrow the space when exhaustive profiling becomes infeasible, and said the prototype used roughly 20 to 25 rewrite rules.^[11][15][16]

The README states the philosophy as "the best heuristic is no heuristic," and the company's best-known claim is that this process rediscovers Flash Attention from a naive attention graph without a hand-written pass.^[1][11] In the talk, Fioti qualified the claim that Luminal was the first compiler to do so as being, to the team's knowledge, the first, and the demonstration did not include a search budget or a reproducible timing comparison.^[11] The contributor guide enforces the design rule that "all graph pattern matching and op selection must be expressed in egglog rewrites," with no Rust-side post-passes that fuse kernels or select backend ops after extraction.^[16] Dynamic shapes are handled symbolically, so a shape such as (s, 4096) stays visible to the compiler and is specialized per bucket rather than forcing a fully dynamic runtime.^[1]

The careers page describes the user-facing consequence: "give the Luminal compiler a time budget and it searches until the time runs out, returning the fastest kernel it found."^[14] In the 2025 Show HN thread Fioti said the compiler generates "millions of possible kernels" per search, that the work is done "without using LLMs or AI," and that at the time Luminal's kernels did not yet match llama.cpp's hand-tuned throughput on Apple silicon.^[13] The README's current performance statement is that Luminal runs Q8 Llama 3 8B "at ~80% of theoretical max performance on an H100," a company figure without a published methodology.^[1]

Backends, frontend and examples

The project is organized as a core crate plus plugin crates. As of September 2026 the public repository contains a CUDA backend (luminal_cuda_lite), a Metal backend for Apple GPUs (luminal_metal), a PyTorch integration (luminal_python, exposing torch.compile(model, backend=luminal_cuda)), neural-network modules, a training/autograd crate, a benchmarking crate and a tracing crate.^[1][5][16] The CUDA backend's search space includes calls into vendor and third-party libraries such as cuBLASLt and FlashInfer alongside generated kernels, and its documentation says profiling, not a cleanup rule, decides between "generic kernels, specialized kernels, and host-library calls."^[1][17] Example programs cover Llama, a paged-attention Llama variant, Gemma, Gemma 4 MoE, Qwen, Qwen3 MoE, Whisper, FLUX.2 and YOLO v11.^[5] The README lists a ROCm backend, warp- and thread-level dialects (including TMA and tcgen.05 instructions on NVIDIA Blackwell), a public benchmarking suite, automatically searched model parallelism and "more public inference accelerator backends" as roadmap items.^[1]

The code is dual licensed under MIT or Apache-2.0 at the user's option; GitHub's automatic license detection reports only Apache-2.0.^[1][6] On September 6, 2026 the repository had 2,968 stars, 231 forks, 61 branches, 50 open issues and pull requests (13 of them issues) and exactly 2,900 commits.^[6]

Megakernels

On January 9, 2026 Luminal published "Compiling Models to Megakernels," a technical post by Fioti describing how the compiler fuses an entire forward pass into one persistent GPU kernel.^[18] The post identifies three sources of idle hardware in the conventional one-kernel-per-operation model: kernel launch overhead (which CUDA Graphs only partly remove), wave quantization (some streaming multiprocessors finish early and wait), and the gap at the start of each kernel while initial weights load. It credits Hazy Research with pioneering the technique by fusing Llama 1B into a single hand-built megakernel, and frames Luminal's contribution as generating such kernels automatically for arbitrary architectures.^[18]

Luminal's megakernel is described as an interpreter running on the GPU: a single global instruction queue shared by all streaming multiprocessors, from which each one atomically pops coarse-grained instructions (for example a matmul fused with a residual add, or RMSNorm plus matmul plus RoPE). The post explains the choice of dynamic over static scheduling as a way to tolerate variable instruction latency and hardware jitter. Fine-grained dependencies use "increment-then-decrement" barrier counters, so a consumer only waits for the number of in-flight producers to reach zero rather than needing to know how many producers exist. Generation takes two passes: rewriting operations into block-level operations partitioned across SMs with strided input and output dependencies (using shape-layout algebra inside the e-graph engine), then deriving barrier strides from producer-consumer pairs. A "symbolic work queue" stores one entry per instruction with symbolic launch dimensions, so a change of sequence length does not require rebuilding the queue.^[18] The post said the work was being released in the compiler repository; the public CUDA backend's documentation describes block operations that "Luminal can fuse ... into megakernels," which are then wrapped as kernel operations inside CUDA graphs, while warp- and thread-level operations were "not yet merged" as of September 2026.^[17]

Zhihao Jia, whose group builds the Mirage Persistent Kernel compiler, replied on the post pointing to that project and inviting a comparison; a Hacker News discussion of the post in late January 2026 drew 35 points and 19 comments.^[18][19]

Numerics

A second technical post, "Producing The Perfect Token" (April 6, 2026), argues that low-precision serving has opened a quality gap between inference providers. The company claims that benchmark scores for the same model "can range as much as 20%" across clouds, citing provider comparisons of Kimi K2 and the Thinking Machines essay on nondeterminism in LLM inference; the 20% figure is Luminal's characterization of third-party benchmarks it does not name.^[20] The post walks through IEEE 754 formats, FP8, MXFP4 and NVFP4, TF32 (which it notes is a 19-bit format that users never handle directly), and the operations most sensitive to reduced precision: long accumulation chains in matrix multiplies, softmax exponentiation, variance in normalization layers, and activation outliers under INT8 quantization.^[20]

Its central claim is about the compiler's obligation: once a compiler rewrites loops and accumulation orders aggressively, "numerical error must not increase due to optimizations, or it must increase in predictable, user-controllable ways." Luminal says it enforces this by measuring absolute and relative error at the operator or subgraph level and pruning every candidate graph that contains a subgraph already shown to lose too much precision, rather than running each full candidate end to end; it discusses static interval analysis and bit-level solvers such as Daisy as complementary approaches.^[20]

Heterogeneous hardware and Positron Atlas

On May 18, 2026 Luminal announced a partnership with Positron AI, an inference-hardware company that had closed a $230 million Series B in February 2026, to bring Positron's Atlas system up as a compiler backend.^[21][22] The post, "Hardware Arbitrage with Luminal and Positron AI," describes Atlas as a set of systolic array accelerators arranged in a star topology around a host CPU, with each Archer device holding a systolic array, SRAM buffers and 32 GB of HBM. Dense matrix multiplies and attention run on the Archer devices while element-wise operations run on the host, so Luminal must generate a heavily multi-threaded host program and search over schedules that overlap host work with device dispatch.^[21]

The demonstration compiled the prefill phase of a model to a CUDA graph on an NVIDIA H100 and the decode phase of the same model definition to Atlas through a backend named luminal_tron, transferring the KV cache between the two: an instance of prefill-decode disaggregated serving across different vendors' hardware. The code change shown is one line per phase, torch.compile(model, backend=luminal_cuda) versus backend=luminal_tron.^[21] Luminal called the backend "our first public demonstration of the flexibility of this technology" and said a follow-up post "with precise performance benchmarks" was coming. As of September 6, 2026 no such post had appeared, no luminal_tron crate was present in the public repository (although an example file in it refers to a luminal_tron::dequant_matmul function), and Positron's press page did not mention the partnership; the post's subtitle and conclusion refer to Nvidia Blackwell while its described experiment used an H100.^{[5][21][22][23]}

Products and business model

Luminal's website offers two ways to use the compiler: Luminal Cloud, described as managed serverless inference endpoints with automatic batching and scale-to-zero, and on-premises or licensed deployments with dedicated engineering support, custom kernel optimization and service-level agreements.^[24] TechCrunch characterized the core business more bluntly: "the company sells compute, just like neo-cloud companies like Coreweave or Lambda Labs," but differentiates on optimization rather than GPU supply, and placed Luminal in a cohort of inference-optimization startups alongside Baseten, Together AI, Tensormesh and Clarifai.^[3] The seed announcement framed the goal as making full hardware utilization "as simple as running luminal.deploy()" and said the company was "working with companies running custom models."^[2]

The homepage carries several company performance claims: "3.2x vs vLLM," sub-10 ms p99 latency, "0% overhead," a statement that compiled models "consistently outperform existing inference engines by 2-3x on standard benchmarks," and a chart labelled "Tokens/sec, GPT-OSS 120B, 8xH100 SXM" showing Luminal at 36k tokens per second against 28k for TensorRT-LLM, 26k for vLLM and 3k for PyTorch.^[24] The chart's own numbers correspond to a 1.38x advantage over vLLM, not 3.2x, and no batch size, precision, sequence length or software versions are given. A "View full benchmarks" link on the site's inference calculator pointed to a page that returned a 404 error on September 6, 2026.^[24][25]

Scroll sideways for more →

These are Luminal's figures with no stated methodology; the wiki has not measured them.^[25]

Fioti has argued publicly that hand tuning will always win for a single model on a single chip, and that Luminal's bet is on everything short of that: "It is always going to be possible to spend six months hand tuning a model architecture on a given hardware, and you're probably going to beat any sorts of, any sort of compiler performance. But our big bet is that anything short of that, the all-purpose use case is still very economically valuable."^[3] The company's own writing extends the argument to new accelerators: because vendor software stacks have historically lagged silicon (it cites Nvidia's Hopper generation reaching "software maturity a full 2 years after release," a Luminal characterization), a search-based compiler that can retarget from PyTorch is presented as "the quickest way for a new chip to start running models."^[2][21]

Funding

Scroll sideways for more →

Y Combinator's Summer 2025 investment preceded the seed round; its standard terms are not disclosed for individual companies. No valuation has been reported.^[3][4]

Public positions on inference architecture

Fioti writes regularly about serving architecture on X. On September 4, 2026 he posted a long explainer on attention-FFN disaggregation that opens: "There's only one technique that will save SRAM-only chips: attention-feedforward disaggregation."^[26] The post separates the two layer types of a transformer by their constraints: feed-forward and mixture-of-experts layers have weights and activations whose sizes are fixed at compile time, while attention is a lower-compute workload dominated by growing, jagged KV caches that are shared through prefix tries, offloaded, or shipped in from prefill nodes. Running both on the same device, he wrote, leaves compute idle during attention and memory idle when caches are small. Splitting them lets operators put SRAM-only accelerators such as Groq's LPU or Cerebras's wafer on the feed-forward side, where they hold only weights and activations and never a KV cache, and keep attention on GPUs with large memories.^[26]

He named two costs. Communication sits inside the decode loop rather than being hidden behind prefill, so activations bounce between device pools hundreds of times per forward pass and only microsecond-scale links are tolerable; and the model topology and device ratio must be fixed in advance from a prediction of load, so a shift in context lengths strands one pool or forces cache eviction and re-prefill. The post ends by presenting Luminal's compiler as the answer to the second problem: a compiler "capable of modeling any topology," so that changing topology "becomes compiling multiple times and dynamically swapping when the demand shape shifts," and states that the company is "driving down joules-per-token and still have a couple orders of magnitude to go," a company claim.^[26]

Fioti also wrote that this is "exactly the approach Nvidia is taking with their SRAM-only Groq LPX system, and Cerebras is targeting with their partnerships with AMD and AWS Trainium." The first half matches NVIDIA's own description: its March 2026 technical blog on the Groq 3 LPX rack says the Vera Rubin decode loop "separates attention from FFN within decode and exchanges intermediate activations for each token," a split it calls "attention-FFN disaggregation (AFD)," orchestrated by NVIDIA Dynamo.^[27] The second half is not supported by the announcements themselves: the AWS-Cerebras collaboration of March 13, 2026 pairs "AWS Trainium optimized for prefill and Cerebras CS-3 optimized for decode," and the AMD-Cerebras partnership of July 23, 2026 has AMD Helios "processing prompts and large context windows" while the Wafer-Scale Engine "accelerates the memory-bandwidth-intensive token generation." Both are prefill-decode splits in which Cerebras runs the whole decode step, attention included, rather than attention-FFN splits.^[28][29]

Timeline

Scroll sideways for more →

References

1. ^luminal-ai/luminal README - GitHub (Luminal), retrieved Sep 6, 2026.
2. ^Announcing our $5.3M Seed Round - Luminal Blog (Luminal), Nov 18, 2025.
3. ^Luminal raises $5.3 million to build a better GPU code framework - TechCrunch (Russell Brandom), Nov 17, 2025.
4. ^Luminal: Making AI run fast on any hardware. - Y Combinator, retrieved Sep 6, 2026.
5. ^luminal-ai/luminal repository metadata and commit history - GitHub REST API, retrieved Sep 6, 2026.
6. ^luminal-ai/luminal contributors - GitHub REST API, retrieved Sep 6, 2026.
7. ^luminal crate - crates.io API, retrieved Sep 6, 2026.
8. ^Luminal: Efficient ML in Rust through graph compilation - Luminal documentation (Luminal), undated (2024).
9. ^Wayback Machine capture of luminalai.com - Internet Archive, Apr 16, 2024.
10. ^luminal-ai organization metadata - GitHub REST API, retrieved Sep 6, 2026.
11. ^Luminal - Search-Based Deep Learning Compilers - Joe Fioti - AI Engineer World's Fair 2025 (Joe Fioti), Jun 3, 2025.
12. ^Luminal - PyTorch for Production - Y Combinator Launches (Joe Fioti, Jake Stevens, Matthew Gunton), Jul 20, 2025.
13. ^Show HN: Luminal - Open-source, search-based GPU compiler - Hacker News (Joe Fioti), Aug 20, 2025.
14. ^Careers - Luminal - Luminal, retrieved Sep 6, 2026.
15. ^Luminal Core Spec (spec.md) - GitHub (Luminal), retrieved Sep 6, 2026.
16. ^Contributor Guide (AGENTS.md) - GitHub (Luminal), retrieved Sep 6, 2026.
17. ^luminal_cuda_lite README - GitHub (Luminal), retrieved Sep 6, 2026.
18. ^Compiling Models to Megakernels - Luminal Blog (Joe Fioti), Jan 9, 2026.
19. ^Compiling models to megakernels - Hacker News discussion, Jan 25, 2026.
20. ^Producing The Perfect Token - Luminal Blog (Joe Fioti), Apr 6, 2026.
21. ^Hardware Arbitrage with Luminal and Positron AI - Luminal Blog (Luminal), May 18, 2026.
22. ^Press - Positron AI, retrieved Sep 6, 2026.
23. ^Luminal Blog archive - Luminal Blog, retrieved Sep 6, 2026.
24. ^Luminal - Inference at the Speed of Light - Luminal, retrieved Sep 6, 2026.
25. ^Inference Calculator - Luminal, retrieved Sep 6, 2026.
26. ^Post on attention-feedforward disaggregation - X (Joe Fioti), Sep 4, 2026.
27. ^Inside NVIDIA Groq 3 LPX: The Low-Latency Inference Accelerator for the NVIDIA Vera Rubin Platform - NVIDIA Technical Blog (Kyle Aubrey and Farshad Ghodsian), Mar 16, 2026.
28. ^AWS and Cerebras Collaboration Aims to Set a New Standard for AI Inference Speed and Performance in the Cloud - Cerebras Systems, Mar 13, 2026.
29. ^AMD and Cerebras Announce Industry-Leading Ultra-Low-Latency and High Throughput AI Inference Solution - Cerebras Systems, Jul 23, 2026.