SubQ

SubQ is a family of large language models and decision models built by Subquadratic, an AI startup that emerged from stealth on May 5, 2026, claiming to have built the first large language model on a fully subquadratic attention architecture.^[2] Subquadratic says its proprietary mechanism, Subquadratic Sparse Attention (SSA), makes attention compute grow linearly rather than quadratically with input length, and that a SubQ research model can read up to 12 million tokens in a single pass. At launch, frontier models from OpenAI, Anthropic and Google offered context windows of up to about 1 million tokens.^[1][2][5] The first model, SubQ 1M-Preview, was followed by SubQ 1.1 Small, whose model card and technical report were published on June 16, 2026 together with benchmark results checked by the data company Appen,^[9][10][11] and by SubQ Snap, a long-context decision model released in public preview on October 12, 2026 and updated to version 1.1 ten days later.^[14][16]

The pitch rests on a single idea. Standard transformer attention scales quadratically with sequence length, so doubling the input quadruples the compute. Subquadratic says it redesigned attention to scale linearly instead, which is what makes a 12-million-token window economically plausible rather than merely possible. The launch drew immediate skepticism because the company offered no technical report, no open weights and no broad public access, and researchers asked for independent proof.^[3] By October 2026 it had published a technical report and commissioned third-party benchmarks, but the report does not describe how SSA selects tokens, the SubQ language model was available only to selected design partners, and the models were offered only through Subquadratic's own API and early-access program.^{[10][12][14][16][20]}

Company and founders

Subquadratic was co-founded by Justin Dangel and Alexander (Alex) Whedon.^[1][2] According to The New Stack, the company was previously called Aldea and worked on speech models before pivoting.^[5] An archived December 2025 version of the Aldea website advertised speech-to-text, speech-to-speech and text-to-speech models and listed a large language model as "coming soon".^[21] Coverage of the May 2026 launch described Subquadratic as a Miami-based startup.^[3][5] By October 2026 The New Stack described it as San Francisco-based, and the company's LinkedIn page listed San Francisco as its headquarters with a second location in Miami.^[15][19]

Dangel, the chief executive at launch, is described as a five-time founder with prior companies in health tech, insurance technology, and consumer products.^[2][4] Whedon, the chief technology officer at launch, was a software engineer at Meta and later head of generative AI at TribeAI, where the company says he led more than 40 enterprise AI deployments.^[2][4] Whedon later succeeded Dangel as CEO, and Dangel became chairman, according to The New Stack's October 2026 reporting;^[15] Whedon's X profile lists him as co-founder and CEO of Subquadratic.^[18]

At launch the research team was small. Subquadratic said it employed 11 PhD researchers and research engineers with backgrounds at Meta, Google, Oxford, Cambridge, ByteDance, Adobe, and Microsoft.^[2][3] VentureBeat noted that neither co-founder had published foundational AI research.^[3] The SubQ 1.1 Small technical report lists Saul Ramirez and Alex Whedon as equal-contribution lead authors, with Ashmal Vayani and Phong Vo as co-authors.^[10]

Timeline

The subquadratic architecture

The core technology is what the company calls Subquadratic Sparse Attention, abbreviated SSA. In a normal transformer, every token attends to every other token, an all-pairs comparison whose cost grows with the square of the sequence length. SSA replaces that with content-dependent selection: for each query, the model selects the positions in the sequence that are worth attending to, then computes exact attention only over those positions.^[3][6] The company says attention cost then grows with the number of selected positions rather than with the full sequence, so compute and memory scale linearly with context length;^[6] it later described its Snap models as having "constant compute per token".^[16]

The distinction from earlier sparse-attention work is that SSA chooses which tokens to attend to based on their content, not on a fixed geometric pattern such as a sliding window or a stride.^[6] Dangel summarized the goal plainly in an interview: "let's try to figure out how to not compare every token to every token to every token."^[1] Whedon framed the payoff in scaling terms: with quadratic scaling, doubling the input requires four times the compute, while with linear scaling it requires only twice as much.^[1]

This places SubQ in a lineage of attempts to escape the quadratic bottleneck, a lineage that includes Mamba, RWKV, DeepSeek Sparse Attention, and Kimi Linear. Reducing the cost of long context is one of the central problems in the field, because the quadratic term is what makes processing a whole codebase or a library of documents expensive. SubQ's specific claim is that it gets the efficiency those approaches aimed for without the accuracy penalty that has historically come with them.

Subquadratic's own account of that history, set out in its research post and technical report, runs as follows. Fixed-pattern sparse attention such as Longformer decides where to look by position, so it misses information outside the pattern. State space models and other recurrent designs compress the past into a fixed-size state and are weaker at exact retrieval. Hybrids such as Jamba, Kimi Linear, Qwen3-Next and Nvidia's Nemotron 3 keep some dense attention layers, whose quadratic cost dominates again at very long lengths.^[6][10] DeepSeek's learned sparse attention selects positions with a "Lightning Indexer" that still scores every query against every key. The report calculates that, in a DeepSeek V3.2-style configuration, the indexer becomes more expensive than the sparse attention it feeds above about 52,000 tokens, reaching about 16 times the attention cost at 1 million tokens and 190 times at 12 million.^[10] Whedon told The New Stack that in Subquadratic's own measurement of GLM 5.2, 58 percent of prefill latency at 1 million tokens came from the selection mechanism.^[13] Subquadratic says SSA's selection, retrieval and attention steps are each linear in sequence length.^[10]

The company has not published how SSA actually chooses positions. The technical report states that "the mechanism by which SSA meets these requirements is outside the scope of this report",^[10] and Whedon told MIT Technology Review that the selection method is "kind of where the secret sauce is."^[12] In a July 2026 interview Whedon said SSA is "one of a couple model architectures" the company has worked with and that it has also been developing architectures that drop attention altogether: "We're not a sparse attention company either."^[13]

Launch model and performance claims

Subquadratic's first model was named SubQ 1M-Preview, and the company presented the 12-million-token figure as a research result.^[2] SiliconANGLE and The New Stack reported that the API would give developers access to the full 12-million-token window.^[1][5] SiliconANGLE estimated that 12 million tokens is about 9 million words, or almost 120 books.^[1]

The company published a set of benchmark results to support its efficiency and accuracy claims. The launch figures below are reported by Subquadratic; some were later checked by Appen under contract with the company, as described in the next section.

The figures varied between sources. Launch-day coverage in The New Stack quoted 97.1 on RULER and 82.4% on SWE-Bench Verified,^[5] while the company's pages list 95.6 and 81.8.^[2][6] The gap between the research-model MRCR v2 score of 83 and the production model's 65.9 was, in VentureBeat's words, "notable and largely unexplained".^[3] The New Stack reported that, according to the company's technical paper, each model was run only once because of high inference cost, and that the paper described the SWE-Bench margin as "harness as much as model".^[5]

Alongside the API, Subquadratic launched SubQ Code, a command-line coding agent built to load an entire repository into one context window so a developer can plan, write, and review changes across a codebase without orchestrating multiple agents.^[1] It also announced SubQ Search, a long-context research tool, which Dangel said would initially be free; all three products entered private beta on launch day.^[1][2][4] The New Stack reported that the products ran on neoclouds rather than the major hyperscalers, which Dangel called "very expensive".^[5] The company said the model would not be open-weight in the near term, though it could be trained for specific customer workloads,^[1] and it set a target of a 50-million-token context window for the fourth quarter.^[3][5]

Third-party evaluation

Subquadratic asked the data company Appen to test its claims.^[11][12] In a report dated May 11, 2026, Appen said it measured SSA against FlashAttention-2 on Nvidia B200 hardware and found a 56.2x end-to-end speedup at 1 million tokens (381 milliseconds against 21.4 seconds) and a 62.8x reduction in attention FLOPs.^[7] For those kernel measurements Appen was given access to the key algorithm code; for the benchmark runs its access was limited to Subquadratic's API endpoints.^[7] Appen reported 95.6% on RULER at 131,072 tokens (with Claude Opus 4.6 used as an LLM judge for the question-answering and extraction tasks), 86.2% on the 8-needle tier of MRCR at 524,288 to 1,048,576 tokens, and 81.8% on SWE-Bench Verified with extended thinking enabled.^[7] Subquadratic updated its SSA research post on May 15 with these figures.^[6]

A second Appen report, published on June 16 alongside SubQ 1.1 Small, covered needle-in-a-haystack retrieval and LiveCodeBench in its public summary, and Appen described the full assessment as also spanning business-workflow automation and graduate-level reasoning benchmarks (see below).^[11] Appen's Jeanine Sinanan-Singh told MIT Technology Review that the results "validated their architecture", adding that "when you have kind of shocking results, it's really not as credible when you say it yourself."^[12] Whedon told the magazine that releasing the third-party benchmarks with the initial announcement "would have preempted much of the skepticism".^[12]

On May 14, 2026, Subquadratic announced a partnership with LayerLens to evaluate SubQ on its Stratix benchmark platform and said the results, including limitations, would be published at stratix.layerlens.ai.^[8]

SubQ 1.1 Small

Subquadratic released the model card and technical report for SubQ 1.1 Small on June 16, 2026, describing it as the second iteration of its SSA model, at the smallest size.^[9] Whedon later said the published model has fewer than 100 billion parameters.^[13]

Rather than training a model from scratch, Subquadratic converted "an existing open-weight frontier model" by replacing its dense attention with SSA.^[10] The donor model supported a 262,000-token context. The team extended it in stages (262K, 512K, 1M and 2M tokens) using YaRN positional rescaling, with long-context continued pretraining between stages, most of it at 1 million tokens.^[10] The company says this involved roughly one trillion tokens of continued pretraining on books, documents and repository-scale code, and more than one hundred long-context experiments across six to seven model generations.^[9][10] The report does not name the donor model.^[10][13] MIT Technology Review reported that Subquadratic reused the weights of a version of the Chinese open-source model Qwen.^[12]

On long-context retrieval, the report gives 99.12% on the 13-task RULER suite at 128K tokens. On single-needle needle-in-a-haystack tests with 50 UUID samples per length, it gives 100% at 1 million and 2 million tokens and 98% at 6 million and 12 million tokens, even though training stopped at 2 million; at 12 million tokens the model attended to 0.13% of token pairs.^[9][10] Appen reported the same retrieval figures, though its summary attributes the 6M and 12M results to a "nano variant" of the model, and it reported 89.7% pass@4 on LiveCodeBench (1,055 problems, four completions each) for a model it identified as "subq-2m-preview-small".^[11]

General-capability results published by Subquadratic, which said its benchmark results had been independently verified by Appen:^[9][10]

The report measures attention FLOPs for one attention layer against the same backbone with dense attention:^[10]

On a single attention layer, the report measures SSA at 56 times faster than FlashAttention-2 at 1 million tokens (966 milliseconds against 54,164 milliseconds), with parity near 16,000 tokens.^[10] The report also lists its own limits. Above roughly 8 million tokens, BF16 underflow and related numerical-stability issues became practical constraints. Its finding that long-context continued pretraining was the strongest lever was not tested with controlled ablations across different backbones. And MRCR v2, a benchmark the company had highlighted at launch, diverged during development from the behaviors the team wanted to improve, so RULER became its main development signal.^[10]

Subquadratic said it was deploying SubQ 1.1 Small with selected design partners and planned a broader lineup of models from 2 million to 12 million tokens later in the year.^[9] The company told MIT Technology Review that tens of thousands of potential users, including more than 500 enterprise customers, had signed up for early access, and that as a small company it could not yet serve them all.^[12] In July, Whedon told The New Stack that the design partners were "mostly enterprises, largely with eight- to nine-figure spend". The New Stack added that a limited individual-access release would come before general availability. Whedon also said that the next model would likely be mid-tier in size rather than frontier-class.^[13] As of October 2026 the SubQ Small product page listed "SubQ 1.2 Medium" as coming soon.^[20]

SubQ Snap

On October 12, 2026, Subquadratic released SubQ Snap, a decision model built on SSA.^[14] A decision model returns a probability for each answer in a fixed, application-defined set (for example, which tool an agent should call next or whether a test result passes) instead of generating free text. The category grew quickly after TypeSafe AI released Jev in September 2026.^[14][15] Whedon told The New Stack that "Snap was developed with inspiration from Jev, because decision models are highly thesis-aligned."^[15] Snap pairs SSA, to reduce the cost of reading long inputs, with a decision head that maps the model's representations directly to a distribution over the allowed answers.^[14]

Snap accepts up to 384,000 tokens of text and images in one request and, according to the company, can process 100,000 tokens and return a decision in about one second.^[14] Whedon said the 384K limit was a temporary restriction for the public preview because of available compute; the company tested the model at 1 million tokens.^[15] It launched at $0.05 per million input tokens with no charge for output tokens.^[14]

SubQ Snap 1.1 followed on October 22, 2026. Subquadratic said its post-training calibrated the model's probabilities, trained it to abstain when it does not know an answer, and distilled multi-step reasoning into a single forward pass; it cut the price to $0.04 per million input tokens.^[16] The company reported that expected calibration error on held-out problem families fell from 0.108 to about 0.014 and that accuracy on the When2Call tool-use benchmark rose from 75.2 to 96.2. It also said Snap 1.1 still trailed Jev 1.14 by 22 points on the PhishNChips phishing benchmark, a gap it attributed mainly to its default classification threshold.^[16]

Except for the Artificial Analysis score, these results come from Subquadratic's own testing. Whedon confirmed that the Snap launch results were produced internally, although the company released the code for its adapted long-context evaluations.^[15] Snap was not on the Decision Index leaderboard kept by that index's authors; Whedon said they told him its current version focused on open-weight models.^[15] On the separate Decision Index run by Artificial Analysis, SubQ Snap 1.1 scored 65 as of October 27, 2026, level after rounding with Liquid AI's d1-ultra and Baseten's Elephant 9B at the top of the table, which Subquadratic describes as a tie for first place.^[16][17]

Funding

Subquadratic announced a $29 million seed round when it emerged from stealth on May 5, 2026.^[1][2] The company did not disclose a valuation. The New Stack reported that it had raised the money at a $500 million valuation,^[5] and VentureBeat repeated that figure with attribution to The New Stack.^[3] Named investors include Javier Villamizar, a former partner at the SoftBank Vision Fund; Justin Mateen, a co-founder of Tinder and founder of the JAM Fund; Grant Gittlin of Lasagna; and Jaclyn Rice Nelson of Coalition Operators.^[1][2][4] The company also lists unnamed early investors in Anthropic, OpenAI, Stripe, and Brex among its backers.^[2][3] VentureBeat called the reported valuation "a steep price for a seed-stage company with no publicly available model, no peer-reviewed research, and no disclosed revenue", and noted that the investor base skewed toward consumer tech and growth investing.^[3] The New Stack's October 2026 coverage still described the company's funding as the $29 million seed.^[15]

Reception and skepticism

VentureBeat set the launch against the history of efficient-attention research. In its account, Kimi Linear, DeepSeek Sparse Attention, Mamba and RWKV all promised subquadratic scaling and ran into the same problem: architectures that are cheap in the asymptotic sense often lose accuracy against dense attention at frontier scale, or they survive only by going hybrid, mixing subquadratic layers with ordinary attention layers and giving up the clean scaling story in the process.^[3]

At launch there was no technical report and there were no open weights, so no independent lab could rerun the benchmarks or inspect how token selection works.^[3] VentureBeat also pointed to the narrow benchmark selection: three tests, all emphasizing the long-context retrieval and coding tasks SubQ was designed for.^[3] Will Depue, an AI researcher who previously worked at OpenAI,^[12] wrote that SubQ was "almost surely a sparse attention finetune of Kimi or DeepSeek"; Whedon replied on X that the company was "using weights from open-source models as a starting point, as a function of our funding and maturity as a company."^[3] Depue later wrote that the O(n) scaling claims and the speedup numbers "don't seem to line up".^[3] Others defended the work; AI researcher John Rysana wrote that it was "just subquadratic attention done well which is very meaningful for long context workloads."^[3]

Critics also raised a cautionary parallel. In August 2024, Magic announced a model with a 100-million-token context window and a claimed 1,000x efficiency advantage; as of early 2026 there was no public evidence of that model, LTM-2-mini, being used outside Magic.^[3][5] The AI commentator Dan McAteer captured the split reaction in a widely shared post: "SubQ is either the biggest breakthrough since the Transformer... or it's AI Theranos."^[3]

The June technical report and the Appen evaluations answered some of these points. MIT Technology Review wrote that the independent results "suggest that the company's claims might be worth paying attention to", while adding that skepticism remained justified until more people could use the model and that reusing Qwen weights "cuts across Subquadratic's claim that it has fully reinvented how LLMs work."^[12] Depue told the magazine: "They may have built something real and useful. But the public evidence does not yet justify the stronger claim that they have solved the quadratic attention bottleneck."^[12] The Appen evaluations were commissioned by Subquadratic,^[11][12] the technical report withholds the selection mechanism,^[10] and as of October 2026 the SubQ language model remained in early access with selected design partners, available only through Subquadratic's own API.^[14][20]

See also

• List of neolabs
• Neolab
• Sparse attention
• DeepSeek Sparse Attention
• Long-context language models
• Magic (AI software engineer)
• Jev (AI model)

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14Dotson, Kyt. "Subquadratic launches with $29M to bring 12M-token context windows to AI." SiliconANGLE, May 5, 2026.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22Dangel, Justin. "Introducing SubQ: The First Fully Subquadratic LLM." Subquadratic, May 5, 2026.
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18Nuñez, Michael. "Miami startup Subquadratic claims 1,000x AI efficiency gain with SubQ model; researchers demand independent proof." VentureBeat, May 5, 2026.
4. ^1 ^2 ^3 ^4Chowdhry, Amit. "Subquadratic: $29 Million Seed Raised For Long-Context AI Architecture." Pulse 2.0, May 6, 2026.
5. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14Lardinois, Frederic. "The context window has been shattered: Subquadratic debuts a 12-million-token window." The New Stack, May 5, 2026.
6. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9"How SSA Makes Long Context Practical." Subquadratic, May 5, 2026, updated May 15, 2026.
7. ^1 ^2 ^3 ^4 ^5Bruccoleri, Sergio; Sinanan-Singh, Jeanine. "Benchmarking Subquadratic's latest model & SSA Kernel." Appen, May 11, 2026.
8. ^1 ^2"We're Partnering with LayerLens to Evaluate SubQ." Subquadratic, May 14, 2026.
9. ^1 ^2 ^3 ^4 ^5 ^6 ^7"Introducing SubQ 1.1 Small." Subquadratic, June 16, 2026.
10. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18Ramirez, Saul; Whedon, Alex; Vayani, Ashmal; Vo, Phong. "SubQ-1.1-Small Technical Report." Subquadratic, June 2026.
11. ^1 ^2 ^3 ^4 ^5 ^6Sinanan-Singh, Jeanine; Rabbani, Tahseen. "Putting SubQ 1.1 Small Preview to the Test on Long-Context Retrieval & LiveCodeBench." Appen, June 16, 2026.
12. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Heaven, Will Douglas. "A startup claims it broke through a bottleneck that's holding back LLMs." MIT Technology Review, June 19, 2026.
13. ^1 ^2 ^3 ^4 ^5 ^6Lardinois, Frederic. "What comes after attention? This startup says it already knows." The New Stack, July 2, 2026.
14. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14"Introducing SubQ Snap: A Long-Context Decision Model for Enterprise Agents." Subquadratic, October 12, 2026.
15. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Sawers, Paul. "'Take work away from frontier LLMs': Subquadratic's decision model claims 12x Jev's context window." The New Stack, October 14, 2026.
16. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10"Introducing SubQ Snap 1.1: The Smartest, Enterprise-Ready Decision Model." Subquadratic, October 22, 2026.
17. ^1 ^2"Artificial Analysis Decision Index." Artificial Analysis, accessed October 27, 2026.
18. ^1 ^2"Alexander Whedon (@alex_whedon)." X profile, accessed October 27, 2026.
19. ^"Subquadratic." LinkedIn company page, accessed October 27, 2026.
20. ^1 ^2 ^3"SubQ 1.1 Small." Subquadratic product page, accessed October 27, 2026.
21. ^1 ^2"Aldea Labs - Smarter AI. Faster voice intelligence." Aldea, Inc. website, archived December 8, 2025.