Stable Diffusion 3
Stable Diffusion 3 (SD3) is a family of text-to-image diffusion models developed by Stability AI, first announced as an early preview on February 22, 2024, and built on a new architecture called the Multimodal Diffusion Transformer (MMDiT) trained with a rectified flow objective. The series spans models "ranging from 800m to 8B parameter models," abandoning the U-Net backbone that had defined every previous Stable Diffusion release. [1] [2] Stability AI positioned SD3 as its answer to closed competitors such as DALL-E 3 and Midjourney v6, with substantially improved prompt following, multi-subject handling, and in-image text rendering compared to Stable Diffusion XL (SDXL). The company's own research summary claimed that "Stable Diffusion 3 outperforms state-of-the-art text-to-image generation systems such as DALL-E 3, Midjourney v6, and Ideogram v1 in typography and prompt adherence, based on human preference evaluations." [1] [2]
The public release did not go to plan. SD3 Medium (2 billion parameters) shipped on June 12, 2024, under a non-commercial research license, with commercial use requiring a paid Creator License whose terms many in the open-source community considered hostile. [23] Within days, the model became the subject of memes about distorted human anatomy, and CivitAI, the largest hub for Stable Diffusion fine-tunes, temporarily banned uploads of SD3-derived models entirely. Four months later, after a near-complete management turnover at Stability AI and the departure of most of the original Stable Diffusion research team to found Black Forest Labs, the company released the SD 3.5 family on October 22, 2024, with a revised license, retrained weights, and three model sizes topping out at the 8.1 billion-parameter SD 3.5 Large. [3] [4] [5]
This article covers the SD3 and SD 3.5 series together, since the two are technically continuous and the SD 3.5 release effectively superseded the original SD3 weights. It also covers the corporate context that shaped the launch, because SD3 is one of the rare cases where a model's reception was driven more by licensing and company drama than by image quality.
Infobox
| Developer | Stability AI |
| Initial preview | February 22, 2024 |
| Open weights release | June 12, 2024 (SD3 Medium) |
| Latest variant | SD 3.5 Large (October 22, 2024) |
| Architecture | Multimodal Diffusion Transformer (MMDiT) |
| Sampler objective | Rectified flow |
| Parameters | 800M to 8.1B (Large) |
| Text encoders | CLIP-L, OpenCLIP-bigG, T5-XXL |
| VAE channels | 16 |
| License | Stability AI Community License |
| Predecessor | SDXL |
| Successor in lineage | Flux (by ex-Stability researchers) |
| Paper | arxiv.org/abs/2403.03206 |
| Hugging Face | huggingface.co/stabilityai/stable-diffusion-3-medium |
Background
Stability AI in late 2023 and early 2024
The period leading up to SD3 was, by the company's own later admission, chaotic. By late 2023 Stability AI was reportedly spending roughly $8 million per month, against monthly revenue of about $3 million in November 2023 and $5.4 million in February 2024. [20] An attempted fundraise at a $4 billion valuation fell apart. In a letter to the board, Lightspeed Venture Partners said founder Emad Mostaque's management had "severely undermined" their confidence in him; Coatue Management pushed for his resignation and opened an internal review. [6]
Emad Mostaque resigned as CEO and from the board on March 23, 2024, exactly one month after SD3 was first shown to the public. He framed the move publicly as a desire to pursue "decentralized AI," though contemporary reporting described it as effectively forced. Days earlier, Forbes had reported that three of the five authors of the original latent diffusion paper, Robin Rombach, Andreas Blattmann, and Dominik Lorenz, had left Stability AI. [28] They went on to co-found Black Forest Labs and ship the Flux image models, which in many respects became the spiritual successor to SD3 within the same year. [7] [8]
The SD3 paper credits a long author list, with Patrick Esser as first author and Rombach as senior author. By the time the open weights came out in June 2024, several names on that paper were no longer Stability employees.
The SDXL ceiling
The immediate technical motivation for SD3 was that SDXL, released July 2023, had hit a ceiling on prompt following. SDXL's 2.6 billion-parameter U-Net [26] produced sharp, well-composed images at 1024x1024 native resolution, but it routinely failed at multi-subject prompts ("a red cube on top of a blue sphere"), legible in-image text, and complex spatial relationships. Closed competitors had moved further: DALL-E 3, released by OpenAI in October 2023 and integrated into ChatGPT, used a captioning rewrite step that gave it noticeably better prompt adherence; Midjourney v6, released in December 2023, also rendered text reasonably well. [9]
Stability AI needed both a quality jump and an architectural story for the next round of fundraising. SD3 was supposed to provide both.
How does the SD3 architecture work?
SD3 is described in the paper "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" (Esser et al., arxiv:2403.03206). The architecture differs from SDXL in three substantive ways: it uses a transformer rather than a U-Net, trains on a rectified flow objective rather than the discrete-time diffusion noise schedule used by SDXL, and conditions on three text encoders rather than two. The paper reports a clean scaling trend, training models "from 15 blocks with 450M parameters to 38 blocks with 8B parameters" and observing a smooth decrease in validation loss as both model size and training steps grow, with "no signs of saturation." [1] [2]
Multimodal Diffusion Transformer (MMDiT)
The MMDiT block is the centerpiece. Earlier diffusion transformers (DiT, PixArt) treated image and text tokens as a single concatenated sequence, fed through ordinary transformer blocks. MMDiT instead, in the authors' words, uses "two separate sets of weights for the two modalities," which "is equivalent to having two independent transformers for each modality, but joining the sequences of the two modalities for the attention operation." The two streams keep their own weight matrices for query, key, value, and feed-forward projections, and meet only inside the attention operation, where their projected keys and values are concatenated, attention is computed jointly, and the resulting context is split back to the two streams. [1]
In practice this means a SD3 transformer block is roughly twice the size of a corresponding DiT block at the same depth, but the two modalities can develop their own representations rather than fighting over a shared embedding space. Stability reports that MMDiT outperforms established backbones such as UViT and DiT when measuring visual fidelity and text alignment over the course of training. [1] [2]
What is rectified flow?
Traditional diffusion models train on a noise schedule, learning to denoise samples drawn at randomly chosen timesteps along a curved trajectory between data and Gaussian noise. Rectified flow, introduced in 2022 by Liu et al., reformulates this as learning a constant velocity field along a straight line in the data-noise space. SD3 uses a rectified flow loss with a logit-normal weighting on the timestep, which the paper finds outperforms standard diffusion across model sizes. [1]
In inference terms this matters because rectified flow models can be sampled with relatively few steps (often 25 to 30 for high quality) and respond well to distillation, which is why Stability could ship SD3 Large Turbo and SD 3.5 Large Turbo as 4-step distilled variants without major quality collapse.
Sixteen-channel VAE
SD 1.x and 2.x used a 4-channel VAE latent. SDXL kept the 4-channel encoder. SD3 moves to a 16-channel VAE with a higher reconstruction fidelity, which the paper credits for sharper detail and substantially better text rendering inside images. The 16-channel VAE is also part of why SD3 weights are not interchangeable with prior Stable Diffusion checkpoints: the latent space has different dimensionality.
Three text encoders
SDXL used CLIP ViT-L/14 plus OpenCLIP ViT-bigG/14. SD3 keeps both of those and adds Google's T5 XXL encoder (about 4.7 billion parameters), giving the model a text encoder stack that is substantially larger than its image transformer at the Medium scale.
The T5 encoder is optional at inference. Users on tight VRAM budgets can drop T5 and run with just the two CLIP encoders. According to the paper, dropping T5 has no effect on aesthetic ratings (50% win rate) and only a small effect on prompt adherence (46% win rate), but a larger effect on typography (38% win rate); the losses concentrate in very complex prompts and prompts with a lot of written text. [1] [2]
How does SD3 differ from SDXL and SD 2?
| Feature | SD 2.1 | SDXL 1.0 | SD3 Medium | SD 3.5 Large |
|---|---|---|---|---|
| Backbone | U-Net | U-Net (larger) | MMDiT | MMDiT (with QK-norm) |
| Diffusion parameters | ~865M | ~2.6B (Base U-Net) + Refiner | 2B | 8.1B |
| Training objective | DDPM (v-prediction) | DDPM (discrete-time) | Rectified flow | Rectified flow |
| VAE channels | 4 | 4 | 16 | 16 |
| Native resolution | 768x768 | 1024x1024 | 1024x1024 | 1024x1024 |
| Text encoder(s) | OpenCLIP ViT-H/14 | CLIP ViT-L/14 + OpenCLIP ViT-bigG/14 | CLIP ViT-L/14 + OpenCLIP ViT-bigG/14 + T5-XXL | CLIP ViT-L/14 + OpenCLIP ViT-bigG/14 + T5-XXL |
| Inference steps (typical) | 30-50 | 25-40 | 28-50 | 28-50 |
| Two-stage pipeline | No | Yes (optional Refiner) | No | No |
MMDiT-X in SD 3.5
The SD 3.5 family does not use exactly the same blocks as SD3. SD 3.5 Medium uses an architecture Stability calls MMDiT-X, which adds self-attention modules in the first 13 transformer layers that the company says improve multi-resolution generation. [27] All three SD 3.5 models add Query-Key Normalization (QK-norm), applied before the attention scores are computed, which Stability says works by "stabilizing the model training process and simplifying further fine-tuning and development." [4]
Variants
Variant lineup
| Variant | Parameters (diffusion model) | Release date | Open weights | Notes |
|---|---|---|---|---|
| SD3 (early preview) | Multiple sizes 800M-8B (paper) | February 22, 2024 | No (API only) | Closed alpha; quality previews shown in paper |
| SD3 Medium | 2B | June 12, 2024 | Yes | First open weights release; restrictive license; widely criticized for anatomy issues |
| SD3 Large | 8B | API only | No | Available via Stability API and Fireworks AI; never open-sourced under SD3 branding |
| SD3 Large Turbo | 8B | API only | No | 4-step distilled SD3 Large; API only |
| SD 3.5 Large | 8.1B | October 22, 2024 | Yes | Flagship open release; MMDiT with QK normalization |
| SD 3.5 Large Turbo | 8.1B | October 22, 2024 | Yes | 4-step distilled SD 3.5 Large |
| SD 3.5 Medium | 2.5B | October 29, 2024 | Yes | MMDiT-X; slightly larger than SD3 Medium; targets consumer GPUs |
A few details are worth pinning down because they often get confused. The SD3 Large and SD3 Large Turbo weights were, as far as the public is aware, never released under permissive open weights. Stability shipped them through the Stability API and through partner inference providers like Fireworks AI, but the open-weights story for the 8B class only began with SD 3.5 Large in October 2024. The SD3 Medium open release in June 2024 was the only SD3-branded checkpoint that ever ended up on Hugging Face for direct download. [3] [10]
SD 3.5 Medium (2.5B) is slightly larger than SD3 Medium (2B). The choice of 2.5B was driven by the desire to fit comfortably in 12 GB of VRAM with the T5 encoder offloaded to system memory.
What is Stable Diffusion 3 used for, and how does it improve on SDXL?
The paper claims state-of-the-art results across three axes: prompt following, typography, and visual aesthetics, evaluated against SDXL, DALL-E 3, Midjourney v6, Ideogram v1.0, and Pixart-alpha. Stability AI summarized the human evaluations by stating that "Stable Diffusion 3 is equal to or outperforms current state-of-the-art text-to-image generation systems" across these areas. [2]
The specific gains over SDXL fall into a few buckets:
- In-image text. SDXL could occasionally produce a readable single word, usually short and centered. SD3 can render longer phrases, including multi-line text, with markedly fewer letter-soup failures. The 16-channel VAE and the T5 encoder both contribute to this.
- Multi-subject prompts. "A cat on a chair next to a dog on a sofa" is the canonical hard case. SDXL frequently merges subjects or assigns the wrong attribute. SD3 holds attribute binding more reliably, particularly when the T5 encoder is used.
- Spatial relationships. "Above," "below," "to the left of," and similar prepositions are slightly more honored, though still far from reliable.
- Style consistency. Photorealistic renders, illustration styles, and 3D render styles are more cleanly separated than in SDXL, which sometimes drifted between styles within a single image.
Where SD3 did not deliver is human anatomy. The June 2024 SD3 Medium release became infamous for hands, elongated limbs, and the now-meme-worthy "woman lying on grass" prompts where the model produced bodies in physically impossible configurations. The cause has been variously attributed to aggressive NSFW filtering of training data (which removed enough human bodies that the model under-trained on them), distillation effects, and bugs in the released checkpoint. SD 3.5 substantially closes the gap, though Flux and Midjourney v7 remain stronger on photorealistic human figures as of early 2026. [3] [11]
When was Stable Diffusion 3 released?
Release timeline
| Date | Event |
|---|---|
| February 22, 2024 | SD3 announced via Stability AI blog post; closed early preview; waitlist opened |
| March 5, 2024 | Paper "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" posted on arXiv (2403.03206) |
| March 23, 2024 | Emad Mostaque resigns as CEO of Stability AI |
| March 2024 | Robin Rombach, Andreas Blattmann, Dominik Lorenz leave Stability AI |
| April 17, 2024 | SD3 API access opens through Stability Developer Platform and Fireworks AI |
| June 12, 2024 | SD3 Medium open weights released on Hugging Face under a non-commercial research license, with commercial use via a paid Creator License |
| June 17, 2024 | CivitAI temporarily bans SD3-based models and models trained on SD3 outputs amid license uncertainty |
| June 25, 2024 | Prem Akkaraju named CEO; Sean Parker named Executive Chairman |
| July 5, 2024 | Stability AI introduces the new Stability AI Community License, free for commercial use under $1M annual revenue and covering SD3 Medium |
| July 22, 2024 | CivitAI lifts its SD3 ban |
| August 1, 2024 | Black Forest Labs (Rombach, Blattmann, Esser) launches FLUX.1 |
| October 22, 2024 | SD 3.5 Large and SD 3.5 Large Turbo released as open weights with revised Community License |
| October 29, 2024 | SD 3.5 Medium released as open weights |
| April 2025 | SD3 (3.0) APIs deprecated; users automatically migrated to SD 3.5 |
A few of these dates are worth elaborating. The original February 22 announcement was a preview only, with access gated behind a waitlist. The actual open-weights release came almost four months later, by which point the founder-CEO had resigned, interim co-CEOs were running the company, several authors had departed, and the open-source community was louder than the company had bargained for. [3] [25]
License history and backlash
The license drama around SD3 is the part of the story most people remember, and it is the most tangled to summarize accurately because the terms shifted in real time during June and July 2024.
Initial June 2024 terms
When SD3 Medium open weights went live on Hugging Face on June 12, 2024, they were released under the Stability Non-Commercial Research Community License, with commercial use offered through a new paid Creator License. [23] These terms, in their initial form, contained provisions that the open-source image generation community read as substantially more restrictive than the OpenRAIL-M license that had governed Stable Diffusion 1.x and 2.x and the SDXL license. The most contested provisions, as written in the original license text and reported across community sources at the time, were:
- Commercial use only through the paid Creator License, which was limited to developers with less than $1 million in annual revenue, less than $1 million in institutional funding and fewer than 1 million monthly active users, and which capped generation at 6,000 images per month. Anyone above those limits needed an Enterprise License. [19]
- Restrictions on training derivative models on outputs of SD3, with language broad enough to be read as banning any LoRA, fine-tune, or distilled model trained on SD3 outputs.
- A requirement to delete model weights and any derivatives if the license was terminated.
- Tracking and reporting requirements for commercial use cases.
The cumulative effect, as the community read it, was that anyone building a paid product on SD3, even a small Patreon-funded image generation site, faced uncertainty about whether they would owe Stability fees, whether their fine-tuned LoRAs were legal, and whether the license could be revoked. [12] [13]
Why did CivitAI ban Stable Diffusion 3?
The largest community hub for Stable Diffusion custom models, CivitAI, responded by banning SD3-based content from its platform on June 17, 2024, five days after the open weights release. The ban covered uploads of SD3 Medium itself, fine-tuned SD3 checkpoints, and LoRAs targeted at SD3. CivitAI's stated reason was that the license granted Stability AI too much power over not only models fine-tuned on SD3 but any other model whose training set included SD3-generated images, exposing both the platform and uploaders to retroactive license claims. The decision was widely covered in tech media and on developer forums; for several days, the consensus among major Stable Diffusion communities was effectively to pretend SD3 did not exist. [13]
AUTOMATIC1111, ComfyUI, and other major tool maintainers were more cautious in their public statements but slow to integrate SD3 fully. ComfyUI added SD3 support quickly, but several extension authors paused work pending license clarification.
Stability's clarifications and revisions
Stability AI responded publicly on July 5, 2024, acknowledging that the commercial license originally attached to SD3 had "caused some confusion and concern in the community." [22] The company stated that:
- Personal and research use was unambiguously free.
- The $1 million revenue threshold applied to total annual revenue, not revenue from SD3 specifically, but smaller commercial users could still use the model freely.
- Fine-tunes and other derivative products were free to create and sell for users below the $1 million threshold, with commercial users asked to fill out a short self-reporting form.
- There would be no limit on the number of images generated, and Stability would never ask qualifying users to delete images, fine-tunes or other derived products.
The July 5, 2024 change replaced the earlier licenses with a new Stability AI Community License that addressed most of the issues CivitAI and others had reported. [22] Whether these clarifications matched the original license text or constituted retroactive interpretation is, frankly, a matter of opinion in the community. The text itself was edited multiple times across June and July 2024, which made any single "what does the license say" answer a moving target. CivitAI lifted its ban on July 22, 2024, saying Stability had addressed its largest concerns, including no longer treating models trained on SD3 outputs as derivative works, though it did not offer on-site SD3 generation. [13] [21]
SD 3.5 and the Community License
Stability AI released SD 3.5 on October 22, 2024 under the same Stability AI Community License it had introduced in July, which also governs SD3 Medium. Those terms brought the license closer to community expectations without making it fully open source by OSI definitions. Stability summarized the terms as free for non-commercial use and "Free for commercial use (up to $1M in annual revenue)," with users able to "retain ownership of the media generated without restrictive licensing implications." [4] The specific terms were:
- Personal and research use: free for individuals and organizations of any size.
- Commercial use: free for organizations with annual revenues up to $1 million.
- Enterprise license required: only for organizations with annual revenues above $1 million using the model in commercial products.
- LoRA and fine-tuning: explicitly permitted under the same license.
- Derivative model distribution: permitted under the Community License.
- No requirement to share outputs back to Stability.
This is the form the license has held through 2025 and into 2026. The model weights remain free for the vast majority of users; the constraint applies only to a relatively small number of medium- and large-revenue commercial deployments. Most observers, even those who criticized the original launch, regarded the revised terms as reasonable, though the revenue gating keeps the license outside the Open Source Definition. [4] [12]
License history table
| Date | License version | Key terms |
|---|---|---|
| June 12, 2024 | Stability Non-Commercial Research Community License, plus paid Creator License | Free non-commercial use; Creator License limited to under $1M revenue, $1M funding and 1M monthly users, with a 6,000 images/month cap; models trained on SD3 outputs treated as derivative works |
| June 17, 2024 | (no change) | CivitAI temporarily bans SD3 models; Stability makes no public clarification until July |
| July 5, 2024 | Stability AI Community License (new) | Free commercial use under $1M annual revenue; no image caps; covers SD3 Medium |
| October 22, 2024 | Stability AI Community License (applied to SD 3.5) | Free under $1M annual revenue; enterprise license required above; outputs owned by the user |
SD 3.5 family
SD 3.5, released October 22, 2024, was the company's chance to reset the SD3 story. By that point Stability had a new CEO (Prem Akkaraju), a recapitalized balance sheet, an executive chairman from Napster (Sean Parker), and the looming presence of Flux, released by ex-Stability researchers two months earlier. The SD 3.5 release had to demonstrate that the company could ship competitive open-weights image models without the original Stable Diffusion team. Stability framed the goal as an effort to "empower builders and creators with tools that are widely accessible, cutting-edge, and free for most use cases." [4] [5]
What changed from SD3 Medium
Several things changed simultaneously, which makes attribution of the quality improvements somewhat murky:
- QK normalization was added to all SD 3.5 models, and SD 3.5 Medium moved to MMDiT-X, which adds self-attention in the first 13 layers.
- Larger flagship. The SD 3.5 Large 8.1B model is the first 8B-class SD3-lineage checkpoint to ship as open weights.
- More development. Stability said that, after SD3 Medium "didn't fully meet our standards or our communities' expectations," it chose "instead of a quick fix" to further develop the models. [4]
- Less aggressive NSFW filtering. Without saying so explicitly, Stability appears to have rebalanced the training data to include more human bodies, which addressed the most prominent SD3 Medium failures.
- New license from day one. SD 3.5 shipped under the Stability AI Community License introduced in July 2024, rather than the non-commercial license SD3 Medium launched with.
The SD 3.5 release page emphasized customizability and ease of fine-tuning as primary goals, inviting users to "easily fine-tune the model to meet your specific creative needs," an explicit acknowledgment that the original SD3 license had alienated the fine-tuning community. [4]
SD 3.5 Large Turbo
SD 3.5 Large Turbo is a 4-step distilled version of SD 3.5 Large that generates, in Stability's words, "high-quality images with exceptional prompt adherence in just 4 steps," trained using Adversarial Diffusion Distillation (ADD), the same family of techniques Stability used for SDXL Turbo. [4] At 4 inference steps, SD 3.5 Large Turbo gives roughly 7-10x faster generation than SD 3.5 Large at the cost of some prompt fidelity, especially on long T5-conditioned prompts. The Turbo variant has the same architecture and parameter count as SD 3.5 Large; it differs only in training, with a student model distilled from a multi-step teacher.
SD 3.5 Medium
SD 3.5 Medium (2.5B), released a week after SD 3.5 Large on October 29, 2024, was positioned as the consumer-hardware option, designed by Stability to run "out of the box" on consumer GPUs. [5] It does not match SD 3.5 Large in absolute quality, but it fits comfortably in 12 GB of VRAM with the T5 encoder offloaded to system memory, and at 16 GB VRAM it can run with full T5 conditioning. QK normalization is present in both Medium and Large, but the MMDiT-X architecture is specific to Medium. [27]
Reception
The SD3 Medium reception was, in a word, ugly. "Sloppy anatomy" memes proliferated on Reddit's Stable Diffusion subreddit during the week of June 12-19, 2024, with the most-shared examples being prompts like "a woman lying on grass" or "a person holding hands with another person" producing bodies that looked, charitably, like they had been assembled by someone who had never seen a human. The license backlash and the anatomy memes reinforced each other: the community read the situation as a company that had alienated its developers and shipped a buggy model on the same day. [3] [11]
Stability AI publicly acknowledged the problems on July 5, 2024, about three weeks after release, saying SD3 Medium "didn't meet our community's high expectations" and that the community had identified "critical quality issues mainly related to body poses and words that were too rarely seen in the training set." The company called SD3 Medium "still a work in progress" and promised "a much improved version in the coming weeks." [22] The Decoder reported the statement on July 7 under the headline that Stability AI had apologized for the disappointing release. [11]
SD 3.5's reception was warmer but quieter. By October 2024, much of the energy that would have gone into evaluating an open-weights image model from Stability had moved to Flux, released August 2024 by ex-Stability researchers under Black Forest Labs. FLUX.1 schnell shipped under Apache 2.0 (genuinely open source by OSI standards), FLUX.1 dev had a clearer non-commercial license, and FLUX.1 pro was available through API. SD 3.5 Large was widely considered to roughly match FLUX.1 dev in image quality on many tasks, with some advantages on artistic styles and disadvantages on photorealistic humans. The community largely accepted SD 3.5 as a viable alternative without crowning it. [4] [14]
Professional reviews of SD 3.5 Large in the months following release tended to highlight its prompt-following improvements and its better-behaved fine-tuning. The flip side, repeatedly noted, was that running SD 3.5 Large with the T5 encoder requires roughly 24 GB of VRAM, which puts it in workstation-card territory; SDXL fine-tunes were comfortable on 12 GB cards.
How does SD 3.5 compare with FLUX, DALL-E 3, and Midjourney?
Peer model comparison
| Model | Developer | Released | Architecture | Open weights | Known strengths |
|---|---|---|---|---|---|
| SD 3.5 Large | Stability AI | October 2024 | MMDiT-X (8.1B) | Yes (Community License) | Style range, prompt following, fine-tuning ecosystem |
| FLUX.1 dev | Black Forest Labs | August 2024 | Hybrid flow transformer (12B) | Yes (non-commercial) | Photorealism, in-image text, anatomy |
| FLUX.1 schnell | Black Forest Labs | August 2024 | Distilled flow transformer (12B) | Yes (Apache 2.0) | Genuinely open source; fast inference |
| FLUX.1 pro | Black Forest Labs | August 2024 | Flow transformer (12B+) | No (API) | Highest BFL quality tier |
| DALL-E 3 | OpenAI | October 2023 | Diffusion + caption rewrite | No (API) | Prompt rewriting, ChatGPT integration |
| Midjourney v6 | Midjourney Inc. | December 2023 | Closed | No (Discord/web) | Aesthetics, stylistic coherence |
| Midjourney v7 | Midjourney Inc. | April 2025 | Closed | No (Discord/web) | Refinement of v6 strengths |
| Imagen 3 | August 2024 | Closed | No (Vertex AI) | Photorealism, text rendering | |
| Imagen 4 | 2025 | Closed | No (Vertex AI / consumer) | Sharper detail, better text | |
| Ideogram v2 | Ideogram | 2024 | Closed | No (web) | In-image text rendering |
A blunt summary in early 2026: SD 3.5 is a credible open-weights model in the second tier of image quality, behind closed leaders (Midjourney v7, Imagen 4, DALL-E 3 in some categories) and behind FLUX in several open categories. Its main argument over FLUX is its larger fine-tuning ecosystem inherited from earlier Stable Diffusion versions, plus the genuine open-weights status of all SD 3.5 variants under the revised Community License. [14]
What hardware do you need to run Stable Diffusion 3?
Hardware requirements by variant
| Variant | Minimum VRAM (with T5 offloaded) | Recommended VRAM (with T5 in GPU) | System RAM | Disk |
|---|---|---|---|---|
| SD3 Medium (2B) | ~10 GB | ~12-16 GB | 16 GB+ | ~10 GB |
| SD 3.5 Medium (2.5B) | ~10 GB | ~12-16 GB | 16 GB+ | ~10 GB |
| SD 3.5 Large (8.1B) | ~16 GB | ~24 GB | 32 GB+ | ~16 GB |
| SD 3.5 Large Turbo (8.1B, 4-step) | ~16 GB | ~24 GB | 32 GB+ | ~16 GB |
The T5 XXL encoder is the largest single consumer of memory at inference. T5 XXL is roughly 4.7 billion parameters in fp16, or about 9.4 GB if loaded into VRAM directly. ComfyUI and similar tools support offloading the encoder to CPU, which lets users run T5-conditioned generation at the cost of slower text encoding. Quantized variants of the T5 encoder (8-bit and 4-bit) are widely used in community workflows to bring the memory footprint down further.
SD 3.5 Large can be run without T5 by passing only CLIP-L and OpenCLIP-bigG embeddings, but the typical recommendation is that anyone interested in SD 3.5 Large's prompt-following advantages over SDXL should keep T5 in the pipeline. The 8.1B image transformer in fp16 is itself about 16 GB.
Stability AI corporate context
The SD3 launch is one of the cleanest examples in recent AI history of a release whose reception was shaped almost as much by company drama as by model quality. The personnel and ownership changes at Stability AI during 2024 deserve a paragraph in any treatment of SD3.
Corporate timeline 2024
| Date | Event |
|---|---|
| October-November 2023 | Lightspeed Venture Partners and Coatue Management criticize Mostaque's management, and Coatue pushes for his resignation; Stability reportedly spending about $8M/month |
| February 22, 2024 | SD3 announced; closed alpha; paper not yet posted |
| March 5, 2024 | SD3 paper posted on arXiv |
| March 23, 2024 | Emad Mostaque resigns as CEO and from board |
| March 2024 | Robin Rombach, Andreas Blattmann, Dominik Lorenz leave Stability |
| March-June 2024 | Shan Shan Wong (former COO) and Christian Laforte (CTO) serve as interim co-CEOs |
| April 17, 2024 | SD3 API access opens via Stability and Fireworks AI |
| June 12, 2024 | SD3 Medium open weights released; license backlash |
| June 25, 2024 | Prem Akkaraju named CEO; Sean Parker named Executive Chairman; ~$80M raised and more than $100M in debt plus $300M in future supplier obligations forgiven (as reported by The Wall Street Journal) |
| August 1, 2024 | FLUX.1 launches from Black Forest Labs (Rombach, Blattmann, Esser) |
| September 2024 | James Cameron joins Stability AI board |
| October 22, 2024 | SD 3.5 Large and Large Turbo released |
| October 29, 2024 | SD 3.5 Medium released |
| December 2024 | Akkaraju reports triple-digit revenue growth; debt eliminated |
Several of these dates are worth re-emphasizing because they line up uncomfortably. SD3 was previewed February 22; the paper went up March 5; Mostaque resigned March 23. The original SD3 research team left in waves through March. The open weights release on June 12 happened in the middle of an interim co-CEO regime, thirteen days before Prem Akkaraju and Sean Parker took over and the recapitalization closed. By the time SD 3.5 launched in October, the company was operating with new leadership, James Cameron on the board, and a clear positioning around enterprise customers and entertainment-industry partnerships. [7] [15] [16]
CoreWeave, the GPU cloud company, was one of Stability's main compute providers alongside Amazon Web Services and Google. [20] As part of the June 2024 rescue, The Wall Street Journal reported, the investor group persuaded Stability's suppliers to forgive more than $100 million in debt and $300 million in future spending obligations; how much of that was owed to CoreWeave has not been reported. [24]
Departures and Black Forest Labs
The departure of Robin Rombach, Andreas Blattmann, and Patrick Esser to found Black Forest Labs is critical context for SD3's trajectory. Rombach was first author on the original 2021 latent diffusion paper. Esser was first author on the SD3 paper in 2024. By August 2024, FLUX.1 had launched. The same lineage that produced Stable Diffusion and SD3 immediately produced FLUX, and FLUX was widely seen as a continuation of where SD3 had been heading without the licensing friction. Black Forest Labs publicly raised $31 million at launch and went on to raise substantially more across 2024 and 2025; and its December 2025 Series B valued it at $3.25 billion post-money. [8] [14]
Continue reading
- Stability AIStability AI is a generative artificial intelligence company best known for helping fund and release the Stable Diffusion family of image models.Mentioned 30 times above
- SDXL (Stable Diffusion XL)SDXL, short for Stable Diffusion XL, is an open-weights latent text-to-image diffusion model released by Stability AI on 26 July 2023, built around a 2.6 billion parameter U-Net backbone, two text encoders…Mentioned 22 times above
- Flux (text-to-image model)Flux (styled FLUX) is a family of image generation models, and since 2026 multimodal video, audio and action models, developed by Black Forest Labs (BFL)Mentioned 20 times above
- Stable DiffusionStable Diffusion is a family of generative image models that can synthesize and edit images from text and other conditions.Mentioned 19 times above
- Black Forest LabsBlack Forest Labs (BFL) is a German-American artificial intelligence company founded in 2024 by Robin Rombach, Andreas Blattmann and Patrick EsserMentioned 11 times above
- Rectified FlowRectified Flow is a generative modeling framework that learns a transport ordinary differential equation between two probability distributions by regressing a velocity field along straight-line interpolations…Mentioned 11 times above
More from See also
- Robin Rombach
- Emad Mostaque
- Sean Parker
- Prem Akkaraju
- James Cameron
- DALL-E 3
- Midjourney v6
- Midjourney V7
- Imagen 3
- Ideogram
- Latent diffusion model
- Diffusion model
- Transformers
- Attention
- Variational Autoencoder
- CLIP (Contrastive Language-Image Pre-training)
- T5 (language model)
- LoRA (Low-Rank Adaptation)
- DreamBooth
- Fine-tuning
- ComfyUI
- AUTOMATIC1111
- Civitai
- Hugging Face
- CoreWeave
- Generative AI
- Text-to-Image Models
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7Esser, P., Kulal, S., Blattmann, A., Entezari, R., Muller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., Podell, D., Dockhorn, T., English, Z., Lacey, K., Goodwin, A., Marek, Y., & Rombach, R. (2024). "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis." arXiv:2403.03206. arxiv.org/...2403.03206
- ^1 ^2 ^3 ^4 ^5 ^6"Stable Diffusion 3: Research Paper." Stability AI, March 5, 2024. stability.ai/...stable-diffusion-3-research-paper
- ^1 ^2 ^3 ^4 ^5"Stable Diffusion 3 Medium." Stability AI News, June 12, 2024. Hugging Face model card: huggingface.co/...stable-diffusion-3-medium
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9"Introducing Stable Diffusion 3.5." Stability AI News, October 22, 2024. stability.ai/...introducing-stable-diffusion-3-5
- ^1 ^2 ^3"Stable Diffusion 3.5 Medium Released." Stability AI News, October 29, 2024.
- ^"Inside Stability AI's Bad Breakup with Coatue and Lightspeed Venture." Fortune, March 2024.
- ^1 ^2"Stability AI CEO Resigns." TechCrunch, March 23, 2024.
- ^1 ^2"Stable Diffusion Creators Launch Black Forest Labs, Secure $31M for FLUX.1 AI Image Generator." VentureBeat, August 1, 2024.
- ^"DALL-E 3 System Card." OpenAI, October 2023.
- ^Stability AI Developer Platform documentation, 2024-2025.
- ^1 ^2 ^3"Stability AI apologizes for disappointing Stable Diffusion 3, promises 'much improved' model soon." The Decoder, July 7, 2024. the-decoder.com/...omises-much-improved-model-soon
- ^1 ^2"Stability AI Community License (October 2024 revision)." Stability AI, October 22, 2024.
- ^1 ^2 ^3"Temporary Stable Diffusion 3 Ban." CivitAI, June 2024. civitai.com/...temporary-stable-diffusion-3-ban
- ^1 ^2 ^3"FLUX.1 Image Models." Black Forest Labs, August 2024 onward.
- ^"Former WETA Digital CEO Prem Akkaraju, Sean Parker Join Stability AI." Deadline, June 2024.
- ^"James Cameron Joins Stability AI Board of Directors." Stability AI News, September 2024.
- "Stable Diffusion 3.5: Architecture and Inference." LearnOpenCV, 2024.
- "Stability AI's New CEO Says Business Growing by Triple Digits and No Debt." Fortune, December 2024.
- ^Lanz, J. A. "Licensing Questions Plague Stability AI as SD3 Image Generator Gets Banned." Decrypt, June 18, 2024. decrypt.co/...sd3-license-stability-ai-civit-ai-ban
- ^1 ^2"Instability at Stability AI." The Batch, DeepLearning.AI, 2024. deeplearning.ai/...financial-and-market-challenges
- ^"SD3 Unbanned: Community Decision on Its Future at Civitai." Civitai, July 22, 2024. civitai.com/...6221
- ^1 ^2 ^3"License Update." Stability AI, July 5, 2024. stability.ai/...license-update
- ^1 ^2"Announcing the Open Release of Stable Diffusion 3 Medium." Stability AI, June 12, 2024 (archived June 13, 2024). web.archive.org/...stable-diffusion-3-medium
- ^"Stability AI appoints new CEO, closes funding round reportedly worth $80M." SiliconANGLE, June 25, 2024. siliconangle.com/...ing-round-reportedly-worth-80m
- ^"Stability AI Secures Significant New Investment from World-Class Investor Group and Appoints Prem Akkaraju as CEO." Stability AI, June 25, 2024. stability.ai/...secures-significant-new-investment
- ^Podell, D., et al. (2023). "SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis." arXiv:2307.01952. arxiv.org/...2307.01952
- ^1 ^2"Stable Diffusion 3.5 Medium" model card. Stability AI, Hugging Face. huggingface.co/...stable-diffusion-3.5-medium
- ^"Three of Stable Diffusion's original developers reportedly leave Stability AI." SiliconANGLE, March 20, 2024. siliconangle.com/...-reportedly-leave-stability-ai
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
6 revisions · v6 · 6,135 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check of the full article against its cited sources (120 claims, 24 sources fetched); 29 material + 22 minor fixed
Cite this page: AI Wiki. "Stable Diffusion 3." aiwiki.ai, updated 28 Oct 2026, fact-checked 2 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/stable_diffusion_3


