Training AI to Paint with Code

Training AI to Paint with Code is an experimental AI art project published by designer and researcher Surya Narreddi in March 2026. The project trains a language model with reinforcement learning to generate editable JavaScript sketches for the p5.brush drawing library. The program is then rendered into an image. This differs from a conventional text-to-image system that returns only a raster image, because a user can inspect and modify the generated drawing instructions.^[1]

The public project page lists Narreddi, Cameron Franz, and Alex Wang as its team. It identifies Franz as a core collaborator who helped build the training infrastructure and says Wang helped guide the project. It does not present the work as a released commercial product or a peer-reviewed model.^[1]

System design

The reported training loop has four main steps:^[1]

1. The model receives a natural-language request for a painting.
2. It writes a complete JavaScript sketch using p5.brush.
3. A sandboxed Puppeteer environment executes the sketch and renders a PNG image.
4. A separate model compares the image with reference paintings, producing a reward used for a Group Relative Policy Optimization (GRPO) update.

p5.brush is a graphics library for p5.js that provides custom brushes, natural-media-style fills, and hatching. The p5.js project lists it in its community library directory.^[2] In the painting project, the generated code is the editable artifact: changing the JavaScript can change individual shapes, colors, strokes, or composition without regenerating the whole image from a new text prompt.^[1]

GRPO was introduced in the DeepSeekMath work as a policy-optimization method that estimates relative advantages from groups of sampled outputs without training a separate value model.^[3] The painting project applies that general approach to a subjective visual task. Its reward is based partly on pairwise aesthetic comparisons rather than on an objectively checkable answer.^[1]

Reward design

Narreddi reports that the first reward rubric combined nine signals. These included whether the sketch compiled, whether it used p5.brush, a code-length target, HPSv3, prompt adherence, recognizability, aesthetics, technique, and depth. The project page says the run plateaued near a reward of 0.65 and repeatedly produced similar flat flower images.^[1]

The project's analysis found that five model-judged quality signals had correlations between 0.85 and 0.95, while the code-length reward had already saturated. Because these figures come from the project's own training logs and have not been independently reproduced, they describe that run rather than a general property of aesthetic reinforcement learning.^[1]

The revised rubric reduced the system to four components:^[1]

Scroll sideways for more →

HPSv3 is a visual preference model introduced with a dataset of 1.08 million images and 1.17 million pairwise annotations. Its authors designed it to rank text-to-image outputs in ways that better reflect human preferences.^[4] In Training AI to Paint with Code, HPSv3 is one part of a project-specific reward, not an independent evaluation of the finished system.^[1]

The project page reports that, with the same base model and training data, the revised reward reached the previous reward plateau three times faster and continued above it. It also reports that generated programs shortened from about 13,500 tokens to fewer than 2,000 tokens. The write-up does not provide enough run-level detail to establish statistical significance or reproduce that comparison.^[1]

Reference pool and prompt optimization

The pairwise judge compares each new render with examples from a pool of 581 model-generated reference paintings. According to the project page, 1,664 images were manually rated, yielding 117 examples in a top tier and 266 in a middle tier. Another 198 generated images supplemented underrepresented colors in the pool. The page states that the pool contains model outputs rather than human-made paintings.^[1]

The reference images came from two generation pipelines. One used several frontier models to iterate against reference photographs under a vision-language-model judge; the other used a larger batch generation run. The project does not publish an independent audit of those reference images or the pairwise judge.^[1]

The team also used GEPA to optimize the system prompt over 200 reported iterations. GEPA is a prompt-optimization method that proposes and tests prompt revisions by reflecting on execution traces and textual feedback.^[5] The project page says an early prompt included about 400 lines of p5.brush documentation and led models to invent nonexistent interfaces. The optimized prompt instead used an allowlist of eight brush methods, without API documentation or examples. The claim that this reduced interface hallucinations is based on the project's own observations.^[1]

Relationship to earlier work

The idea of generating images through executable drawing actions predates this project. SPIRAL, published at ICML in 2018, trained an adversarial reinforcement-learning agent to generate programs that a graphics engine executed. Its discriminator supplied the reward signal.^[6] A 2019 ICCV paper, Learning to Paint With Model-Based Deep Reinforcement Learning, trained agents to reconstruct images as ordered sequences of parameterized strokes through a learned neural renderer.^[7]

Training AI to Paint with Code uses a different interface: a language model writes a complete, human-readable JavaScript sketch in response to text, and the reward incorporates a manually curated aesthetic reference pool. The project therefore combines earlier programmatic-painting ideas with language-model code generation and contemporary preference optimization. The public evidence does not establish that it outperforms those systems or modern image generators.^[1][6][7]

Status and limitations

As of September 3, 2026, the public project page describes the work as ongoing. The March write-up said that one final training run was planned and that a full technical report would be published in June 2026, but the page did not link a technical report when accessed in September.^[1]

The page also does not link model weights, training code, a dataset, or a reproducible evaluation package. Its body does not identify the exact base-model checkpoint, training-compute budget, number of optimization steps, random seeds, or uncertainty estimates. Social-media posts have supplied additional model attributions and qualitative descriptions, but those details are not treated here as verified specifications because the project page does not substantiate them.

Narreddi explicitly describes the approach as slower than direct image generation and says it is not necessarily a better way to make images. Its demonstrated benefit is editability of the generated code. Claims about image quality, learning speed, reward correlations, and code compression remain creator-reported results from an exploratory project rather than independently validated benchmarks.^[1]

References

1. ^Surya Narreddi. "Training AI to Paint with Code." March 2026. Accessed September 3, 2026. surya.website/rling-qwen-to-paint-with-code
2. ^p5.js. "Libraries." Accessed September 3, 2026. p5js.org/libraries
3. ^Zhihong Shao et al. "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models." arXiv:2402.03300, February 2024. arxiv.org/...2402.03300
4. ^Yuhang Ma et al. "HPSv3: Towards Wide-Spectrum Human Preference Score." ICCV 2025. arxiv.org/...2508.03789
5. ^Lakshya A. Agrawal et al. "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning." arXiv:2507.19457, July 2025. arxiv.org/...2507.19457
6. ^Yaroslav Ganin et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." Proceedings of Machine Learning Research 80, 2018. proceedings.mlr.press/...ganin18a
7. ^Zhewei Huang, Wen Heng, and Shuchang Zhou. "Learning to Paint With Model-Based Deep Reinforcement Learning." ICCV 2019. openaccess.thecvf.com/..._Learning_ICCV_2019_paper