Starchild-1

Starchild-1 is a real-time audio-video world model built by the AI lab Odyssey. It generates synchronized video and sound autoregressively, chunk by chunk, while a user streams new text, speech and action inputs into the running simulation. Odyssey previewed the model on 17 May 2026 in a post by co-founder Oliver Cameron (the post's byline; Odyssey's blog index lists it under 18 May, and the technical report PDF was uploaded on 16 May), describing it as "the world's first multimodal world model", and published a self-hosted technical report alongside it ^[1][2]. The company repeated the framing on X the following day, calling Starchild-1 "the first ever real-time multimodal world model" ^[4]. That priority claim is Odyssey's own. The earlier joint audio-video systems the report surveys are offline bidirectional generators, but the report itself names OmniForcing (Su et al., 2026) as concurrent work on causal, streaming audio-video generation ^[2].

The report gives the model a plainer title, "Starchild-1: A real-time multimodal world model", carries a "Preprint" marker, and is credited to "Team Odyssey" rather than to individual first authors ^[2]. According to the abstract, the model runs "at up to 24 fps on modern hardware" ^[2].

What Odyssey released

Cameron's post frames the launch as "a preview of Starchild-1" rather than a general release, and pairs it with embedded audio-video samples and a link to the technical report ^[1]. Odyssey's research index lists Starchild-1 among its models with the one-line description "A multimodal world model that generates audio-visual simulations", and the accompanying panel page "The Making of Starchild-1" features researchers Jeff Hawke, Jenny Seidenschwarz and Vighnesh Birodkar discussing the work ^[7][8].

The problem it addresses

Most world models to that point operated on pixels alone. Odyssey's argument is that sound carries a large share of the information a system needs in order to model how an environment behaves, and that a silent simulator discards it: the post lists conversation, emotion, crashing waves and chirping birds as signals humans use constantly ^[1].

The technical report sets out the engineering reason this is hard rather than merely unfashionable. Audio and video "evolve at fundamentally different temporal frequencies and information densities", so a chunk of audio latents and a chunk of video latents do not line up on the same clock ^[1][2]. During long-horizon rollout, errors compound across the two streams: an artifact in one modality propagates through cross-modal attention and destabilizes the other ^[2]. The report also notes that techniques developed for causal adaptation of text-to-video models generalize poorly here, because much of their demonstrated stability relies on the visual structure and next-frame predictiveness of video latents, which the audio stream does not share ^[2].

Architecture and training

Starchild-1 is not trained from scratch. It starts from Ovi, an open bidirectional "twin backbone" joint audio-video diffusion transformer published by Chetwin Low, Weimin Wang and Calder Katyal in 2025, which couples symmetric audio and video towers through blockwise cross-attention and scaled rotary position embeddings ^[2][3]. The audio stream operates on latents derived from compressed Mel spectrograms; at inference, audio latents are decoded to Mel spectrograms through an audio variational autoencoder and converted to waveforms by a neural vocoder ^[2].

Odyssey's contribution is the pipeline that converts that offline bidirectional generator into a causal, streaming one. The report describes three stages ^[2]:

Two findings from the second stage are worth singling out. Initializing causal adaptation from the few-step distilled model rather than directly from the 50-step teacher produced markedly more stable rollout, which the report attributes to splitting one hard optimization problem into two easier ones ^[2]. And raising the classifier-free guidance scale on the audio stream during ODE trajectory generation produced more pronounced, better-aligned lip motion, giving the student a stronger multimodal alignment signal to learn from ^[2]. In the third stage the report likewise raises the audio guidance scale to 7 for the frozen Ovi real critic, to maximize the critic's quality advantage over the fake critic during rollout adaptation ^[2].

Asynchronous KV-cache

The KV cache is where the different clocks of the two modalities are reconciled. Starchild-1 caches cross-attention keys and values across modalities in addition to the usual self-attention states, which the report says improves multimodal temporal consistency during streaming ^[2]. The cache is deliberately asymmetric: sink tokens in the video cache stabilize long-horizon rollout, while equivalent sink tokens in the audio cache gave limited benefit and are evicted, so the video cache carries an extra chunk of latent frames ^[2].

The report also departs from prior causal-generation work on prompt changes. Rather than re-caching after a prompt transition, Starchild-1 preserves the original video sink tokens, on the reasoning that aggressive cache resets introduce distribution shift in the visual latents that then propagates into audio through cross-attention ^[2]. A further adjustment concerns the granularity of audio state: defining audio-visual state at the phoneme or sub-phoneme level made the next acoustic state poorly conditioned, so Odyssey defines it at the level of single words or short utterances instead ^[2].

Rollout configuration

Orchestration layer

Because the model predicts chunks at the token level, Odyssey adds a higher-level component that turns user intent into a synchronized action stream. The orchestrator builds what the report calls Campaigns: temporally ordered prompt sequences describing evolving scene dynamics, speech content, narration, ambient audio, music and interaction events over future rollout windows. Campaigns are generated online by an external language model conditioned on the current interaction state and the rollout history so far ^[2].

Campaign actions are injected asynchronously at chunk boundaries through a rolling prompt-update interface, and the system decides per update whether to trigger cross-attention recaching or a soft prompt transition ^[2]. Two smaller mechanisms handle speech specifically. Chunk-local syllable budgets, derived from the duration of each audio segment, constrain prompt generation so that requested speech actually fits in the chunk and is not truncated. And because repeating an identical prompt across consecutive chunks degraded speech quality, the orchestrator structures updates as temporally local action deltas rather than re-conditioning on the same text ^[2].

Training data

Odyssey built specialized prompt datasets for the causal adaptation stages rather than relying on the base model's priors alone. The report identifies seven categories of soundscape scenario: SFX only; monologue in-world character speech; conversational in-world character dialogue; character speech with SFX; monologue disembodied narrator speech; disembodied narrator dialogue; and disembodied narrator speech with SFX. Disembodied narrator dialogue is arranged so that speech turns fall on rollout chunk boundaries, letting the model learn responses through cache context and prompt changes at natural transitions. Odyssey reports training on 2.5 million representative prompt samples across the seven categories ^[2].

Candidate generations were filtered with a separate vision-language-model judge to favor prompts that produce well-conditioned teacher samples. The report lists prompt-writing heuristics that worked: ordering speech prompts before general ambience prompts, anchoring on continuous broadband sounds when generating sounds from narrowband sources, and emphasizing amplitude and amplitude changes in descriptions ^[2].

Evaluation

The report is candid that existing benchmarks do not measure what Starchild-1 is for. Standard joint audio-video metrics evaluate short offline clips of roughly 4-5 seconds, whereas the model targets long-horizon interactive generation under continuously changing conditioning ^[2]. Odyssey therefore treats the quantitative table as evidence that causal adaptation preserved the base model's behavior, not as a demonstration of superiority.

Measured against Ovi, Starchild-1 improves on perceptual visual quality and on audio-visual event synchronization, and regresses on several semantic audio and cross-modal alignment measures ^[2]:

Odyssey's reading of the table is that autoregressive rollout adaptation encourages temporally smoother latent dynamics, which helps perceptual image quality, while the largest degradations fall on CLAP and ImageBind semantic alignment and likely reflect distribution shift accumulated during long-horizon rollout. Lip-sync measures LSE-C and LSE-D are essentially unchanged, which the report reads as evidence that low-level temporal alignment survived causal adaptation ^[2].

Three of the metrics are Odyssey's own definitions ^[2]. Audio-visual Event Synchronization cross-correlates a per-frame perceptual-hash change signal against the RMS envelope of the soundtrack within a plus or minus 0.5 second lag window, giving a content-agnostic temporal-alignment score usable on clips where SyncNet-style lip-sync metrics cannot be computed. Audio Alignment checks semantic correctness: speech clips are transcribed with Whisper-large-v3 and scored as max(0, 1 - WER) against the speech text in the prompt, while non-speech clips are scored by LAION-CLAP audio-text cosine similarity remapped to [0, 1]. Audio-visual Coherence is a weighted sum of the three:

AV coherence = 0.40 * Subject Consistency + 0.35 * Audio Alignment + 0.25 * AV Event Sync

Interaction modes

The qualitative evaluation covers four interaction regimes, all supported by a single model without task-specific specialization: interactive world exploration, scripted dialogue control, conversational interaction, and narrator-style companionship ^[2]. The report illustrates mid-rollout steering with five-second rollouts in which a new prompt is injected after 2.5 seconds ("A frog croaks", "A penguin emerges", "Lightning strike", "The volcano erupts"), showing that the accompanying Mel spectrograms change structurally rather than merely spiking in amplitude. Longer examples are shown as 30-second autoregressive rollouts ^[2].

Limitations

Odyssey lists the shortcomings explicitly. Starchild-1 inherits the failure modes of its bidirectional teacher and adds rollout-specific ones: extended interactive sessions still show gradual drift in scene identity and acoustic environment over long horizons. The conversational interaction mode has not been evaluated against dialogue coherence standards from the language-modeling community. And the evaluation is deliberately qualitative, so the report claims to have demonstrated that the four interaction modes are achievable without establishing quantitative benchmarks that would let future causal audio-video models be compared rigorously against it ^[2]. Future work named in the conclusion includes scaling model capacity and training data, extending stable rollout horizons, and developing quantitative evaluation protocols for interactive causal generation ^[2].

Comparison with offline audio-video generators

Odyssey's announcement contrasts Starchild-1 with what it calls traditional audio-video models, naming Google DeepMind's Veo as an example of a system that generates clips of a specific length offline. The post credits those models with major progress in visual fidelity and audio synchronization, and locates the difference elsewhere: once generation begins, the future trajectory of the output is fixed, whereas a causal model conditions each next state on past observations plus whatever the user streams in ^[1]. The technical report describes Veo 3 as having set the closed-source standard for bidirectional audio-visual generation at the time of its release ^[2], so the distinction Odyssey draws is about control flow rather than output quality: Veo 3 and Veo 3.1 return a finished clip, while Starchild-1 accepts new conditioning at every chunk boundary of an ongoing rollout ^[1][2].

The technical report places Starchild-1 within a longer lineage. It traces bidirectional text-to-video generation through latent diffusion and flow matching; joint bidirectional audio-video work split into multi-tower designs (JavisDiT, UniVerse-1, Veo 3, Ovi, Harmony, LTX-2, ALIVE, MOVA) and single-tower designs (Apollo, JavisDiT++); and the causal turn represented by Diffusion Forcing, CausVid, Self-Forcing and their successors. Action-conditioned interactive systems including Hunyuan-Gamecrafter-2, LingBot, GAIA-2 and Genie 3 are cited as the move from open-loop prediction to closed-loop world modeling, with Starchild-1 positioned as the extension of that shift into joint audio-visual space ^[2].

Team

The technical report and the announcement give identical contributor lists, in alphabetical order ^[1][2]:

A further nineteen people are credited as the full team ^[1][2]. Hawke, Odyssey's co-founder and chief technology officer ^[5][7], appears as "Jeffrey Hawke" in the report and as "Jeff Hawke" in the blog post ^[1][2].

Availability and reception

Starchild-1 was presented as a preview accompanied by video samples and the technical report ^[1][2]. Writing two days after the announcement, Nathan Benaich of Air Street Press noted that the Starchild-1 preview and technical report were live, that Agora-1 could be played at agora.odyssey.ml, and that Odyssey-2 was the model then available through Odyssey's developer API ^[5]. Air Street Capital is an investor in Odyssey, which Odyssey's own funding announcements confirm, so that write-up is not disinterested coverage ^[5][10].

Benaich's summary matched the report's own framing closely, including the 24 fps figure, the identification of Ovi as the base model, the asynchronous KV-cache, the four interaction regimes, and the acknowledged drift in scene and acoustic identity over long horizons ^[5]. He also characterized Odyssey's models at that point as research substrate rather than products ^[5].

Trade coverage was more skeptical. WinBuzzer, reporting on 18 May 2026, wrote that "a launch video can illustrate responsiveness, but it cannot settle durability, controllability, or whether the model stays stable once a scene becomes crowded", and argued that update speed, scene consistency, controllability, shared-environment behavior and tooling access would determine whether Starchild-1 could support real workflows "rather than remain a directional research debut" ^[6]. The same piece placed Starchild-1 in a crowded field alongside Genie 3, Sora, Veo and NVIDIA's robot-training world models, describing its distinct lane as real-time audiovisual simulation that stays responsive while users, agents, motion and sound all change the environment at once ^[6].

Odyssey has continued to cite Starchild-1 in later announcements. The June 2026 Series B post lists it among the company's research results with the line "With Starchild-1, we introduced the first real-time multimodal world model", again an in-house claim ^[9].

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Oliver Cameron, "Starchild-1: The First Real-Time Multimodal World Model", Odyssey, 17 May 2026. odyssey.systems/introducing-starchild-1
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36 ^37 ^38 ^39 ^40 ^41 ^42Team Odyssey, "Starchild-1: A real-time multimodal world model" (technical report, preprint). starchild.odyssey.ml/starchild-1.pdf
3. ^1 ^2Chetwin Low, Weimin Wang and Calder Katyal, "Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation", arXiv:2510.01284. arxiv.org/...2510.01284
4. ^1 ^2Odyssey (@odysseyml), announcement post, X, 18 May 2026. x.com/...2056416808580816957
5. ^1 ^2 ^3 ^4 ^5Air Street Press and Nathan Benaich, "From clip-makers to simulators: Odyssey's new world models", 19 May 2026. press.airstreet.com/...odyssey-starchild-1-agora-1
6. ^1 ^2Markus Kasanmascheff, "Odyssey Launches Starchild-1 Real-Time World Model", WinBuzzer, 18 May 2026. winbuzzer.com/...al-time-multimodal-world-m-xcxwbn
7. ^1 ^2"A leading body of world model research", Odyssey research index. odyssey.systems/research
8. ^"The making of Starchild-1", Odyssey. odyssey.systems/the-making-of-starchild-1
9. ^"Our $310 Million Fundraise to Accelerate World Simulation", Odyssey. odyssey.systems/our-series-b
10. ^"Investment from NVIDIA and Samsung", Odyssey. odyssey.systems/investment-from-nvidia-and-samsung