# Kyutai > Source: https://aiwiki.ai/wiki/kyutai_labs > Updated: 2026-10-28 > Fact-checked: 2026-10-27 > Categories: AI Research, Research Organizations > License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)" > Cite as: AI Wiki. "Kyutai." aiwiki.ai, 28 Oct 2026. https://aiwiki.ai/wiki/kyutai_labs > From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution. **Kyutai** is a privately funded nonprofit [artificial intelligence](https://aiwiki.ai/wiki/artificial_intelligence) research laboratory based in Paris, France, launched on 17 November 2023 with close to 300 million euros in initial funding.[^1][^2] The lab was created jointly by the French telecommunications group Iliad (Xavier Niel), the shipping and logistics group CMA CGM (Rodolphe Saadé), and Eric Schmidt's philanthropy, then operating as Schmidt Futures (Kyutai's website now credits Schmidt Sciences), and was announced at the Scaleway ai-PULSE conference held at Station F in Paris.[^1][^2][^3][^11] Kyutai describes itself as an open-science laboratory: it said at launch that it would share models, code and technical details for each project, and most of its releases come with model weights, inference code and a paper or technical write-up.[^38][^4][^7] Since 2024 the lab has concentrated on speech and audio: it has shipped the full-duplex spoken-dialogue model **Moshi**, the streaming neural audio codec **Mimi**, the speech-to-speech translation models **Hibiki** and **Hibiki-Zero**, the small multilingual [language model](https://aiwiki.ai/wiki/language_model) **Helium**, the modular voice stack **Unmute** with its streaming speech-to-text and text-to-speech models, and the CPU text-to-speech model **Pocket TTS**, among other artifacts.[^4][^5][^6][^7][^8][^10][^52] In December 2025 three of its founding scientists launched [Gradium](https://aiwiki.ai/wiki/gradium), a commercial voice AI company in which Kyutai holds shares, and during 2026 Kyutai's research widened into vision, [world models](https://aiwiki.ai/wiki/world_model) and robotics, including KE:SAI, a physical-AI laboratory set up with the ELLIS Institute Tübingen.[^47][^48][^59][^65][^70]
Kyutai logo
Kyutai logo, from the official website kyutai.org. Trademark of its owner.
## Infobox | Field | Value | |---|---| | Type | Privately funded nonprofit research laboratory | | Founded | November 2023 (announced 17 November 2023, Paris)[^1] | | Headquarters | Paris, France[^1] | | Founding backers | Iliad (Xavier Niel), CMA CGM (Rodolphe Saadé), Schmidt Futures (Eric Schmidt)[^1][^2] | | Initial funding | Nearly 300 million euros at launch, including 100 million euros each from Iliad and CMA CGM[^2] | | Chief Executive | Patrick Pérez, per Kyutai's team page as of October 2026; his personal website describes him as Kyutai's former CEO (see Leadership changes)[^9][^75] | | Scientific committee | [Yann LeCun](https://aiwiki.ai/wiki/yann_lecun), [Yejin Choi](https://aiwiki.ai/wiki/yejin_choi), Bernhard Schölkopf[^39] | | Compute | At-cost access to about 1,000 [NVIDIA H100](https://aiwiki.ai/wiki/nvidia_h100) GPUs at Scaleway (2023)[^1] | | Staff | 28 people listed on the team page (October 2026)[^9] | | Spinout | [Gradium](https://aiwiki.ai/wiki/gradium) (voice AI company, launched December 2025; Kyutai is a shareholder)[^47][^48] | | Affiliated lab | KE:SAI, with the ELLIS Institute Tübingen (May 2026; initial funding from Kyutai)[^60] | | Headline releases | Moshi and Mimi (2024-09), Helium 1 (2025), Hibiki (2025-02), MoshiVis (2025-03), Kyutai STT, Kyutai TTS and Unmute (2025-06/07), Pocket TTS (2026-01), Hibiki-Zero (2026-02), MIRA (2026-07, with General Intuition), REKI (2026-10)[^4][^5][^6][^7][^8][^10][^42][^43][^44][^52][^65][^70] | | Licensing | Code mostly MIT or Apache 2.0; weights mostly CC-BY 4.0, with some under CC-BY-SA 4.0 or the non-commercial CC-BY-NC-SA 4.0[^4][^7][^24][^25][^81] |
"Unveiling of Moshi: the first voice-enabled AI openly accessible to all.", official video from kyutai, published July 4, 2024. Watch on YouTube.
## History ### Founding and announcement Kyutai was unveiled on 17 November 2023 at Scaleway's ai-PULSE conference at Station F in Paris, in the presence of its scientific team and its three co-founders, Niel, Saadé and Schmidt; French president Emmanuel Macron addressed the conference in a prerecorded video message.[^1][^2][^31] The launch press release described Kyutai as the first European private-initiative laboratory dedicated to open research in artificial intelligence.[^2] Iliad and CMA CGM each contributed 100 million euros, Schmidt Futures joined as the third co-founder, and the release said that with "nearly €300 million already invested" the co-founders were inviting other entities to provide long-term financing for the nonprofit.[^2] Fortune, publishing a Bloomberg report, said Schmidt would invest an undisclosed amount, and TechCrunch described Schmidt's foundation as one of several smaller contributors alongside unnamed donors.[^1][^3] TechCrunch's launch report gave the budget as $330 million.[^1] Kyutai's first blog post said that for each project the lab would "share models, codes and technical details" with the wider AI community, and that its first project would focus on voice AI.[^38] Its researchers described the approach as "open science" rather than only open source: TechCrunch reported that they planned to release not just models but also the training code and data explaining how the models were built.[^1] Niel tied the project to publication: "Unfortunately, big tech companies tolerate scientific publications less and less," he said at the launch press conference.[^1] ### Initial scientific team The six scientists named at launch were Patrick Pérez (formerly VP of AI and scientific director of valeo.ai, Valeo's AI research unit), Hervé Jégou (who joined Facebook AI Research when its Paris lab was founded in 2015, serving as a manager and then as a director), Edouard Grave (formerly of Apple's machine learning research group and [Meta FAIR](https://aiwiki.ai/wiki/meta_fair), where he co-led work on LLaMA and the fastText library), Alexandre Défossez (formerly Meta FAIR, where he led the open-source AudioCraft framework), Neil Zeghidour (formerly [Google DeepMind](https://aiwiki.ai/wiki/google_deepmind), where he started and led a generative-audio team) and Laurent Mazaré (formerly a researcher at the trading firm Jane Street, with an earlier stint as a research engineer in DeepMind's computer vision group).[^1][^39][^75][^12] Several of these researchers were already well known in the speech and audio community: Défossez was the first author of the EnCodec paper and Zeghidour the first author of the [SoundStream](https://aiwiki.ai/wiki/soundstream) paper, while Jégou had founded the [FAISS](https://aiwiki.ai/wiki/faiss) vector-search library at FAIR.[^88][^87][^39] Pérez took the role of CEO, and the lab named [Yann LeCun](https://aiwiki.ai/wiki/yann_lecun) (Meta), [Yejin Choi](https://aiwiki.ai/wiki/yejin_choi) (then at the University of Washington), and Bernhard Schölkopf (Max Planck Institute for Intelligent Systems) to its scientific committee; TechCrunch reported that the advisors would review the lab's work once or twice a year.[^1][^39] ### Compute partnership with Scaleway Because Iliad also controls the French cloud provider Scaleway, the launch press release said Kyutai would use computing power made available by Scaleway.[^2] TechCrunch reported at launch that Scaleway had recently acquired about 1,000 [NVIDIA H100](https://aiwiki.ai/wiki/nvidia_h100) GPUs and that they would be available to Kyutai at cost.[^1] ### First public output: Moshi (2024) Kyutai's first headline release came on 3 July 2024, when it presented Moshi, a real-time spoken conversational model, in a keynote with a live demonstration on stage.[^40] The technical report, weights and code followed on 18 September 2024; the preprint, [arXiv](https://aiwiki.ai/wiki/arxiv):2410.00037, is dated 17 September with a revision on 2 October.[^41][^4][^13] The paper calls Moshi "the first real-time full-duplex spoken large language model", with a theoretical end-to-end latency of 160 milliseconds and about 200 milliseconds in practice, and it was released with [open weights](https://aiwiki.ai/wiki/open_weights).[^4][^13] Released alongside Moshi was the streaming neural audio codec Mimi, which Moshi uses to tokenize its input and output audio at 12.5 Hz.[^4][^5][^13] ### Helium, Hibiki, Unmute and Pocket TTS (2025-2026) Through 2025 the laboratory broadened its release portfolio. In January 2025 it released a preview of **Helium-1**, a 2-billion-parameter multilingual text model, followed on 30 April 2025 by the full model, its [model card](https://aiwiki.ai/wiki/model_card) and a write-up on modular language models.[^6][^14] In February 2025 it released **Hibiki**, a streaming French-to-English speech-to-speech translation model.[^7][^15] In March 2025 it released **MoshiVis**, which adds image understanding to Moshi by training 206 million additional parameters on top of the frozen model.[^42] In May 2025 the laboratory unveiled **Unmute**, a modular pipeline that wraps any text-only large language model with streaming speech-to-text and [text-to-speech](https://aiwiki.ai/wiki/text_to_speech_ai) components; the speech-to-text models were open-sourced on 19 June 2025, and Kyutai TTS 1.6B and the Unmute code followed on 3 July 2025.[^8][^43][^16] Papers on the underlying method, Delayed Streams Modeling, and on Continuous Audio Language Models (CALM) appeared in September 2025.[^45][^46] In January 2026 Kyutai released **Pocket TTS**, a 100-million-parameter text-to-speech model designed to run in real time on a CPU and built on the CALM method; multilingual support for six languages (English, French, German, Spanish, Portuguese and Italian) followed on 4 May 2026.[^10][^17] ### Gradium spinout (2025) In 2025 three of Kyutai's founding scientists, Neil Zeghidour, Laurent Mazaré and Alexandre Défossez, co-founded [Gradium](https://aiwiki.ai/wiki/gradium) with Olivier Teboul, formerly of Google Brain, to sell voice models commercially.[^49][^48] According to TechCrunch, Gradium was founded in September 2025 and came out of stealth on 2 December 2025 with a $70 million seed round led by FirstMark Capital and Eurazeo, with participation from Niel, DST Global Partners, Eric Schmidt and other investors; Maddyness, which put the round at 60 million euros, also listed Saadé among the investors.[^47][^48] On 8 July 2026 Gradium said an extension of its seed financing, with new investors including NVIDIA, had brought its funding to $100 million.[^50] Former Kyutai technical-staff member Eugene Kharitonov joined Gradium as a founding scientist.[^49][^9] Zeghidour, Gradium's chief executive, told Maddyness at the launch that the spinout was not a divorce from Kyutai: the two organizations shared premises separated only by a corridor, and Kyutai was a Gradium shareholder that would benefit from Gradium's valuation, though it had no exclusive relationship with the company.[^48] He said businesses, notably in gaming, had wanted Kyutai's open-source models with wider language coverage, a commercial opportunity that fell outside the nonprofit's research mission, and he added that Kyutai was working on new projects around 3D and that other spinouts would follow.[^48] Kyutai's website calls Gradium "our first spin-off" and says it is committed to turning Kyutai's open speech research into production-ready systems.[^11] As of October 2026, Kyutai's team page still listed Mazaré as head of engineering, Défossez as head of research and Zeghidour as audio research advisor.[^9] ### KE:SAI (2026) On 20 May 2026 Kyutai and the ELLIS Institute Tübingen announced KE:SAI (Kyutai ELLIS Scalable Autonomous Intelligence), a nonprofit open-science laboratory for world models, autonomous systems and [physical AI](https://aiwiki.ai/wiki/physical_ai), with offices in Tübingen and Paris.[^59][^60] Its founding team is Andreas Geiger (CEO), Kashyap Chitta (CTO) and Bernhard Schölkopf, with co-founders Bernhard Jaeger and Daniel Dauner; Schölkopf was one of the three members of Kyutai's scientific committee named at its 2023 launch.[^60][^39] Kyutai provides KE:SAI's initial funding, and the new lab's first stated milestone is a fully open self-driving software stack.[^60][^61] In the launch press release Défossez, described as a co-founder of Kyutai, said that with KE:SAI "Kyutai is expanding its capacity to develop and ship frontier AI research and models".[^60] ### Releases since mid-2025 From the second half of 2025, Kyutai's output widened from speech into vision, language-model research, 3D, world models and robotics.[^11][^72] | Date | Release | Area | Summary | |---|---|---|---| | 19 June 2025 | Kyutai STT | Speech recognition | Streaming models stt-1b-en_fr (English and French, with semantic voice activity detection) and stt-2.6b-en; the larger model transcribes 400 real-time streams on one H100[^43] | | 3 July 2025 | Kyutai TTS 1.6B and Unmute code | Speech synthesis, voice agents | Text-to-speech that streams in text as well as audio; English and French[^44][^16] | | September 2025 | Delayed Streams Modeling and CALM papers | Methods | Streaming sequence-to-sequence modelling; continuous (non-token) audio language models[^45][^46] | | December 2025 | CASA | Vision-language | Cross-attention fusion of images into language models; ECCV 2026 paper[^51][^72] | | 13 January 2026 | Pocket TTS | Speech synthesis | 100M-parameter TTS with voice cloning that runs on a CPU[^17] | | January-February 2026 | Invincible Voice | Assistive technology | Open-source dialogue assistant for people living with ALS; online demo on 24 February[^54][^55] | | 12 February 2026 | Hibiki-Zero | Speech translation | 3B model translating French, Spanish, Portuguese and German speech into English, trained without word-level aligned data[^52][^53] | | 14 April 2026 | OVIE | Novel-view synthesis | New viewpoints of a scene from a single photograph[^56] | | 28 April 2026 | ARC-Encoder | LLM efficiency | Compresses text into dense representations for a frozen LLM[^57] | | 30 April 2026 | MoshiRAG | Speech and retrieval | Moshi with asynchronous knowledge retrieval[^58] | | 4 May 2026 | Pocket TTS multilingual | Speech synthesis | Six languages[^10] | | 20 May 2026 | KE:SAI | Physical AI | New lab with the ELLIS Institute Tübingen[^59][^60] | | 26 May 2026 | Kairos | LLM pretraining | Effect of data chronology on time-sensitive knowledge[^62] | | 10 June 2026 | Interactivity post-training | Full-duplex speech | Reinforcement learning for turn-taking, backchanneling and interruptions[^63] | | 1 July 2026 | Surflo | 3D reconstruction | Surfaces from unposed photos; NeurIPS 2026 paper[^64][^72] | | 6 July 2026 | MIRA | World model | Multiplayer Rocket League world model, with General Intuition and Epic Games[^65][^84] | | 10 July 2026 | MuScriptor | Music transcription | Multi-instrument transcription to MIDI, with Mirelo[^66] | | 25 August 2026 | Pocket TTS training code | Speech synthesis | Full training stack, data pipeline and evaluations[^67] | | 28 September 2026 | Pocket TTS with a drifting objective | Speech synthesis | Sampler head trained with drifting instead of flow matching[^68] | | 16 October 2026 | REKI | Robotics | 2.4B vision-language-action model pretrained on public data[^70] | | 20 October 2026 | Small Pocket TTS models | Speech synthesis | Models from 18.2M down to 1.9M parameters[^69] | | 22 October 2026 | CADRE | World models | Open video autoencoders for interactive world models[^71] | ### Leadership changes (2026) Kyutai's team page changed during 2026. An archived copy from June 2026 lists Edouard Grave as chief language officer and Défossez as head of audio research; by August 2026 the page listed Grave among the alumni and Défossez as head of research.[^77][^78][^9] Hervé Jégou is also listed among the alumni.[^9] As of 27 October 2026 the team page still named Patrick Pérez as chief executive officer.[^9] Pérez's personal website, which in April 2026 described him as CEO at Kyutai, had by early October 2026 been changed to describe him as a research director at Google DeepMind in Paris and Kyutai's former "co-founding CEO", and Kyutai's CADRE paper page of 22 October 2026 marks his contribution as "work done while at Kyutai".[^76][^89][^75][^71] Kyutai's blog had not announced a change of chief executive as of 27 October 2026.[^84] ## Mission, structure and funding Kyutai is organised as a privately funded nonprofit and does not have commercial customers.[^1][^2] Its website gives its mission as "to build and democratize artificial general intelligence through open science".[^11] Its outputs are papers, model weights, code and datasets released publicly.[^11][^72] At launch Iliad and CMA CGM each contributed 100 million euros, Schmidt Futures joined as a co-founder, and the total was put at nearly 300 million euros; the co-founders said they were inviting other entities to provide long-term financing.[^2][^1] Kyutai's website lists Iliad Group, CMA CGM Group and Schmidt Sciences as its funders and thanks unnamed "generous donors".[^11] The lab has two institutional offshoots: it is a shareholder in Gradium, and it provides the initial funding for KE:SAI.[^48][^60] As of October 2026 the laboratory's public team page listed 28 people: Pérez as chief executive, Laurent Mazaré as head of engineering, Alexandre Défossez as head of research, Neil Zeghidour as audio research advisor, Sarah Hôte as head of operations, Jennifer Coscas as head of legal and partnerships, Guillaume Rouzaud as HR advisor, plus seven technical-staff members, seven PhD students, three postdoctoral researchers and four research interns.[^9] The alumni list includes Edouard Grave (chief language officer), Hervé Jégou (chief vision officer) and Eugene Kharitonov.[^9] ## Released models and software The following sections summarise the principal artifacts released or co-released by Kyutai through October 2026. Releases are distributed from the kyutai-labs organisation on [GitHub](https://aiwiki.ai/wiki/github) and the kyutai organisation on Hugging Face.[^18][^19] ### Mimi: streaming neural audio codec Mimi is the neural audio codec at the centre of Kyutai's audio stack. It processes 24 kHz mono audio, produces a discrete-token representation at 12.5 Hz, and operates at a bitrate of 1.1 kbps.[^5][^13] Architecturally, Mimi pairs a convolutional encoder-decoder with Transformer modules in the bottleneck and a residual vector quantizer; in the configuration used by Moshi it has eight quantizers with 2,048-entry codebooks, the first of which is trained to carry semantic information distilled from the self-supervised speech model WavLM while the other seven encode acoustic detail.[^4][^17] The default MimiConfig in the Hugging Face transformers library uses a 24 kHz sampling rate, a hidden size of 512, a codebook size of 2,048, and 32 quantizers.[^20] A key engineering property of Mimi is that it is fully streaming and causal: the encoder uses causal convolutions and can be fed one frame at a time, with an 80-millisecond frame size matching Moshi's 12.5 Hz token rate.[^4][^13] In the Moshi paper and release post Kyutai reports that Mimi improves over streaming codecs such as [EnCodec](https://aiwiki.ai/wiki/encodec) and SoundStream and outperforms SpeechTokenizer, RVQGAN and SemantiCodec, while also serving as a semantic tokeniser of the kind required for downstream speech language modelling.[^4][^41] The semantic-token stream is obtained via distillation from a self-supervised speech encoder, an approach inspired by SpeechTokenizer.[^4][^21][^41] Mimi was released in September 2024 alongside Moshi, with weights under CC-BY 4.0 and code under MIT (Python) and Apache 2.0 (Rust).[^4][^18] It has since been integrated as a first-class model in the Hugging Face `transformers` library under the `MimiModel` class.[^20] Kyutai later used a continuous-latent variant of Mimi as the codec for Pocket TTS.[^17] ### Moshi: full-duplex speech-text foundation model Moshi is Kyutai's full-duplex spoken-dialogue model. Its public release on 18 September 2024 and the corresponding technical report (arXiv:2410.00037) were authored by Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave and Neil Zeghidour.[^4][^13] Architecturally, Moshi consists of a 7-billion-parameter temporal "backbone" [Transformer](https://aiwiki.ai/wiki/transformers), initialised from Kyutai's 7B text model Helium (trained on 2.1 trillion tokens), that predicts the next set of multistream tokens at each 80-millisecond frame, paired with a smaller depth Transformer that models inter-codebook dependencies within a frame.[^4][^18][^41] At each timestep the model jointly predicts: (i) the next text token of its own internal monologue, (ii) the next set of acoustic Mimi tokens corresponding to its own speech, and (iii) the next set of Mimi tokens for the user's input speech, which lets the model listen and speak simultaneously without explicit speaker turns.[^4][^13] Kyutai calls the text-prefix construction "Inner Monologue".[^4] The Moshi paper reports a theoretical latency of 160 ms (one Mimi frame plus 80 ms of acoustic delay), and the project README gives a practical overall latency as low as 200 ms on an NVIDIA L4 GPU.[^4][^13] Two voice-tuned variants were released: **Moshiko**, fine-tuned on a male synthetic voice, and **Moshika**, fine-tuned on a female synthetic voice; both are available in bfloat16, int8, and (via MLX) int4 quantisations through [PyTorch](https://aiwiki.ai/wiki/pytorch), MLX, and Rust/Candle backends.[^18] Code is MIT (Python) and Apache 2.0 (Rust), weights are CC-BY 4.0.[^18] MarkTechPost's report on the September 2024 release contrasted Moshi's 200-millisecond practical latency with "the several-second delays observed in existing systems".[^22] Later Kyutai work extended Moshi with vision (MoshiVis), retrieval (MoshiRAG) and reinforcement-learning post-training for interactivity.[^42][^58][^63] ### Helium: small multilingual language model **Helium-1** is a 2-billion-parameter [small language model](https://aiwiki.ai/wiki/small_language_model) aimed at edge and mobile deployments. A preview was released on 13 January 2025; the full release, with an updated technical write-up, appeared on 30 April 2025.[^6][^14][^23] Helium-1 is a Transformer-based decoder model whose architecture Kyutai describes as very similar to LLaMA 2, with grouped-query attention added for inference efficiency.[^14] The preview was trained on 2.5 trillion tokens with a [context window](https://aiwiki.ai/wiki/context_window) of 4,096 tokens, using token-level distillation from a 7-billion-parameter model, and Kyutai said it would release its [JAX](https://aiwiki.ai/wiki/jax) training codebase.[^6][^23] The full model was trained on 64 NVIDIA H100 GPUs for 500,000 steps with a batch of 4 million tokens, distilling Gemma 2 9B, on data filtered from [Common Crawl](https://aiwiki.ai/wiki/common_crawl) with Kyutai's open-source `dactory` pipeline.[^14] The released Helium 1 is a weighted average ("model soup") of the main training run and versions fine-tuned on book, Wikipedia and multilingual subsets of the data.[^14] It covers the 24 official languages of the European Union and is distributed on Hugging Face as `kyutai/helium-1-2b` under a CC-BY-SA 4.0 licence; the preview was released under CC-BY 4.0.[^14][^24][^90] In Kyutai's preview benchmarks, Helium-1 Preview averaged 42.1 on a multilingual suite, against 42.3 for [Gemma](https://aiwiki.ai/wiki/gemma) 2 (2.6B), 43.6 for [Llama 3.2](https://aiwiki.ai/wiki/llama_3_2) (3B) and 40.0 for Qwen 2.5 (1.5B).[^6] Beyond its standalone use, Helium-1 is the language backbone of the CASA-Helium1-VL-2B vision-language model, and the Hibiki-Zero model card names Helium-1-2B as its underlying text model.[^25][^81] In the April 2025 write-up Kyutai presented Helium as a step toward modular model development, showing that models fine-tuned on different data subsets can be combined by averaging their weights.[^14] ### Hibiki: simultaneous speech-to-speech translation **Hibiki** is Kyutai's simultaneous speech-to-speech and speech-to-text translation model. Kyutai introduced it on 5 February 2025, the date of its arXiv paper (2502.03382); Kyutai's blog post about it is dated 10 February 2025.[^26][^29][^7] The paper's authors are Tom Labiausse, Laurent Mazaré, Edouard Grave, Patrick Pérez, Alexandre Défossez and Neil Zeghidour.[^26] Hibiki uses the multistream architecture pioneered in Moshi to model source and target speech jointly, producing aligned text and audio tokens at 12.5 Hz.[^7][^15][^26] The released model supports French-to-English only.[^7][^15] According to the paper, the full Hibiki model has 2.7 billion parameters (a 2.2B temporal Transformer plus a distilled depth Transformer) and models 16 audio codebooks, while a distilled variant called Hibiki-M (1.7 billion parameters, 8 codebooks) runs in real time on device; Kyutai's blog post rounds these to a 2-billion-parameter backbone and a 1-billion-parameter Hibiki-M.[^26][^7] The model card lists a pretraining collection of about 7 million hours of English audio and 450,000 hours of French audio, about 40,000 hours of synthetic parallel French-English speech, and a 900-hour fine-tuning mixture.[^82][^15] The paper reports an ASR-BLEU score of 30.5 in a comparison with offline baselines.[^26][^15] In Kyutai's human evaluations on European Parliament recordings, Hibiki scored 3.73 out of 5 for naturalness against 4.12 for human interpreters, and its objective speaker-similarity score was 0.52 against 0.43 for Meta's Seamless model.[^7] The system supports batch inference of up to 320 concurrent sequences on a single H100 GPU (160 with classifier-free guidance).[^7] Weights are released under CC-BY 4.0 with inference code for PyTorch, MLX (including iOS) and Rust.[^27][^7] ### Hibiki-Zero **Hibiki-Zero**, released on 12 February 2026, is Hibiki's successor. It translates French, Spanish, Portuguese and German speech into English with voice transfer, using a 3-billion-parameter backbone.[^52][^81] Instead of word-level aligned synthetic data it trains on sentence-level aligned data and then uses a [GRPO](https://aiwiki.ai/wiki/grpo)-based [reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning) stage to cut latency while preserving translation quality; Kyutai reports that the model can be adapted to a new input language with less than 1,000 hours of speech, and demonstrated this with Italian.[^52][^53] The paper (arXiv:2602.11072, ICML 2026) also released a 45-hour multilingual evaluation benchmark.[^53][^72] The weights are licensed CC-BY-NC-SA 4.0.[^81] ### Unmute, Kyutai STT, Kyutai TTS and Delayed Streams Modeling **Unmute** is Kyutai's modular voice stack, introduced on 22 May 2025 and fully open-sourced on 3 July 2025.[^8][^16] Where Moshi is an end-to-end speech-text foundation model that handles dialogue itself, Unmute wraps any existing text-only [large language model](https://aiwiki.ai/wiki/large_language_model) with low-latency streaming speech-to-text (STT) and text-to-speech (TTS) models; the STT includes a semantic voice activity detector that predicts whether the user has finished speaking.[^8] Kyutai said it built Unmute because Moshi did not yet match text models in function calling, reasoning and in-context learning.[^8] Kyutai STT, released on 19 June 2025, comes in two versions: stt-1b-en_fr, a low-latency English and French model, and stt-2.6b-en, a larger English-only model; Kyutai says the 2.6B model can transcribe 400 audio streams at once in real time on a single H100.[^43] Kyutai TTS 1.6B, released on 3 July 2025, supports English and French and starts producing audio 220 ms after receiving the first text token; in the Unmute.sh deployment, serving 32 requests at once on an L40S GPU, Kyutai measured 350 ms.[^44] Because it streams in text as well as audio, it can start speaking before a language model has finished writing its reply.[^44] The TTS takes its voice from a 10-second sample, but Kyutai did not release the voice-embedding model, saying it wanted voices to be cloned only with consent, and instead published a repository of voices drawn from datasets and anonymous donations; Kyutai engineer Václav Volhejn wrote that the donation project had collected 174 voices by December 2025.[^44][^30] Unmute's code is MIT-licensed and the model weights are CC-BY 4.0.[^16][^19] Underpinning these models is Delayed Streams Modeling (DSM), described in a September 2025 paper by Zeghidour and colleagues.[^45] DSM models already time-aligned text and audio streams with a decoder-only model and fixed delays between streams: delaying the text stream yields streaming speech recognition, while delaying the audio stream yields streaming text-to-speech.[^45][^44] The `delayed-streams-modeling` repository on GitHub hosts Kyutai's STT and TTS models built on the method.[^19] ### Pocket TTS **Pocket TTS** is Kyutai's CPU-targeted text-to-speech model, introduced on 13 January 2026.[^17] Its generative model (a causal Transformer backbone plus a small sampling head) has 90 million parameters and its codec decoder 10 million, for about 100 million in total; rather than discrete codec tokens, it predicts continuous latent vectors, following Kyutai's CALM research.[^17][^46] It was trained only on public English datasets totalling 88,000 hours and can clone a voice from about 5 seconds of audio.[^17] The project README reports about 200 ms to the first audio chunk and generation roughly six times faster than real time on a MacBook Air M4 CPU, using two CPU cores.[^73] In Kyutai's evaluation on LibriSpeech test-clean, Pocket TTS matched Kyutai TTS 1.6B's word error rate (1.84) and was the only compared model with voice cloning that ran faster than real time on a CPU.[^17] The May 2026 release added French, German, Spanish, Portuguese and Italian while keeping the 100-million-parameter size, and by October 2026 the README also listed Dutch.[^10][^73] On 25 August 2026 Kyutai open-sourced the full training stack, saying a model could be trained in 10 to 20 hours on eight H100 GPUs, and released an internally trained Czech model as a test.[^67] On 28 September 2026 it released an English model whose sampler head was trained with the "drifting" objective instead of flow matching, a 6-layer model distilled from a 24-layer teacher.[^68] On 20 October 2026 it released smaller models from 18.2 million down to 1.9 million parameters, the largest of which Kyutai says matches the quality of earlier Pocket TTS releases.[^69] The code is MIT-licensed and the weights CC-BY 4.0.[^17][^74] ### Vision, world models and robotics Kyutai's vision work began with MoshiVis (March 2025) and CASA (December 2025). CASA, short for Cross-Attention over Self-Attention, revisits [cross-attention](https://aiwiki.ai/wiki/cross_attention) as a way to fuse image features into a language model; Kyutai argues that because image tokens do not accumulate in the key-value cache, memory use stays nearly constant as more images are processed, which suits streaming uses such as live video captioning.[^51] The released models include CASA-Helium1-VL-2B, built on Helium-1 and the image encoder of Qwen2.5-VL-3B, and CASA-Qwen2_5-VL-3B-LiveCC, a variant for live video captioning; the weights are CC-BY-NC-SA 4.0.[^25] OVIE (April 2026) generates new viewpoints of a scene from a single photograph. It is trained on 30 million unpaired internet images, using a frozen monocular depth estimator to create pseudo-target views during training only, and Kyutai says it is 600 times faster than competing approaches.[^56] Surflo (July 2026), co-authored by Kyutai postdoctoral researcher Nicolas Dufour with researchers from École Polytechnique, Kyoto University and UC Berkeley, reconstructs 3D surfaces from a variable number of unposed photos.[^64] MIRA, released in July 2026 by [General Intuition](https://aiwiki.ai/wiki/general_intuition) and Kyutai in collaboration with Epic Games, is a multiplayer world model of the car-football game Rocket League. It pairs a 5-billion-parameter diffusion transformer with a 600-million-parameter video codec and lets four players play a 2v2 match at 20 frames per second; Kyutai says it runs on a single B200 GPU.[^65][^11] It was trained on about 10,000 match-hours of bot-versus-bot gameplay, with no human gameplay, and the team released a 1,000-match-hour slice of the data along with training and inference code; the authors call the project "a stepping stone to physical AI".[^65] CADRE (October 2026) is a family of open video autoencoders designed for interactive world models, with which Kyutai built world models for manipulation and navigation that run in real time on a single GPU.[^71] REKI (October 2026), developed with Inria and the École nationale des ponts et chaussées, is a 2.4-billion-parameter [vision-language-action model](https://aiwiki.ai/wiki/vision_language_action_model) pretrained only on public robot, egocentric human and vision-language data; Kyutai reports that on three robot-manipulation benchmarks it is outperformed only by models about twice its size.[^70] ### Other research (2026) MoshiRAG (April 2026, ICML 2026) adds asynchronous retrieval to Moshi: the model emits a special token when it needs outside knowledge, keeps talking with a natural lead-in while a text-based back end (an LLM or a search engine) fetches an answer, and folds the result into its reply.[^58] Kyutai says its ARC-Encoder, a 3-billion-parameter encoder that compresses text into dense representations fed to a frozen LLM and cuts the sequence length by four or eight times, was used in the MoshiRAG work.[^57] In June 2026 Kyutai published work on post-training full-duplex speech models with GRPO to improve pause handling, turn-taking, backchanneling and responses to interruptions, applied to Moshi and to NVIDIA's PersonaPlex.[^63] Kairos (May 2026) studies how the order of pretraining data affects an LLM's knowledge of time-sensitive facts, using a benchmark of more than 7,000 questions and two 6-billion-parameter models trained on the same Common Crawl snapshots in chronological versus shuffled order.[^62] MuScriptor (July 2026), built with Mirelo, transcribes multi-instrument music recordings into MIDI; it was fine-tuned on 170,000 real recordings (11,000 hours) aligned to note-level annotations, with model sizes from 100 million to 1.3 billion parameters.[^66] Kyutai's publication list for 2026 also includes papers on reinforcement learning for spoken math (Voice of Reason, COLM 2026, whose released models are fine-tuned from [GLM-4-Voice](https://aiwiki.ai/wiki/glm_4_voice)), modular tokenizers for multilingual LLMs (Findings of EMNLP 2026), the hidden randomness of the FID image-generation metric, and drifting-based generative models.[^72][^83] ### Other repositories Beyond the headline models, the kyutai-labs GitHub organisation hosts auxiliary projects including `moshi-finetune` (scripts for fine-tuning Moshi), `tts_longeval` (a framework and benchmark for evaluating TTS models at scale), `flashy` (a training-loop framework), `dactory` (the Common Crawl filtering and data preparation pipeline used for Helium), `flash-attn3-jax` (JAX bindings for FlashAttention 3) and `nanoGPTaudio` (code for a 2025 tutorial on neural audio codecs).[^19] In 2026 the organisation added `invincible-voice` (the code of the Invincible Voice communication assistant), `ovie`, `moshi-rag`, `kairos`, `cadre-autoencoders-and-world-models` and `reki`, among others.[^19] ## Technical themes A consistent set of technical commitments runs across Kyutai's releases. **Streaming and low-latency inference.** Mimi, Moshi, Hibiki, Kyutai STT and Kyutai TTS are explicitly designed to consume input frame by frame and produce output frame by frame, with documented latency budgets in the few-hundred-millisecond range.[^4][^7][^43][^44] This contrasts with batch encoders and decoders used by many open speech models such as [Whisper](https://aiwiki.ai/wiki/whisper). **Joint semantic and acoustic tokenisation.** Mimi's design adopts the SpeechTokenizer pattern of distilling a self-supervised semantic encoder into the first codebook of a residual vector quantiser while training the remaining codebooks on acoustic reconstruction.[^4][^5] This dual representation supports Moshi's Inner Monologue construction, in which time-aligned text tokens are predicted ahead of the audio tokens.[^4][^13] **Multistream language modelling.** Moshi's design of jointly predicting text, the model's own audio, and the user's audio at every frame is generalised in Delayed Streams Modeling to arbitrary combinations of time-aligned streams.[^45] The same machinery underlies simultaneous translation (Hibiki and Hibiki-Zero) and streaming STT and TTS (Unmute).[^7][^16][^45][^52] **Small, specialised, edge-deployable models.** Helium-1 (2B parameters), Hibiki-M (1.7B), Pocket TTS (100M) and the October 2026 Pocket TTS models (down to 1.9M) are explicitly designed for resource-constrained deployment.[^6][^26][^17][^69] In the April 2025 Helium write-up Kyutai presented model soups of specialised Helium variants as a route to modular model development.[^14] **Open release including training pipelines.** Beyond model weights, Kyutai has released `dactory` (Helium's data pipeline), `flashy` (a generic training loop), the Pocket TTS training stack and, with General Intuition, MIRA's training code and a dataset slice.[^14][^19][^67][^65] Licensing is not uniform: while most weights are CC-BY 4.0, Helium 1 is CC-BY-SA 4.0 and some 2025-2026 releases, including CASA, Hibiki-Zero and REKI, use the non-commercial CC-BY-NC-SA 4.0 licence.[^24][^25][^81][^91] ## Applications and adoption Kyutai's models have been adopted as building blocks in third-party projects across [speech recognition](https://aiwiki.ai/wiki/speech_recognition), translation, and conversational interfaces. As of October 2026 the Moshi GitHub repository had more than 11,000 stars and the Pocket TTS repository more than 10,000, and the Mimi codec has been upstreamed into the Hugging Face `transformers` library, where it is exposed under the `MimiModel` class.[^19][^20] Other developers' models build directly on Kyutai components. The Conversational Speech Model (CSM) from [Sesame](https://aiwiki.ai/wiki/sesame) generates Mimi audio codes, and NVIDIA's PersonaPlex, a full-duplex conversational model, is based on the Moshi architecture and weights.[^80][^79] Gradium sells commercial voice models built on the same line of research, and it provides the transcription and voice-cloning text-to-speech components of Invincible Voice, an open-source dialogue assistant for people living with ALS that Kyutai developed with Olivier Goy and released in early 2026; the assistant suggests complete replies and reads the chosen one in the user's own voice.[^54][^55][^11] In the speech recognition space, a June 2025 tutorial on the Scalastic blog argued that Kyutai's streaming speech-to-text could replace OpenAI's Whisper and described how to install it on macOS.[^28] In the translation space, the localisation-industry publication Slator reported Kyutai's claim that Hibiki was "the first model to provide an experience of interpretation close to human professionals".[^29] Volhejn, for whom the Unmute demo was a first project at Kyutai, wrote that because Unmute is a cascaded system, the voice or the LLM can be swapped without retraining.[^30] Within Kyutai itself, the Helium-1 language model is reused as the backbone for CASA-Helium1-VL-2B, a vision-language model built on Helium-1 and the Qwen2.5-VL-3B image encoder using the CASA cross-attention fusion method; CASA-Qwen2_5-VL-3B and a live-captioning variant, CASA-Qwen2_5-VL-3B-LiveCC, have also been published on Hugging Face.[^25][^51] ## Limitations and criticisms Independent coverage and the laboratory's own technical reports identify several limitations. **Limited language coverage in speech models.** Moshi's text backbone was trained on English data and most of its pretraining audio was English speech, and Hibiki supported only French-to-English translation; Slator noted the single language pair while reporting Kyutai's plan to add more.[^4][^7][^29] Later releases widened input coverage: Hibiki-Zero translates from four languages, and Pocket TTS speaks six or more, but Kyutai's translation models still output English, and Kyutai said it aimed to add output languages.[^52][^10][^73] **Interactivity and knowledge in full-duplex models.** Kyutai has described Moshi's limits itself. When introducing Unmute in May 2025 it said Moshi did not yet match text models in function calling, reasoning and in-context learning; its MoshiRAG work aimed to make Moshi better at hard questions and to reduce hallucination; and its June 2026 post-training work said full-duplex models such as Moshi still stay silent when they should not, talk at the wrong time and fail to backchannel.[^8][^58][^63] **Scale.** Helium-1 at 2 billion parameters is far smaller than the [frontier models](https://aiwiki.ai/wiki/frontier_models) released by [Mistral AI](https://aiwiki.ai/wiki/mistral_ai), [Meta AI](https://aiwiki.ai/wiki/meta_ai), or [OpenAI](https://aiwiki.ai/wiki/openai) during the same period, reflecting Kyutai's focus on edge-deployable models.[^6][^14] On Kyutai's own English benchmarks the preview averaged 60.7, below Gemma-2 2.6B (62.2), Qwen2.5 1.5B (63.6) and Llama-3.2 3B (64.7).[^6] **Funding model.** Kyutai has no commercial revenue. At launch its co-founders invited other organisations to provide long-term financing, and Saadé told journalists the project could interest other investors.[^2][^1] Its stake in Gradium gives it a possible financial return from commercial use of its research: Zeghidour told Maddyness that Kyutai would benefit from Gradium's valuation.[^48] **Voice cloning risks.** Kyutai's speech models can reproduce a voice from a short sample, which raises concerns about impersonation and consent (see [voice cloning](https://aiwiki.ai/wiki/voice_cloning); comparable systems include those built on neural codecs in the lineage of [VALL-E](https://aiwiki.ai/wiki/neural_codec_language_models_are_zero-shot_text_to_speech_synthesizers_vall-e)).[^44][^17] Kyutai has handled this differently across releases. For Kyutai TTS 1.6B it withheld the voice-embedding model "to ensure people's voices are only cloned consensually" and offered a curated voice repository instead; for Pocket TTS it released voice cloning but gated those weights on Hugging Face behind terms prohibiting "voice impersonation or cloning without explicit and lawful consent", alongside an ungated version without cloning.[^44][^74][^73] The release posts for these models do not describe audio watermarking.[^44][^17] ## Comparison with other European and global AI laboratories Kyutai is one of several AI research organisations with a large presence in Paris. The comparison below summarises how it relates to its closest neighbours. | Organisation | HQ | Structure | Headline focus | Open weights? | Founded | |---|---|---|---|---|---| | Kyutai | Paris | Private nonprofit | Speech, audio, small LMs, world models | Yes (mostly CC-BY 4.0; some non-commercial)[^4][^7][^81] | November 2023[^1] | | [Mistral AI](https://aiwiki.ai/wiki/mistral_ai) | Paris | For-profit company | General LLMs, code, OCR | Mixed (some open, some commercial)[^32] | April 2023[^32] | | [Hugging Face](https://aiwiki.ai/wiki/hugging_face) | New York / Paris | For-profit company | Hub, libraries, hosting | Distributor of open models[^33] | 2016[^33] | | [Meta FAIR](https://aiwiki.ai/wiki/meta_fair) (Paris) | Paris | Corporate research arm | LLMs, vision, audio (Llama, [AudioCraft](https://aiwiki.ai/wiki/audiocraft), [MusicGen](https://aiwiki.ai/wiki/musicgen)) | Mixed (Llama open-weights)[^34] | 2015 (Paris site)[^34][^39] | | Google DeepMind (Paris) | Paris / London | Corporate research arm | Frontier LLMs, robotics ([Google DeepMind](https://aiwiki.ai/wiki/google_deepmind)) | Selective (e.g. [Gemma](https://aiwiki.ai/wiki/gemma))[^35] | 2014 (DeepMind acquisition)[^35] | Relative to Mistral AI, Kyutai differs both in structure (nonprofit vs. for-profit) and in research scope (audio-first vs. general text-LLM-first).[^31][^32] At Kyutai's launch, Défossez said the lab's aim was not necessarily "to go as fast as Mistral", because it wanted to provide "a scientific purpose, an understanding and a code base to explain the results".[^1] Relative to [Hugging Face](https://aiwiki.ai/wiki/hugging_face), Kyutai is a research producer rather than a distribution platform, although in practice it relies on Hugging Face to host its weights.[^19][^33] Several of Kyutai's founding scientists, including Jégou, Grave and Défossez, came from Meta FAIR, and Zeghidour came from Google DeepMind, where he had led a generative-audio team; unlike those corporate labs, Kyutai is a nonprofit whose stated mission is open science.[^39][^11] TechCrunch's May 2026 list of 21 European startups to watch included Gradium, which it described as "a spinout of French AI lab Kyutai".[^36] ## Significance Coverage of Kyutai has focused on its place in France's AI strategy and on the companies it has produced. Fortune, carrying a Bloomberg report, described the 2023 launch as part of "France's latest push to develop sovereign AI technology".[^3] Kyutai describes Moshi as "the first publicly available full-duplex speech-to-speech model" and says it "has inspired and powered many speech models that came after it"; NVIDIA's PersonaPlex and Sesame's CSM are examples of later models built on Moshi and Mimi respectively.[^58][^79][^80] After Gradium's launch, Science|Business headlined its report "How a non-profit research lab produced France's hottest new AI start-up", and Maddyness opened its coverage by asking whether a "mafia Kyutai" was emerging.[^85][^48] Debarghya Das's "The Ultimate Neolab List" (October 2026) includes Kyutai and rates it "Borderline", with the note: "Passes research/no-revenue/model tests, but a nonprofit with no venture equity."[^86] ## Recent updates (2025-2026) Through 2025 and 2026, the most consequential developments at Kyutai have been the steady expansion of the speech and audio stack, its first commercial spinout, and the start of work beyond audio: - **January 2025**: Helium-1 preview released as a 2-billion-parameter multilingual base model.[^6] - **February 2025**: Hibiki and Hibiki-M released for streaming French-to-English speech translation, with the corresponding arXiv paper 2502.03382.[^7][^15][^26] - **March 2025**: MoshiVis released, letting Moshi converse about images.[^42] - **April 2025**: Helium-1 full release and the "modular and multilingual LLM" technical write-up.[^14] - **May to July 2025**: Unmute introduced (22 May); Kyutai STT open-sourced (19 June); Kyutai TTS 1.6B and Unmute open-sourced (3 July).[^8][^43][^16] - **September 2025**: Delayed Streams Modeling and CALM papers published.[^45][^46] - **December 2025**: Gradium, founded by Zeghidour, Mazaré, Défossez and Olivier Teboul, launched with a $70 million seed round; Kyutai is a shareholder. CASA released.[^47][^48][^51] - **January 2026**: Pocket TTS released as a 100M-parameter text-to-speech model that runs faster than real time on a CPU.[^17] - **February 2026**: Hibiki-Zero released; Invincible Voice online demo opened.[^52][^54] - **April 2026**: OVIE, ARC-Encoder and MoshiRAG released.[^56][^57][^58] - **May 2026**: Pocket TTS extended to six languages (4 May), KE:SAI announced with the ELLIS Institute Tübingen (20 May), and Kairos released (26 May).[^10][^59][^62] - **June to July 2026**: reinforcement-learning post-training for full-duplex interactivity (10 June), Surflo (1 July), MIRA with General Intuition and Epic Games (July), and MuScriptor (10 July).[^63][^64][^65][^66] - **August to October 2026**: Pocket TTS training code (25 August), drifting-objective Pocket TTS (28 September), REKI (16 October), small Pocket TTS models (20 October) and CADRE (22 October); Kyutai's team page lists Grave among alumni, and Pérez's personal website describes him as having moved to Google DeepMind.[^67][^68][^70][^69][^71][^9][^75] ## Related work Kyutai's releases sit within a wider lineage of work on streaming neural audio codecs and speech-text [foundation models](https://aiwiki.ai/wiki/foundation_models), including SoundStream (the earlier 2021 [Google](https://aiwiki.ai/wiki/google) streaming codec, co-authored by Kyutai's later co-founder Zeghidour), EnCodec (Meta's 2022 streaming codec, co-authored by Kyutai's later co-founder Défossez), Meta's AudioCraft toolkit and [MusicGen](https://aiwiki.ai/wiki/musicgen) release, and [Microsoft](https://aiwiki.ai/wiki/microsoft)'s VALL-E (a neural codec language model for zero-shot TTS).[^87][^88][^21][^37][^39] In language modelling, Helium-1 belongs to the family of small language models that includes [Mistral 7B](https://aiwiki.ai/wiki/mistral_7b), Gemma 2B, and Llama 3.2 3B; comparisons against Gemma 2, Llama 3.2 and Qwen 2.5 are reported in the Helium-1 release notes.[^6][^23][^32] In streaming translation, Hibiki is best understood in comparison with Meta's Seamless model family and with prior cascaded approaches to simultaneous interpretation; Hibiki's distinguishing claim is the joint preservation of voice characteristics and timing alongside translation quality.[^7][^15][^29] ## See also - [List of neolabs](https://aiwiki.ai/wiki/list_of_neolabs) - [Neolab](https://aiwiki.ai/wiki/neolab) - [Gradium](https://aiwiki.ai/wiki/gradium) - [General Intuition](https://aiwiki.ai/wiki/general_intuition) - [Moshi](https://aiwiki.ai/wiki/moshi) - [Sesame](https://aiwiki.ai/wiki/sesame) - [GLM-4-Voice](https://aiwiki.ai/wiki/glm_4_voice) - [EnCodec](https://aiwiki.ai/wiki/encodec) - [SoundStream](https://aiwiki.ai/wiki/soundstream) - [VALL-E](https://aiwiki.ai/wiki/neural_codec_language_models_are_zero-shot_text_to_speech_synthesizers_vall-e) - [Whisper](https://aiwiki.ai/wiki/whisper) - [Mistral AI](https://aiwiki.ai/wiki/mistral_ai) - [Mistral 7B](https://aiwiki.ai/wiki/mistral_7b) - [Hugging Face](https://aiwiki.ai/wiki/hugging_face) - [Hugging Face Transformers](https://aiwiki.ai/wiki/transformers_library) - [Meta AI](https://aiwiki.ai/wiki/meta_ai) - [Google DeepMind](https://aiwiki.ai/wiki/google_deepmind) - [Gemma](https://aiwiki.ai/wiki/gemma) - [AudioCraft](https://aiwiki.ai/wiki/audiocraft) - [MusicGen](https://aiwiki.ai/wiki/musicgen) - [Large Language Model](https://aiwiki.ai/wiki/large_language_model) - [Small language model](https://aiwiki.ai/wiki/small_language_model) - [Foundation models](https://aiwiki.ai/wiki/foundation_models) - [Vision language model](https://aiwiki.ai/wiki/vision_language_model) - [Vision-language-action model](https://aiwiki.ai/wiki/vision_language_action_model) - [World model](https://aiwiki.ai/wiki/world_model) - [Physical AI](https://aiwiki.ai/wiki/physical_ai) - [Transformers](https://aiwiki.ai/wiki/transformers) - [Text-to-Speech](https://aiwiki.ai/wiki/text_to_speech_ai) - [Speech recognition](https://aiwiki.ai/wiki/speech_recognition) - [Machine translation](https://aiwiki.ai/wiki/machine_translation) - [Voice cloning](https://aiwiki.ai/wiki/voice_cloning) - [Knowledge Distillation](https://aiwiki.ai/wiki/knowledge_distillation) - [Model soups](https://aiwiki.ai/wiki/model_soups) - [GRPO](https://aiwiki.ai/wiki/grpo) - [Common Crawl](https://aiwiki.ai/wiki/common_crawl) - [JAX](https://aiwiki.ai/wiki/jax) - [PyTorch](https://aiwiki.ai/wiki/pytorch) - [MLX](https://aiwiki.ai/wiki/mlx) - [NVIDIA H100](https://aiwiki.ai/wiki/nvidia_h100) - [OpenAI Realtime API](https://aiwiki.ai/wiki/openai_realtime_api) - [GPT-4o](https://aiwiki.ai/wiki/gpt_4o) - [Yann LeCun](https://aiwiki.ai/wiki/yann_lecun) - [FAISS](https://aiwiki.ai/wiki/faiss) - [Edge AI](https://aiwiki.ai/wiki/edge_ai) - [Paris AI Action Summit](https://aiwiki.ai/wiki/paris_ai_action_summit) ## References [^1]: Romain Dillet, "Kyutai is a French AI research lab with a $330 million budget that will make everything open source", TechCrunch, 2023-11-17. https://techcrunch.com/2023/11/17/kyutai-is-an-french-ai-research-lab-with-a-330-million-budget-that-will-make-everything-open-source/. Accessed 2026-10-27. [^2]: Iliad Group, "Launch of Kyutai, Europe's first independent research lab dedicated to AI open science", iliad.fr, 2023-11-17. https://www.iliad.fr/en/actualites/article/launch-of-kyutai-europe-s-first-independent-research-lab-dedicated-to-ai-open-science. Accessed 2026-10-27. [^3]: Benoit Berthelot and Tara Patel (Bloomberg), "Billionaires including Eric Schmidt plow $300 million into a non-profit that is France's latest push to catch up in AI", Fortune, 2023-11-17. https://fortune.com/2023/11/17/ai-billionaires-xavier-niel-rodolphe-saade-eric-schmidt-france-lab/. Accessed 2026-10-27. [^4]: Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave and Neil Zeghidour, "Moshi: a speech-text foundation model for real-time dialogue", arXiv:2410.00037, 2024-09-17 (rev. 2024-10-02). https://arxiv.org/abs/2410.00037. Accessed 2026-10-27. [^5]: Kyutai Labs, "kyutai/mimi" model card, Hugging Face, 2024-09. https://huggingface.co/kyutai/mimi. Accessed 2026-05-20. [^6]: Kyutai Labs, "Announcing Helium-1 Preview", kyutai.org blog, 2025-01-13. https://kyutai.org/blog/2025-01-13-helium/. Accessed 2026-10-27. [^7]: Kyutai Labs, "Simultaneous, on-device, high fidelity speech-to-speech translation with Hibiki", kyutai.org blog, 2025-02-10. https://kyutai.org/blog/2025-02-10-hibiki. Accessed 2026-10-27. [^8]: Kyutai Labs, "Unmute: Make LLMs listen and speak", kyutai.org blog, 2025-05-22. https://kyutai.org/blog/2025-05-22-unmute/. Accessed 2026-10-27. [^9]: Kyutai Labs, "Team", kyutai.org/team. https://kyutai.org/team. Accessed 2026-10-27. [^10]: Kyutai Labs, "Pocket TTS now supports six languages", kyutai.org blog, 2026-05-04. https://kyutai.org/blog/2026-05-04-pocket-tts-multilingual. Accessed 2026-10-27. [^11]: Kyutai Labs, "kyutai: open-science AI lab", kyutai.org homepage. https://kyutai.org/. Accessed 2026-10-27. [^12]: Aurélie Barbaux, "Les grands modèles de langage au coeur de Kyutai, le labo en intelligence artificielle lancé par Xavier Niel", L'Usine Nouvelle, 2023-11-17. https://www.usinenouvelle.com/article/les-grands-modeles-de-langage-au-c-ur-de-kyutai-le-labo-en-intelligence-artificielle-lance-par-xavier-niel.N2196568. Accessed 2026-05-20. [^13]: Kyutai Labs, "Moshi" GitHub repository (kyutai-labs/moshi), README and model card, 2024-09. https://github.com/kyutai-labs/moshi. Accessed 2026-10-27. [^14]: Kyutai Labs, "Helium 1: a modular and multilingual LLM", kyutai.org blog, 2025-04-30. https://kyutai.org/blog/2025-04-30-helium/. Accessed 2026-10-27. [^15]: Asif Razzaq, "Kyutai Releases Hibiki: A 2.7B Real-Time Speech-to-Speech and Speech-to-Text Translation with Near-Human Quality and Voice Transfer", MarkTechPost, 2025-02-08. https://www.marktechpost.com/2025/02/08/kyutai-releases-hibiki-a-2-7b-real-time-speech-to-speech-and-speech-to-text-translation-with-near-human-quality-and-voice-transfer/. Accessed 2026-10-27. [^16]: Kyutai Labs, "Kyutai TTS and Unmute now open-source", kyutai.org blog, 2025-07-03. https://kyutai.org/blog/2025-07-03-tts-unmute-open-source. Accessed 2026-10-27. [^17]: Manu Orsini, Simon Rouard, Gabriel de Marmiesse, Václav Volhejn, Neil Zeghidour and Alexandre Défossez, "Pocket TTS: a high-quality TTS with voice cloning that runs on CPU", kyutai.org blog, 2026-01-13. https://kyutai.org/blog/2026-01-13-pocket-tts. Accessed 2026-10-27. [^18]: Kyutai Labs, "moshi" GitHub repository (kyutai-labs/moshi), Apache-2.0 license, accessed 2026-05-20. https://github.com/kyutai-labs/moshi. Accessed 2026-05-20. [^19]: Kyutai Labs, "kyutai-labs" GitHub organisation and repository list. https://github.com/kyutai-labs. Accessed 2026-10-27. [^20]: Hugging Face, "Mimi" documentation in the transformers library. https://huggingface.co/docs/transformers/main/en/model_doc/mimi. Accessed 2026-10-27. [^21]: Xin Zhang, Dong Zhang et al., "SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models", arXiv:2308.16692, 2023-08. https://arxiv.org/abs/2308.16692. Accessed 2026-05-20. [^22]: Asif Razzaq, "Kyutai Open Sources Moshi: A Breakthrough Full-Duplex Real-Time Dialogue System that Revolutionizes Human-like Conversations with Unmatched Latency and Speech Quality", MarkTechPost, 2024-09-18. https://www.marktechpost.com/2024/09/18/kyutai-open-sources-moshi-a-breakthrough-full-duplex-real-time-dialogue-system-that-revolutionizes-human-like-conversations-with-unmatched-latency-and-speech-quality/. Accessed 2026-10-27. [^23]: Asif Razzaq, "Kyutai Labs Releases Helium-1 Preview: A Lightweight Language Model with 2B Parameters, Targeting Edge and Mobile Devices", MarkTechPost, 2025-01-15. https://www.marktechpost.com/2025/01/15/kyutai-labs-releases-helium-1-preview-a-lightweight-language-model-with-2b-parameters-targeting-edge-and-mobile-devices/. Accessed 2026-10-27. [^24]: Kyutai Labs, "kyutai/helium-1-2b" model card (CC-BY-SA 4.0), Hugging Face, 2025. https://huggingface.co/kyutai/helium-1-2b. Accessed 2026-10-27. [^25]: Kyutai Labs, "kyutai/CASA-Helium1-VL-2B" model card, Hugging Face, 2025-2026. https://huggingface.co/kyutai/CASA-Helium1-VL-2B. Accessed 2026-10-27. [^26]: Tom Labiausse, Laurent Mazaré, Edouard Grave, Patrick Pérez, Alexandre Défossez and Neil Zeghidour, "High-Fidelity Simultaneous Speech-to-Speech Translation", arXiv:2502.03382, 2025-02-05. https://arxiv.org/abs/2502.03382. Accessed 2026-10-27. [^27]: Kyutai Labs, "hibiki" GitHub repository (kyutai-labs/hibiki), 2025. https://github.com/kyutai-labs/hibiki. Accessed 2026-05-20. [^28]: Jean-Jerome Levy, "Why Moshi STT Could Replace Whisper (and How to Install It on macOS!)", Scalastic, 2025-06-25. https://scalastic.io/en/moshi-stt-vs-whisper/. Accessed 2026-10-27. [^29]: Maria Stasimioti, "Makers of Open Source Model Hibiki Promise 'High-Fidelity' Speech Translation", Slator, 2025-02-25. https://slator.com/makers-of-open-source-model-hibiki-promise-high-fidelity-speech-translation/. Accessed 2026-10-27. [^30]: Václav Volhejn, "Unmute.sh, the most modular voice AI around", vvolhejn.com, 2025-07-03. https://vvolhejn.com/blog/2025-07-03-unmute. Accessed 2026-10-27. [^31]: Roberta T., "Kyutai: Europe's Pioneering AI Open-Science Lab Launched in Paris", Move It Magazine, 2023-11-20. https://moveitmagazine.com/2023/11/20/kyutai-europes-pioneering-ai-open-science-lab-launched-in-paris/. Accessed 2026-10-27. [^32]: Mistral AI, "About" and company information, mistral.ai, accessed 2026-05-20. https://mistral.ai/. Accessed 2026-05-20. [^33]: Hugging Face, "About" page, huggingface.co/about, accessed 2026-05-20. https://huggingface.co/about. Accessed 2026-05-20. [^34]: Meta AI, "About FAIR" page, ai.meta.com, accessed 2026-05-20. https://ai.meta.com/research/. Accessed 2026-05-20. [^35]: Google DeepMind, "About Google DeepMind", deepmind.google, accessed 2026-05-20. https://deepmind.google/about/. Accessed 2026-05-20. [^36]: Anna Heim, "Beyond Lovable and Mistral: 21 European startups to watch", TechCrunch, 2026-05-02. https://techcrunch.com/2026/05/02/beyond-lovable-and-mistral-21-european-startups-to-watch/. Accessed 2026-10-27. [^37]: Chengyi Wang et al., "Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (VALL-E)", arXiv:2301.02111, 2023-01. https://arxiv.org/abs/2301.02111. Accessed 2026-05-20. [^38]: Kyutai, "Hello Kyutai!", kyutai.org blog, 2023-11-17. https://kyutai.org/blog/2023-11-17-hello-kyutai/. Accessed 2026-10-27. [^39]: Kyutai, "Launch of Kyutai: Europe's first independent research lab dedicated to AI open science, co-founded by the iliad Group, CMA CGM and Schmidt Futures" (press release, PDF), 2023-11-17. https://kyutai.org/CP_Kyutai_AI_EN.pdf. Accessed 2026-10-27. [^40]: Kyutai, "Meet Moshi, the first real-time voice AI", kyutai.org blog, 2024-07-03. https://kyutai.org/blog/2024-07-03-meet-moshi/. Accessed 2026-10-27. [^41]: Kyutai, "Moshi open-source release: run Moshi locally!", kyutai.org blog, 2024-09-18. https://kyutai.org/blog/2024-09-18-moshi-release/. Accessed 2026-10-27. [^42]: Amélie Royer, Moritz Böhle, Gabriel de Marmiesse, Laurent Mazaré, Neil Zeghidour, Alexandre Défossez and Patrick Pérez, "MoshiVis: Teaching Moshi to Converse about Images", kyutai.org, 2025-03-21. https://kyutai.org/moshivis/. Accessed 2026-10-27. [^43]: Kyutai, "Kyutai Speech-To-Text released as open-source", kyutai.org blog, 2025-06-19. https://kyutai.org/blog/2025-06-19-stt-open-source/. Accessed 2026-10-27. [^44]: Kyutai, "Kyutai TTS 1.6B", kyutai.org blog, 2025-07-03. https://kyutai.org/blog/2025-07-03-kyutai-tts-1-6b/. Accessed 2026-10-27. [^45]: Neil Zeghidour, Eugene Kharitonov, Manu Orsini, Václav Volhejn, Gabriel de Marmiesse, Edouard Grave, Patrick Pérez, Laurent Mazaré and Alexandre Défossez, "Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling", arXiv:2509.08753, 2025-09-10. https://arxiv.org/abs/2509.08753. Accessed 2026-10-27. [^46]: Simon Rouard, Manu Orsini, Axel Roebel, Neil Zeghidour and Alexandre Défossez, "Continuous Audio Language Models", arXiv:2509.06926, 2025-09-08 (rev. 2026-01-13). https://arxiv.org/abs/2509.06926. Accessed 2026-10-27. [^47]: Julie Bort, "Paris-based AI voice startup Gradium nabs $70M seed", TechCrunch, 2025-12-02. https://techcrunch.com/2025/12/02/paris-based-ai-voice-startup-gradium-nabs-70m-seed/. Accessed 2026-10-27. [^48]: Maxence Fabrion, "Gradium, premier spin-off de Kyutai, lève 60 millions d'euros pour devenir une référence mondiale de l'IA vocale", Maddyness, 2025-12-02. https://www.maddyness.com/2025/12/02/gradium-premier-spin-off-de-kyutai-leve-60-millions-deuros-pour-devenir-une-reference-mondiale-de-lia-vocale/. Accessed 2026-10-27. [^49]: Gradium, "Gradium: Solving voice", gradium.ai blog, 2025-12-02. https://gradium.ai/blog/gradium. Accessed 2026-10-27. [^50]: Gradium, "Gradium Extends Funding to $100 Million and Expands to Silicon Valley", gradium.ai blog, 2026-07-08. https://gradium.ai/blog/gradium-100-million-funding-nvidia. Accessed 2026-10-27. [^51]: Kyutai, "CASA: Cross-Attention over Self-Attention" project page, kyutai.org, December 2025. https://kyutai.org/casa/. Accessed 2026-10-27. [^52]: Kyutai, "Hibiki-Zero: Simultaneous Speech-to-Speech Translation Without Aligned Data", kyutai.org blog, 2026-02-12. https://kyutai.org/blog/2026-02-12-hibiki-zero/. Accessed 2026-10-27. [^53]: Tom Labiausse, Romain Fabre, Yannick Estève, Alexandre Défossez and Neil Zeghidour, "Simultaneous Speech-to-Speech Translation Without Aligned Data", arXiv:2602.11072, 2026-02-11. https://arxiv.org/abs/2602.11072. Accessed 2026-10-27. [^54]: Kyutai, "Invincible Voice online demo released", kyutai.org blog, 2026-02-24. https://kyutai.org/blog/2026-02-24-invincible-voice/. Accessed 2026-10-27. [^55]: Gradium, "Invincible Voice: How Gradium's Real-Time Voice AI Helps ALS Patients Speak Again", gradium.ai blog, 2026-01-28. https://gradium.ai/blog/invincible-voice. Accessed 2026-10-27. [^56]: Kyutai, "OVIE: One View Is Enough", kyutai.org blog, 2026-04-14. https://kyutai.org/blog/2026-04-14-ovie/. Accessed 2026-10-27. [^57]: Kyutai, "ARC-Encoder: learning compressed text representations for LLMs", kyutai.org blog, 2026-04-28. https://kyutai.org/blog/2026-04-28-arc-encoder/. Accessed 2026-10-27. [^58]: Kyutai, "MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models", kyutai.org blog, 2026-04-30. https://kyutai.org/blog/2026-04-30-moshi-rag/. Accessed 2026-10-27. [^59]: Kyutai, "Introducing KE:SAI", kyutai.org blog, 2026-05-20. https://kyutai.org/blog/2026-05-20-kesai-launch/. Accessed 2026-10-27. [^60]: Kyutai and ELLIS Institute Tübingen, "Kyutai and ELLIS Tübingen launch KE:SAI: A Premier Franco-German Partnership for Open Science in Physical AI" (press release, PDF), 2026-05-20. https://kyutai.org/assets/blog/20260520-kesai/2026-05-20-kesai-press-release-en-de-fr.pdf. Accessed 2026-10-27. [^61]: University of Tübingen, "Franco-German partnership for open science in physical AI", uni-tuebingen.de, 2026-05-20. https://uni-tuebingen.de/en/university/news-and-publications/news/franco-german-partnership-for-open-science-in-physical-ai/. Accessed 2026-10-27. [^62]: Kyutai, "Kairos: Understanding Data Temporality Impact on LLM pre-training", kyutai.org blog, 2026-05-26. https://kyutai.org/blog/2026-05-26-kairos/. Accessed 2026-10-27. [^63]: Kyutai, "Post-training speech models for better interactivity", kyutai.org blog, 2026-06-10. https://kyutai.org/blog/2026-06-10-interactivity/. Accessed 2026-10-27. [^64]: Kyutai, "Surflo: Consistent 3D surfaces from a global state", kyutai.org blog, 2026-07-01. https://kyutai.org/blog/2026-07-01-surflo/. Accessed 2026-10-27. [^65]: General Intuition and Kyutai, in collaboration with Epic Games, "MIRA" (project blog post), mira-wm.com, July 2026. https://mira-wm.com/blog-post/. Accessed 2026-10-27. [^66]: Kyutai, "MuScriptor: Automatic Multi-instrument Transcription", kyutai.org blog, 2026-07-10. https://kyutai.org/blog/2026-07-10-muscriptor/. Accessed 2026-10-27. [^67]: Manu Orsini and Gabriel de Marmiesse, "Pocket TTS training code released", kyutai.org blog, 2026-08-25. https://kyutai.org/blog/2026-08-25-pocket-tts-training/. Accessed 2026-10-27. [^68]: Kyutai, "Building a Pocket TTS with a drifting objective", kyutai.org blog, 2026-09-28. https://kyutai.org/blog/2026-09-28-pocket-tts-drifting/. Accessed 2026-10-27. [^69]: Kyutai, "The tiny TTS was actually 10x too big", kyutai.org blog, 2026-10-20. https://kyutai.org/blog/2026-10-20-pocket-tts-small/. Accessed 2026-10-27. [^70]: Juliette Marrie, Paul Pacaud, Adrien Ramanana Rahary, Shizhe Chen and Moritz Böhle, "REKI: Registering Knowledge in VLAs", kyutai.org, October 2026. https://kyutai.org/reki/. Accessed 2026-10-27. [^71]: Adrien Ramanana Rahary, Yutong Bai, Nicolas Dufour, Patrick Pérez and David Picard, "CADRE: Rethinking Video Autoencoders for Interactive World Models", kyutai.org blog, 2026-10-22. https://kyutai.org/blog/2026-10-22-cadre/. Accessed 2026-10-27. [^72]: Kyutai, "Papers" (list of publications). https://kyutai.org/papers/. Accessed 2026-10-27. [^73]: Kyutai Labs, "pocket-tts" GitHub repository README. https://github.com/kyutai-labs/pocket-tts. Accessed 2026-10-27. [^74]: Kyutai, "kyutai/pocket-tts" model page (gated, with prohibited-use terms) and "kyutai/pocket-tts-without-voice-cloning", Hugging Face. https://huggingface.co/kyutai/pocket-tts. Accessed 2026-10-27. [^75]: Patrick Pérez, personal website. https://ptrckprz.github.io/. Accessed 2026-10-27. [^76]: Patrick Pérez, personal website, Internet Archive snapshot of 2026-04-20. https://web.archive.org/web/20260420160305/https://ptrckprz.github.io/. Accessed 2026-10-27. [^77]: Kyutai Labs, "Team", Internet Archive snapshot of 2026-06-16. https://web.archive.org/web/20260616032919/https://kyutai.org/team. Accessed 2026-10-27. [^78]: Kyutai Labs, "Team", Internet Archive snapshot of 2026-08-09. https://web.archive.org/web/20260809125621/https://kyutai.org/team/. Accessed 2026-10-27. [^79]: NVIDIA, "personaplex" GitHub repository README. https://github.com/NVIDIA/personaplex. Accessed 2026-10-27. [^80]: Sesame, "csm" GitHub repository README. https://github.com/SesameAILabs/csm. Accessed 2026-10-27. [^81]: Kyutai, "kyutai/hibiki-zero-3b-pytorch-bf16" model card, Hugging Face, 2026. https://huggingface.co/kyutai/hibiki-zero-3b-pytorch-bf16. Accessed 2026-10-27. [^82]: Kyutai, "kyutai/hibiki-2b-pytorch-bf16" model card, Hugging Face, 2025. https://huggingface.co/kyutai/hibiki-2b-pytorch-bf16. Accessed 2026-10-27. [^83]: Kyutai, "kyutai/glm-4-voice-of-reason-9b" model page, Hugging Face, 2026. https://huggingface.co/kyutai/glm-4-voice-of-reason-9b. Accessed 2026-10-27. [^84]: Kyutai, "Blog" (index of posts). https://kyutai.org/blog/. Accessed 2026-10-27. [^85]: Martin Greenacre, "How a non-profit research lab produced France's hottest new AI start-up", Science|Business, 2025-12-09. https://sciencebusiness.net/news/r-d-funding/ai/how-non-profit-research-lab-produced-frances-hottest-new-ai-start. Accessed 2026-10-27. [^86]: Debarghya Das, "The Ultimate Neolab List", debarghyadas.com. https://debarghyadas.com/neolabs/. Accessed 2026-10-27. [^87]: Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund and Marco Tagliasacchi, "SoundStream: An End-to-End Neural Audio Codec", arXiv:2107.03312, 2021-07-07. https://arxiv.org/abs/2107.03312. Accessed 2026-10-27. [^88]: Alexandre Défossez, Jade Copet, Gabriel Synnaeve and Yossi Adi, "High Fidelity Neural Audio Compression" (EnCodec), arXiv:2210.13438, 2022-10-24. https://arxiv.org/abs/2210.13438. Accessed 2026-10-27. [^89]: Patrick Pérez, personal website, Internet Archive snapshot of 2026-10-06. https://web.archive.org/web/20261006162809/https://ptrckprz.github.io/. Accessed 2026-10-27. [^90]: Kyutai, "kyutai/helium-1-preview-2b" model card (CC-BY 4.0), Hugging Face, 2025. https://huggingface.co/kyutai/helium-1-preview-2b. Accessed 2026-10-27. [^91]: Kyutai Labs, "kyutai-labs/reki" GitHub repository, README (License section: code MIT, weights CC-BY-NC-SA 4.0), 2026. https://github.com/kyutai-labs/reki. Accessed 2026-10-27.