AI Wiki

Voice assistant

AI Tools & ProductsConversational AISpeech & Audio AIVoice AI
18 min read
Updated
Suggest editHistoryTalk39 citationsFact-checked
RawGraph

A voice assistant is a software agent whose primary interface is spoken language: it listens for a trigger, converts speech to a machine-readable request, decides what the speaker wants, and answers with synthesized speech or by carrying out an action such as setting a timer, playing music, or switching off a lamp. The category includes phone assistants such as Siri and Google Assistant, the smart home speakers built around Alexa, the voice modes of chat products, and the automated agents that now answer customer service calls.

The installed base is large by any measure. Amazon states that there are over 600 million Alexa devices in use [24]. Apple said in January 2018 that Siri was actively used on over half a billion devices [12], and Google said in January 2020 that its Assistant helped more than 500 million people a month across more than a billion devices in over 90 countries [30].

What changed between 2023 and 2026 is not the interface but the machinery behind it. For a decade, assistants ran a fixed chain of specialized components: wake word detection, speech recognition, natural language understanding, dialogue management, and text to speech. Large language models, and then models that consume and produce audio directly, collapsed much of that chain. All three of the major consumer assistants were rebuilt on that basis: Amazon announced Alexa+ in February 2025 [24], Google began replacing Assistant with Gemini in March 2025 [25], and Apple unveiled a rebuilt Siri in June 2026 [27].

Before the assistants

Machine recognition of speech is much older than the assistant products. In 1952, Davis, Biddulph, and Balashek at Bell Laboratories built a system for isolated digit recognition for a single speaker, using the formant frequencies measured during the vowel regions of each digit [1][2]. Work of that kind, aimed at small vocabularies of carefully separated words, continued for two decades.

Scale came with government funding. DARPA, then called ARPA, paid for the Speech Understanding Research program in the early 1970s, which produced Carnegie Mellon's Harpy, which recognized speech from a vocabulary of 1,011 words by searching a connected network derived from lexical, syntactic, and word boundary rules, along with CMU's Hearsay-II and BBN's HWIM; the program concluded in 1976 [1]. Statistical modeling in the 1980s, above all hidden Markov models, made large-vocabulary continuous recognition practical [1].

Commercially, speech reached consumers first over the telephone. Interactive voice response systems automated call handling with recorded prompts, touch-tone input, and eventually constrained speech recognition. The W3C published VoiceXML 2.0 as a Recommendation on 16 March 2004, defining markup for audio dialogs featuring synthesized speech, digitized audio, recognition of spoken and DTMF key input, and mixed initiative conversation, with the explicit goal of bringing web development practices to interactive voice response applications [3]. These systems were narrow by design: they worked because the set of things a caller could usefully say was small.

The assistant era, 2011 to 2018

Siri came out of the same funding stream. SRI International developed the technology in the Cognitive Assistant that Learns and Organizes (CALO) project inside DARPA's Personalized Assistant that Learns program, spun off Siri, Inc. in 2007 with $24 million raised across two rounds, and sold the company to Apple in April 2010 [4]. Apple introduced Siri with the iPhone 4S on 4 October 2011, describing it as an intelligent assistant that helps you get things done just by asking, shipping it in beta on 14 October in English for the United States, United Kingdom, and Australia plus French and German [5].

Google's first mass-market effort, Google Now, arrived with Android 4.1 in 2012 and worked mainly by pushing predictive cards about flights, traffic, and appointments rather than by holding a conversation [6]. Google Assistant replaced that model with two-way dialogue and was paired with hardware: on 4 October 2016 Sundar Pichai introduced Google Home as a voice-activated speaker powered by the Assistant, which had first appeared in the Allo messaging app [11].

Microsoft shipped Cortana with Windows Phone 8.1, unveiling it on 2 April 2014 and naming it after the artificial intelligence character from the Halo games [8]. It was wound down in stages. Microsoft retired the standalone Cortana app in Windows during 2023, ended Cortana support in Teams and Microsoft 365 mobile apps that autumn, removed the last Cortana features from Outlook mobile in June 2024, and now directs users to Copilot and to voice access in Windows 11 [7].

Amazon took the opposite bet, putting the assistant in a dedicated appliance. The first Echo was announced on 6 November 2014 at $199, or $99 for Prime members, initially by invitation only, with seven microphones and beam-forming to pick a voice out of a room and the wake word "Alexa" [9]. Samsung followed on 20 March 2017 with Bixby, which its head of R&D described in terms of three properties: completeness (a Bixby-enabled app should support by voice nearly everything it supports by touch), context awareness (the assistant should understand the current state of the app), and cognitive tolerance (it should act on incomplete commands and then ask for the missing pieces), backed by a dedicated hardware button on the Galaxy S8 [10]. Apple entered the speaker market late, with HomePod at $349 on 9 February 2018 [12].

AssistantFirst shownFirst platformNotable detail
Siri4 October 2011 [5]iPhone 4SSpun out of SRI in 2007, bought by Apple in April 2010 [4]
Google Now2012 [6]Android 4.1Predictive cards rather than dialogue [6]
Cortana2 April 2014 [8]Windows Phone 8.1Named after the Halo character [8]
Alexa6 November 2014 [9]Amazon Echo$199, invitation only, seven-microphone array [9]
Google Assistant2016 [11]Allo and Google HomeSpeaker announced 4 October 2016 [11]
Bixby20 March 2017 [10]Galaxy S8Dedicated side button [10]

How the classic pipeline works

Wake word detection runs locally and continuously, which makes it the most power-constrained part of the system. Apple has published the design used for "Hey Siri": a deep neural network with five hidden layers converts each acoustic frame into a probability distribution over roughly 20 sound classes, and dynamic programming accumulates those scores across frames. On iPhone the detector runs in two passes, a small network of 32 units per layer on the always-on processor and a larger 192-unit network on the main processor once the first pass crosses a threshold. Apple describes a second, lower threshold that puts the phone into a more sensitive state for a few seconds so a user who was not heard can simply repeat the phrase, and it tunes the tradeoff between false accepts and false rejects toward roughly one spurious activation per week [13].

After the trigger, audio has traditionally been streamed to a server for recognition. Accuracy on conversational speech improved sharply with deep learning: Microsoft reported a 5.1 percent word error rate on the Switchboard task in August 2017, down from the 5.9 percent it had claimed as human parity a year earlier [14]. The trend continued with large weakly supervised models such as Whisper, trained on 680,000 hours of multilingual audio and designed to work zero-shot without dataset-specific fine-tuning [15].

Understanding was handled separately. Alexa's developer model is representative: a skill declares intents (the actions it can perform), sample utterances (phrasings that map to an intent), and slots (the variable arguments inside an utterance, such as the origin and destination cities in a travel request) [16]. A dialogue manager tracked which slots were still missing, asked for them, and called the fulfillment code.

Output came from a speech synthesizer. Neural synthesis reached production in 2017, when DeepMind reported that a redesigned WaveNet was generating the US English and Japanese Assistant voices: a thousand times faster than the research model, taking 50 milliseconds to create one second of speech at 24,000 samples per second and 16 bits per sample, and scoring 4.347 on a five-point mean opinion scale against 4.667 for recorded human speech [17].

StageJobTypical failure
Wake wordDetect the trigger phrase on device, at low powerFalse activations and missed triggers [13]
Speech recognitionTurn audio into textErrors on accents, noise, and rare names [14][36]
UnderstandingMap text to an intent and its slotsRequests outside the declared intent set [16]
Dialogue and fulfillmentTrack state, call the service, decide the replyLost context between turns
Speech synthesisTurn the reply into audioFlat prosody, wrong emphasis [17]

The rebuild on language models, 2023 to 2026

OpenAI shipped voice conversation in ChatGPT on 25 September 2023 by chaining components: Whisper transcribed the user, the language model produced text, and a new text-to-speech model, able to generate human-like audio from text plus a few seconds of sample speech, spoke the reply in one of five voices built with professional voice actors. It went to Plus and Enterprise users on iOS and Android over the following two weeks [18]. Building a usable voice from seconds of sample audio is the same capability that underlies voice cloning, and the arrangement inherits the weaknesses of any cascade.

OpenAI documented those weaknesses when it announced GPT-4o on 13 May 2024. The old pipeline averaged 2.8 seconds of latency with GPT-3.5 and 5.4 seconds with GPT-4, and the text bottleneck discarded information: the model could not directly observe tone, multiple speakers, or background noise, and could not output laughter, singing, or emotion. GPT-4o was trained end to end across text, vision, and audio in a single network, responding to audio in as little as 232 milliseconds and 320 milliseconds on average, which is the range of human conversational turn-taking [19]. That model became the basis of ChatGPT's Advanced Voice Mode.

Developers got the same capability through the Realtime API, released as a public beta on 1 October 2024 with six preset voices [20] and made generally available on 28 August 2025 alongside gpt-realtime, which added remote MCP server support, image input, and telephone calling over SIP, plus two new voices [21]. Research systems pushed in the same direction: Kyutai's Moshi, released in 2024, models user and system speech as parallel streams for full-duplex conversation with interruptions and overlapping speech, at a theoretical latency of 160 milliseconds and about 200 milliseconds in practice [22]. Amazon's Nova Sonic, announced on 8 April 2025, unified speech understanding and generation in one model on Amazon Bedrock, explicitly to replace the orchestration of separate recognition, language, and synthesis models [23]. Google's Project Astra prototype, shown on 14 May 2024, framed the same goal for an assistant that takes in video and audio continuously and remembers what it has seen and heard [37].

The consumer assistants followed. Amazon announced Alexa+ on 26 February 2025, built on large language models available through Bedrock and organized around what it calls experts, groups of systems, APIs, and instructions that handle specific classes of task; it costs $19.99 a month and is included with Prime. By mid-2026 Amazon listed it as available to all customers in the United States and Canada, with early access in eight further countries [24]. Google said on 14 March 2025 that it was upgrading mobile users from Assistant to Gemini and that the classic Assistant would stop being available on most mobile devices later that year, with tablets, cars, watches, headphones, and home devices to follow [25]. On 1 October 2025 it launched Gemini for Home, which replaces Assistant on speakers and displays, upgrades cameras and doorbells, and sits behind a Google Home Premium subscription bundled with Google AI Pro and Ultra [26].

Apple moved last, and with outside help. On 12 January 2026 Apple and Google jointly confirmed a non-exclusive, multi-year partnership under which Google's Gemini models and cloud technology would underpin Apple Foundation Models, including Siri features; reports put the cost at roughly $1 billion a year [28]. On 8 June 2026 Apple introduced Siri AI, describing an assistant with personal context understanding, broad world knowledge, and onscreen awareness that can take actions inside apps. Apple credits its own on-device models and Private Cloud Compute servers for the processing, with developer testing from June, a user beta later in 2026 in English first, and availability tied to recent hardware; it is not offered at launch in China, or in the European Union on iPhone, iPad, and Apple Watch [27]. The rebuilt Siri arrives alongside Apple Intelligence, which shipped on 28 October 2024 with a redesigned Siri interface, a glowing light wrapped around the edge of the screen [38], and added ChatGPT access inside Siri and Writing Tools that December [39].

Adoption and business model

Adoption ran ahead of the hardware category. Pew Research Center found in 2017 that 46 percent of US adults used voice assistants, 42 percent on smartphones and only 8 percent on a standalone device such as an Echo or Google Home, with hands-free operation the most cited reason [29]. The 2022 Smart Audio Report from NPR and Edison Research put voice assistant use at 62 percent of Americans aged 18 and over across all device types, with 57 percent of voice command users using them daily [31].

The commercial model has shifted. For the first decade the assistant came free with the phone or the speaker; the current generation is sold as a subscription, either directly (Alexa+ at $19.99 a month, free for Prime members [24]) or as part of a broader plan (Google Home Premium [26]). Voice is also a growing enterprise category, where the same speech-to-speech models power voice agents for support calls, a use case OpenAI addressed directly by adding SIP telephony and function calling reliability to its production API [21].

Privacy and oversight

The design of a voice assistant requires a microphone that is always on. Vendors answer that concern by keeping wake word detection local and streaming only after the trigger [13], but the arrangement depends on trust, and the record has been uneven.

In 2019 the human side of quality review became public. Apple suspended its grading program and said on 28 August 2019 that it would stop retaining audio by default, use recordings only with explicit opt-in, allow users to withdraw consent, and restrict listening to Apple employees, adding that it had not been living up to its own ideals and apologizing [32]. Google, after a language reviewer leaked Dutch audio snippets, said on 11 July 2019 that language experts review around 0.2 percent of audio snippets and that reviewers are instructed to transcribe only queries directed at Google [33]. Bloomberg reported that Amazon employees listened to as many as 1,000 audio samples during nine-hour shifts, work Amazon has described as training for its recognition and understanding systems [35].

Enforcement followed. On 31 May 2023 the FTC and the Department of Justice required Amazon to pay a $25 million civil penalty and overhaul its practices to settle charges under the Children's Online Privacy Protection Act, alleging that Amazon kept children's voice recordings indefinitely, failed to honor parents' deletion requests, and used retained data to improve its algorithms; the proposed order also barred using data subject to deletion requests for product improvement [34].

The move to cloud-hosted language models reversed one privacy option. In March 2025, before the Alexa+ rollout, Amazon told customers it would stop supporting the "Do Not Send Voice Recordings" setting, which had let owners of the Echo (4th generation), Echo Show 10, and Echo Show 15 keep processing local. From 28 March, every request spoken to those devices is sent to Amazon's cloud, a change Amazon attributed to generative AI features that rely on its servers [35]. Apple's counter-positioning is architectural: it says Siri AI requests handled by Private Cloud Compute leave personal data inaccessible to Apple or anyone else, with core capabilities running on device [27].

Limitations

Recognition quality is not evenly distributed. A 2020 study in PNAS tested five commercial systems from Amazon, Apple, Google, IBM, and Microsoft on interviews with 42 white and 73 black speakers, 19.8 hours of matched audio in total, and found an average word error rate of 0.35 for black speakers against 0.19 for white speakers; the authors traced the gap to the acoustic models, since it persisted on identical phrases, and recommended more representative training data [36].

Reliability was a complaint from the beginning: in Pew's 2017 survey, 16 percent of users said their assistant responded accurately not very often, and only 39 percent said it did most of the time [29]. Language models change the shape of the problem rather than removing it. An assistant that can answer anything can also hallucinate an answer, and errors are harder to spot when the reply is spoken once rather than displayed. The economics also changed: answering a request with a large model costs more than matching an utterance against a fixed intent list, and the 2025 and 2026 generation arrived attached to subscriptions and, in Apple's case, to hardware requirements that leave older devices out [24][26][27].

See also

References

  1. ^Juang, B. H. and Rabiner, L. R. "Automatic Speech Recognition: A Brief History of the Technology Development" (2004). web.ece.ucsb.edu/...LALI-ASRHistory-final-10-8.pdf
  2. ^Davis, K. H., Biddulph, R. and Balashek, S. "Automatic Recognition of Spoken Digits," Journal of the Acoustical Society of America, Vol. 24, No. 6 (1952). pubs.aip.org/...matic-Recognition-of-Spoken-Digits
  3. ^W3C. "Voice Extensible Markup Language (VoiceXML) Version 2.0," W3C Recommendation, 16 March 2004. w3.org/...voicexml20
  4. ^SRI International. "Siri" (History of Innovation). sri.com/...siri
  5. ^Apple. "Apple Launches iPhone 4S, iOS 5 & iCloud," 4 October 2011. apple.com/...04Apple-Launches-iPhone-4S-iOS-5-iCloud
  6. ^Wikipedia. "Google Now." en.wikipedia.org/...Google_Now
  7. ^Microsoft Support. "End of support for Cortana." support.microsoft.com/...5b-4836-a954-0ab646ee1efa
  8. ^Warren, Tom. "The story of Cortana, Microsoft's Siri killer," The Verge, 2 April 2014. theverge.com/...indows-phone-8-1-digital-assistant
  9. ^Etherington, Darrell. "Amazon Echo Is A $199 Connected Speaker Packing An Always-On Siri-Style Assistant," TechCrunch, 6 November 2014. techcrunch.com/...amazon-echo
  10. ^Rhee, Injong. "Bixby: A New Way to Interact with Your Phone," Samsung Newsroom, 20 March 2017. news.samsung.com/...ay-to-interact-with-your-phone
  11. ^Pichai, Sundar. "A personal Google, just for you," Google blog, 4 October 2016. blog.google/...personal-google-just-you
  12. ^Apple. "HomePod arrives February 9, available to order this Friday," 23 January 2018. apple.com/...uary-9-available-to-order-this-friday
  13. ^Apple Machine Learning Research. "Hey Siri: An On-device DNN-powered Voice Trigger for Apple's Personal Assistant," October 2017. machinelearning.apple.com/...hey-siri
  14. ^Microsoft Research Blog. "Microsoft researchers achieve new conversational speech recognition milestone," 20 August 2017. microsoft.com/...onal-speech-recognition-milestone
  15. ^Radford, A. et al. "Robust Speech Recognition via Large-Scale Weak Supervision," arXiv:2212.04356, December 2022. arxiv.org/...2212.04356
  16. ^Amazon. "Create Intents, Utterances, and Slots," Alexa Skills Kit documentation. developer.amazon.com/...tents-utterances-and-slots
  17. ^DeepMind. "WaveNet launches in the Google Assistant," 4 October 2017. deepmind.google/...aunches-in-the-google-assistant
  18. ^OpenAI. "ChatGPT can now see, hear, and speak," 25 September 2023. openai.com/...chatgpt-can-now-see-hear-and-speak
  19. ^OpenAI. "Hello GPT-4o," 13 May 2024. openai.com/...hello-gpt-4o
  20. ^OpenAI. "Introducing the Realtime API," 1 October 2024. openai.com/...introducing-the-realtime-api
  21. ^OpenAI. "Introducing gpt-realtime and Realtime API updates for production voice agents," 28 August 2025. openai.com/...introducing-gpt-realtime
  22. ^Defossez, A. et al. "Moshi: a speech-text foundation model for real-time dialogue," arXiv:2410.00037, 2024. arxiv.org/...2410.00037
  23. ^AWS News Blog. "Introducing Amazon Nova Sonic: Human-like voice conversations for generative AI applications," 8 April 2025. aws.amazon.com/...s-for-generative-ai-applications
  24. ^Amazon. "Introducing Alexa+, the next generation of Alexa," 26 February 2025 (updated 21 July 2026). aboutamazon.com/...erative-artificial-intelligence
  25. ^Google. "Google Assistant is upgrading to Gemini on mobile devices," 14 March 2025. blog.google/...google-assistant-gemini-mobile
  26. ^Google. "The next era of Gemini for Home," 1 October 2025. blog.google/...next-era-gemini-google-home-launch
  27. ^Apple. "Apple introduces Siri AI, a profoundly more capable and personal assistant," 8 June 2026. apple.com/...y-more-capable-and-personal-assistant
  28. ^Bellan, Rebecca. "Google's Gemini to power Apple's AI features like Siri," TechCrunch, 12 January 2026. techcrunch.com/...wer-apples-ai-features-like-siri
  29. ^Olmstead, Kenneth. "Nearly half of Americans use digital voice assistants, mostly on their smartphones," Pew Research Center, 12 December 2017. pewresearch.org/...nts-mostly-on-their-smartphones
  30. ^Google. "Hey Google, what's new? Assistant updates from CES 2020," 7 January 2020. blog.google/...ces-2020-google-assistant
  31. ^NPR and Edison Research. "The Smart Audio Report," June 2022. nationalpublicmedia.com/...smart-audio-report
  32. ^Apple. "Improving Siri's privacy protections," 28 August 2019. apple.com/...improving-siris-privacy-protections
  33. ^Monsees, David. "More information about our processes to safeguard speech data," Google blog, 11 July 2019. blog.google/...our-processes-safeguard-speech-data
  34. ^Federal Trade Commission. "FTC and DOJ Charge Amazon with Violating Children's Privacy Law by Keeping Kids' Alexa Voice Recordings Forever and Undermining Parents' Deletion Requests," 31 May 2023. ftc.gov/...ing-kids-alexa-voice-recordings-forever
  35. ^Harding, Scharon. "Everything you say to your Echo will be sent to Amazon starting on March 28," Ars Technica, 14 March 2025. arstechnica.com/...-to-amazon-starting-on-march-28
  36. ^Koenecke, A. et al. "Racial disparities in automated speech recognition," Proceedings of the National Academy of Sciences, 2020. pmc.ncbi.nlm.nih.gov/...PMC7149386
  37. ^Google. "Google I/O 2024: An I/O for a new generation," 14 May 2024. blog.google/...i-update-flash-ai-assistant-io-2024
  38. ^Apple. "Apple Intelligence is available today on iPhone, iPad, and Mac," 28 October 2024. apple.com/...vailable-today-on-iphone-ipad-and-mac
  39. ^Apple. "Apple Intelligence now features Image Playground, Genmoji, Writing Tools enhancements, seamless support for ChatGPT, and visual intelligence," 11 December 2024. apple.com/...res-image-playground-genmoji-and-more

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.

v1 · 3,541 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent adversarial fact-check at creation (wanted175 campaign, 2026-07-24): every claim verified against primary sources by a dedicated verification agent; corrections applied before publication.

Cite this page: AI Wiki. "Voice assistant." aiwiki.ai, updated 24 Jul 2026, fact-checked 24 Jul 2026. CC BY 4.0. https://aiwiki.ai/wiki/voice_assistant

Suggest edit