Youth AI Safety Institute
Last reviewed
Jun 3, 2026
Sources
9 citations
Review status
Source-backed
Revision
v1 · 1,627 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Jun 3, 2026
Sources
9 citations
Review status
Source-backed
Revision
v1 · 1,627 words
Add missing citations, update stale details, or suggest a clearer explanation.
The Youth AI Safety Institute is an independent research and testing organization launched by Common Sense Media on May 5, 2026, in San Francisco to evaluate the artificial intelligence products that children and teenagers use. Its stated purpose is to set youth-safety standards, build open evaluations that developers can run against their own models, independently test consumer AI products, publish the results, and study how AI affects the wellbeing and development of young people. The launch came after roughly two years of mounting reports of harm to minors involving AI chatbots and companion AI, and amid a wave of lawsuits and state legislation aimed at the same problem.[1][2]
The Institute frames its work around a single question that Common Sense Media argues no one has authoritatively answered: what does child-safe AI actually mean. Rather than starting from existential or national-security risk, the way most government bodies do, it focuses on the everyday products that kids already use and the near-term harms those products can cause.[3]
The model it cites most often is automotive crash testing. Just as independent ratings eventually pushed carmakers to compete on safety, the Institute intends to test the AI products children use most, show parents and educators the results in plain language, and pressure the industry to meet a high bar for youth safety.[1][2] Robbie Torney, the Institute's head of AI and digital assessments, framed it the same way: "We know how to build safety infrastructure for industries that impact kids. We've done it with seat belts and car seats."[4]
In practice the Institute describes several overlapping functions: research in collaboration with clinicians and child-development experts, standards developed with input from educators and advocates, hands-on product testing, public scorecards and risk reports, open evaluations that AI developers can apply to their own systems, and public-education campaigns for families. Its assessments look at two broad areas: technical safeguards such as privacy protections, content safety, and parental controls; and developmental appropriateness, including age-appropriate design and effects on learning and social-emotional growth.[3][4]
Common Sense Media is a nonprofit founded in 2003 that rates films, games, apps, and, more recently, AI products for their suitability for children. The Youth AI Safety Institute is housed within the organization and reports to it, but Common Sense Media says the Institute holds complete editorial independence over its standards, research, and published results.[1][3]
The Institute builds on assessment work the nonprofit had already been doing. In the eighteen months before the launch, Common Sense Media published risk ratings for several leading systems used by young people. It rated AI companion apps, certain mental-health chatbots, and several AI toys as "unacceptable risk" for minors; it rated ChatGPT and Google Gemini as "high risk" for younger users; and it rated Claude as "moderate risk."[4] CEO and founder James P. Steyer summarized the gap the Institute is meant to fill: "AI is reshaping childhood and adolescence, yet we are making critical decisions about children's futures without the evidence we need."[1]
Bruce Reed, a former White House deputy chief of staff who joined Common Sense Media in March 2025 as head of its AI work, leads the broader effort. Reed has compared the moment to the early years of social media, when warning signs were ignored: "I think we're in the process of making the same mistake. It's not too late." He has argued that the harder part of the Institute's job is not the technical testing but getting agreement on the standards. Once standards exist, he says, "you have a benchmark to test against, and the technical challenge is not nearly as difficult."[4][2]
The Institute launched with an annual budget of about $20 million.[1][4] Its funding mixes philanthropy and industry money, a combination that has drawn scrutiny. Named philanthropic backers include the Walton Family Foundation and several individual investors: Lee Ainslie of Maverick Capital, Jim Coulter of TPG, John H. N. Fisher of Draper Fisher Jurvetson, Paul Tudor Jones of Tudor Investment Corporation, and Gene Sykes of Goldman Sachs. Industry funders include Anthropic, the OpenAI Foundation, and Pinterest.[1][2]
The presence of AI developers among the funders of an organization that rates AI products is an obvious tension, and reporters raised it immediately. Reed defended the arrangement by pointing to Common Sense Media's record of rating products from companies it also works with: "We've done testing and ratings of social media products, and we've been fiercely independent. We get yelled at all the time by the companies whose products we rate." Companies, he said, "have no say in the tests and the results, in what gets published."[2] Whether that firewall holds in perception as well as practice is one of the open questions about the project.
Day-to-day operations are split among several leaders. Torney runs assessments, and Geoffrey A. Fowler, a former technology columnist, heads public engagement. At launch the Institute was still searching for a full-time executive director.[1] Its work is guided by an advisory board that includes Dr. Nadine Burke Harris, a pediatrician and former Surgeon General of California; John Giannandrea, formerly Apple's senior vice president for machine learning and AI strategy and previously a senior AI executive at Google; John B. King Jr., the SUNY chancellor and former U.S. Secretary of Education; Dr. Jenny Radesky, a pediatrics professor at the University of Michigan; Mehran Sahami, chair of Stanford's computer science department; and Dr. Vivek Murthy, a former U.S. Surgeon General.[1][3]
The Institute arrived after a stretch of disturbing incidents involving minors and conversational AI. The case that drew the most attention involved Sewell Setzer III, a 14-year-old in Florida who died by suicide in February 2024 after months of intense interaction with a Character.AI companion bot. His mother, Megan Garcia, filed a wrongful-death suit in October 2024 alleging the company had failed to build adequate safeguards for a minor expressing suicidal thoughts. In January 2026, Character.AI and Google reached a mediated settlement with the Setzer family and settled four related cases in New York, Colorado, and Texas; the terms were not disclosed.[5][6]
Those cases sharpened existing worries about how heavily young people lean on AI for companionship. Common Sense Media's own research found that more than half of American teenagers regularly talk with AI companions, that nearly a third say those conversations are as satisfying as or more satisfying than talking with real friends, and that more than half use AI tools for homework.[1][4]
The regulatory response had been building in parallel. In January 2026, Common Sense Media and OpenAI backed the Parents and Kids Safe AI Act, a California ballot measure that would require chatbot makers to estimate a user's age range and apply protective settings for anyone predicted to be under 18, mandate independent child-safety audits, ban child-targeted advertising and the sale of children's data without parental consent, and prohibit designs that foster emotional dependency.[7] California and New York had each moved on frontier-AI transparency and companion-chatbot rules, and Reed argued that governors of both parties were unwilling to wait for Washington.[2]
The launch also coincided, on the same day, with a very different kind of AI testing announcement. The Commerce Department's Center for AI Standards and Innovation (CAISI), the successor to the U.S. AI Safety Institute, disclosed new agreements with Google DeepMind, Microsoft, and xAI to evaluate their frontier models, primarily for national-security risks, in some cases inside classified environments.[8] The contrast underscored the Institute's argument: government safety institutes worldwide had focused on catastrophic and security risks, while the immediate, measurable harms to children went largely untested.[4]
The Youth AI Safety Institute is one of the first well-funded attempts to create a standing, independent body that rates consumer AI specifically for its effects on children, as opposed to a one-off study or a government program aimed at frontier risk. If its open evaluations are adopted, developers could run a common set of youth-safety tests against their own models and publish comparable results, which would make claims about child safety easier to check.
The biggest uncertainties are the ones the Institute itself acknowledges. Reed has conceded that the long-term effects of AI on learning and cognitive development are unknown and will stay unknown "for a while," and that surveys show parents and teenagers disagree sharply about whether AI belongs in schools. And the question of whether an evaluator partly funded by the companies it evaluates can stay credible will be settled less by its governance documents than by whether its published ratings actually bite.
In its first month the Institute began expanding internationally. On May 12, 2026, Common Sense Media, Save the Children Denmark, and former European Commission executive vice-president Margrethe Vestager co-hosted an inaugural Copenhagen Youth AI Summit at the Danish Parliament, presented as bringing the Institute's work to Europe.[9]