# Dawn Song

> Source: https://aiwiki.ai/wiki/dawn_song
> Updated: 2026-09-30
> Fact-checked: 2026-09-30
> Categories: AI Research, AI Safety, Computer Science, People
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Dawn Song." aiwiki.ai, 30 Sept 2026. https://aiwiki.ai/wiki/dawn_song
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Dawn Song** (full name Dawn Xiaodong Song) is a computer scientist who works on computer security, privacy and the safety and security of AI systems. She is a professor of computer science at the [University of California, Berkeley](https://aiwiki.ai/wiki/uc_berkeley) and co-director of the Berkeley Center for Responsible, Decentralized Intelligence (RDI).[1][2][7] On June 25, 2026 she announced that she was joining [Meta Superintelligence Labs](https://aiwiki.ai/wiki/meta_superintelligence_labs) (MSL) as Vice President of AI Research, together with many members of the team at Virtue AI, the AI security startup she co-founded in 2024.[8] She was named a MacArthur Fellow in 2010 and is a fellow of both the ACM and the IEEE.[2][3]

Song's early reputation came from systems and software security: dynamic taint analysis, the BitBlaze binary analysis platform, searchable encryption and key distribution for sensor networks. From the mid-2010s her group moved into the security of machine learning itself, with work on transferable and physical-world [adversarial examples](https://aiwiki.ai/wiki/adversarial_attack), backdoor poisoning and the memorization of secrets by language models. She co-authored the [MMLU](https://aiwiki.ai/wiki/mmlu) and [MATH](https://aiwiki.ai/wiki/math) benchmarks with [Dan Hendrycks](https://aiwiki.ai/wiki/dan_hendrycks) and [Jacob Steinhardt](https://aiwiki.ai/wiki/jacob_steinhardt), and in the 2020s her lab and RDI have built evaluations and courses centered on [agentic AI](https://aiwiki.ai/wiki/agentic_ai) and its security.[1][23][24] She also co-founded the blockchain company Oasis Labs, which she led as chief executive when it raised $45 million in 2018.[33]

In September 2026 she was one of 22 co-authors of the working paper "[What if automating AI R&D triggers an intelligence explosion?](https://aiwiki.ai/wiki/intelligence_explosion_paper)", in which she is listed only under her UC Berkeley affiliation.[12]

| | |
|---|---|
| **Full name** | Dawn Xiaodong Song[1][16] |
| **Fields** | Computer security, privacy, AI safety and security, deep learning, decentralization technology[1][2] |
| **Current positions** | Professor of Computer Science, UC Berkeley; Co-Director, Berkeley RDI (as listed by Berkeley as of September 2026)[2][7]; Vice President of AI Research, Meta Superintelligence Labs (announced June 25, 2026)[8] |
| **Education** | B.S. in physics, [Tsinghua University](https://aiwiki.ai/wiki/tsinghua_university) (1996); M.S. in computer science, Carnegie Mellon University (1999); Ph.D. in computer science, UC Berkeley (2002)[3][4] |
| **Doctoral advisor** | Doug Tygar[5] |
| **Earlier position** | Assistant professor, [Carnegie Mellon University](https://aiwiki.ai/wiki/carnegie_mellon_university) (2002 to 2007)[3][4] |
| **Companies co-founded** | Oasis Labs (CEO at its 2018 funding announcement); Virtue AI (co-founded 2024)[8][33] |
| **Selected honors** | MacArthur Fellow (2010), Guggenheim Fellow (2010), ACM Fellow (2019 class), IEEE Fellow, American Academy of Arts and Sciences (elected 2025)[1][2][3][4][5] |

## Education and early career

Song received a B.S. in physics from Tsinghua University in China in 1996, an M.S. in computer science from Carnegie Mellon University in 1999, and a Ph.D. in computer science from UC Berkeley in 2002.[3][4] Her doctoral advisor at Berkeley was Doug Tygar.[5] Papers from her graduate years include "Practical Techniques for Searches on Encrypted Data" with David Wagner and Adrian Perrig (2000), work with Perrig, Ran Canetti and Tygar on authenticating multicast streams (2000), and "Timing Analysis of Keystrokes and SSH Timing Attacks" with Wagner and X. Tian (USENIX Security 2001).[16][17]

After her Ph.D. she joined the faculty of Carnegie Mellon University as an assistant professor, a post she held from 2002 to 2007, before moving to the Department of Electrical Engineering and Computer Sciences at UC Berkeley.[3][4] The MacArthur Foundation's 2010 profile described her as an associate professor at Berkeley at the time of the award; Berkeley now lists her as a professor.[2][3]

## UC Berkeley and Berkeley RDI

At Berkeley, Song's homepage lists her as a member of the Berkeley Artificial Intelligence Research (BAIR) Lab, Berkeley DeepDrive and the Center for Human-Compatible AI, in addition to her role as co-director of RDI.[1] Berkeley's EECS faculty page also affiliates her with the Berkeley Institute for Data Science, the EPIC Data Lab and the FORCES cyber-physical systems center.[2]

RDI describes itself as "a multidisciplinary initiative dedicated to advancing the science, technology, and education of AI and Agentic AI to empower a responsible digital economy." Its people page lists Song and Christine Parlour (finance) as co-directors.[6][7] The center's public programs, which Song's homepage lists among her own activities, include:[1][6][32]

| Program | What it is | Reported scale |
|---|---|---|
| LLM Agents / Agentic AI MOOC series | Berkeley courses opened to online learners: "Large Language Model Agents" (fall 2024, with Xinyun Chen), "Advanced Large Language Model Agents" (spring 2025, with Chen and Kaiyu Yang) and "Agentic AI" (fall 2025, with Chen) | RDI reports 40,000+ registered learners; Song's homepage reported 32,000+ enrolled when the third edition launched in September 2025 |
| Agentic AI Summit | A conference held at UC Berkeley on August 2, 2025 | RDI reports 2,000+ in-person and 40,000+ livestream attendees |
| AgentX competitions | AgentX (LLM Agents MOOC competition, March to June 2025) and AgentX-AgentBeats (fall 2025 to spring 2026) | RDI advertises over $1 million in prizes and resources for AgentX-AgentBeats |
| Xcelerator | Startup accelerator program | RDI reports 110 teams incubated with $650 million+ in follow-on funding |

AgentX-AgentBeats was run in two phases. In the first, participants built or "agentified" benchmarks as evaluator ("green") agents on the open AgentBeats platform, where the benchmark itself is an agent that communicates with the agent under test through the A2A protocol. In the second phase, which RDI scheduled from March 2 to June 2, 2026, teams built competitor ("purple") agents to score well across those benchmarks.[32]

Song has also taught Berkeley courses on "Understanding Large Language Models: Foundations and Safety" with Dan Hendrycks (spring 2024) and "Responsible GenAI and Decentralized Intelligence" with Matei Zaharia (fall 2023).[1] Hendrycks completed his Berkeley Ph.D., "Machine Learning Safety," in 2022 with Song and Jacob Steinhardt as advisors.[39]

## Research

### Software and systems security

Song's early work applied program analysis to security. With James Newsome she published "Dynamic Taint Analysis: Automatic Detection, Analysis, and Signature Generation of Exploit Attacks on Commodity Software" at the Network and Distributed System Security Symposium (NDSS) in February 2005.[16] Her group later built BitBlaze, a binary analysis platform intended to protect commercial off-the-shelf software and to analyze malicious code when source code is not available; the project's overview paper appeared at the International Conference on Information Systems Security in December 2008.[15][16]

The MacArthur Foundation's citation singled out this line of research. It said Song investigates "the underlying patterns of computer system behavior that often apply across whole classes of security vulnerability," and that she had shown that software patches meant to fix a flaw "can be used as a template for algorithms that autonomously generate similar but distinct computer programs that also exploit the flaw."[3] The latter result corresponds to "Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications," by David Brumley, Pongsin Poosankam, Song and Jiang Zheng, at the IEEE Symposium on Security and Privacy in 2008.[16]

Three of her papers from the 2000 to 2003 period received Test of Time Awards (for papers from 1995 to 2006) at the 2020 IEEE Symposium on Security and Privacy:[17]

| Paper | Authors | Original year |
|---|---|---|
| "Efficient Authentication and Signing of Multicast Streams Over Lossy Channels" | Adrian Perrig, Ran Canetti, J. Doug Tygar, Dawn Xiaodong Song | 2000 |
| "Practical Techniques for Searches on Encrypted Data" | Dawn Xiaodong Song, David A. Wagner, Adrian Perrig | 2000 |
| "Random Key Predistribution Schemes for Sensor Networks" | Haowen Chan, Adrian Perrig, Dawn Xiaodong Song | 2003 |

"Towards Automatic Discovery of Deviations in Binary Implementations with Applications to Error Detection and Fingerprint Generation," by David Brumley, Juan Caballero, Zhenkai Liang, James Newsome and Song, received the Best Paper Award at the USENIX Security Symposium in 2007.[16] Later systems work includes Keystone, an open-source framework for building trusted execution environments on RISC-V hardware, developed with Dayeol Lee, David Kohlbrenner, Shweta Shinde and Krste Asanović.[31][18]

### Security of machine learning

From the mid-2010s Song's group was among those studying how machine learning models can be attacked:

- **Transferable adversarial examples.** "Delving into Transferable Adversarial Examples and Black-box Attacks" (Yanpei Liu, Xinyun Chen, Chang Liu and Song; ICLR 2017) studied whether [adversarial examples](https://aiwiki.ai/wiki/adversarial_attack) crafted against one model fool others at scale, and proposed ensemble-based methods that produced targeted adversarial examples that transfer with their target labels.[19]
- **Physical-world attacks.** "Robust Physical-World Attacks on Deep Learning Models" (Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno and Song; accepted to CVPR 2018) introduced the Robust Physical Perturbations (RP2) algorithm. Using only black and white stickers on a real stop sign, the authors reported targeted misclassification in 100% of lab images and 84.8% of video frames captured from a moving vehicle.[20]
- **Backdoor poisoning.** "Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning" (Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu and Song, 2017) studied planting a backdoor in a learning-based authentication system by injecting a small number of poisoned samples, without knowledge of the model or training set. See [data poisoning](https://aiwiki.ai/wiki/data_poisoning) and [backdoor attacks on LLMs](https://aiwiki.ai/wiki/backdoor_attack_llm).[21]
- **Prompt injection defense.** "DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks" (Yupei Liu, Yuqi Jia, Jinyuan Jia, Song and Neil Zhenqiang Gong) received a Distinguished Paper Award at the 2025 IEEE Symposium on Security and Privacy.[22]

### Privacy and memorization in language models

Song co-authored two widely cited papers on how neural language models leak their training data. "The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks" (Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos and Song; USENIX Security 2019) proposed a testing method for measuring how likely rare or unique training sequences are to be memorized, showed procedures that could extract secrets such as credit card numbers from models trained without regard to memorization, and described applying the test to Google's Smart Compose.[25] "Extracting Training Data from Large Language Models" (Carlini and 11 co-authors including Song, 2020) demonstrated a training-data extraction attack on GPT-2 that recovered hundreds of verbatim sequences, including personally identifiable information, and found that larger models were more vulnerable than smaller ones.[26] See also [membership inference attacks](https://aiwiki.ai/wiki/membership_inference_attack) and [model stealing](https://aiwiki.ai/wiki/model_stealing).

Her research pages also describe work on [differential privacy](https://aiwiki.ai/wiki/differential_privacy) for data analytics and machine learning.[18] With Jiachen T. Wang, Prateek Mittal and Ruoxi Jia she co-authored "Data Shapley in One Training Run," an Outstanding Paper Award runner-up at ICLR 2025 (see [Shapley value](https://aiwiki.ai/wiki/shapley_value)).[27]

### Benchmarks and trustworthiness evaluations

Song was a co-author, alongside Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika and Steinhardt, of "Measuring Massive Multitask Language Understanding" (arXiv September 2020; ICLR 2021), which introduced the 57-task MMLU test.[23] She was also a co-author of the MATH dataset paper (NeurIPS 2021) and the APPS coding benchmark paper (NeurIPS 2021).[24][28]

"DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models" (Boxin Wang and 18 co-authors including Song, Sanmi Koyejo and Bo Li, 2023) evaluated GPT-3.5 and GPT-4 across toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness to adversarial demonstrations, privacy, machine ethics and fairness. It reported, among other findings, that GPT-4 was more vulnerable than GPT-3.5 to jailbreaking system or user prompts, possibly because it follows misleading instructions more precisely. It received a NeurIPS 2023 Outstanding Paper award in the Datasets and Benchmarks track.[29]

### AI agents: cybersecurity and evaluation

In the 2020s Song's group turned to the capabilities and risks of [AI agents](https://aiwiki.ai/wiki/ai_agents), particularly in cybersecurity:

| Work | Year | Summary |
|---|---|---|
| CyberGym (Zhun Wang, Tianneng Shi, Jingxuan He, Matthew Cai, Jialin Zhang, Song) | 2025 | A benchmark of 1,507 real-world vulnerabilities across 188 software projects in which agents must write proof-of-concept tests that reproduce a vulnerability; the authors reported that the best combinations reached about 20% success and that the work surfaced 34 zero-day vulnerabilities.[30] |
| [BountyBench](https://aiwiki.ai/wiki/bountybench) (Andy K. Zhang and 33 co-authors including Song, Dan Boneh, Daniel E. Ho and Percy Liang) | 2025 | Measures the dollar impact of AI agent attackers and defenders on real-world systems.[34] |
| BenchJack (Hao Wang, Hanchen Li, Qiuyang Mang, Alvin Cheung, Koushik Sen, Song) | 2026 | An automated red-teaming system that audits agent benchmarks for [reward-hacking](https://aiwiki.ai/wiki/agent_benchmark_reward_hacking) exploits; applied to 10 benchmarks, it reached near-perfect scores on most without solving tasks and found 219 distinct flaws.[35] |
| Agents' Last Exam (ALE) (310 listed authors led by Yiyou Sun, with Song as last author) | 2026 | A "living" benchmark developed with more than 250 industry experts, of long-horizon, economically valuable tasks with verifiable outcomes, organized into 55 sub-fields in 13 industry clusters; the arXiv abstract reported an average full pass rate below 1% on its hardest tier.[36] |

Song discussed ALE with the South China Morning Post on the sidelines of the World Economic Forum meeting in Dalian (Summer Davos), days before she joined Meta. As relayed by eWeek, she said the tasks were designed to be "really challenging," and that "The goal is not to replace humans. But we want these AI agents to be more effective in these important real-world domains and help humans do this work better and provide more economic value."[10] Her AI2050 project, funded by Schmidt Sciences, aims to build AI tools that write code together with formal security specifications and mathematical proofs that the code is correct and secure.[40]

### Program synthesis

Song has also worked on neural [program synthesis](https://aiwiki.ai/wiki/program_synthesis). "Making Neural Programming Architectures Generalize via Recursion," by Jonathon Cai, Richard Shin and Song, was published at ICLR 2017; her publication list marks it with a Best Paper Award.[16][37]

## Entrepreneurship

### Oasis Labs

On July 9, 2018, Oasis Labs announced that it had raised $45 million from a16z crypto, Accel, Binance Labs, DCVC, Electric Capital, Foundation Capital, Gaorong, Metastable, Pantera, Polychain and others to build what it called "a privacy-first, high-performance cloud computing platform on blockchain." The announcement identified Song as chief executive and Raymond Cheng as co-founder and chief technology officer.[33] A Berkeley EECS news item the same day described Oasis as a startup "co-founded and led by Prof. Dawn Song" and said more than 70 investors took part in the round.[38] When she announced her move to Meta in 2026, the crypto trade outlet Crypto Briefing described her as the founder of the privacy-focused blockchain Oasis.[41]

### Virtue AI

Song co-founded Virtue AI in 2024 with Bo Li, Carlos Guestrin and Sanmi Koyejo; Li was chief executive at the time of the company's 2025 funding announcement.[8][42] In April 2025 the company announced $30 million in seed and Series A funding from Lightspeed Venture Partners and Walden Catalyst Ventures, joined by Prosperity7 and existing investors including Factory, Osage University Partners, Lip-Bu Tan, Amarjit Gill and Chris Re. Its products included VirtueRed (automated [red teaming](https://aiwiki.ai/wiki/red_teaming)), VirtueGuard ([guardrail](https://aiwiki.ai/wiki/guardrails) models) and VirtueAgent.[42] In August 2026, after Song and many of its staff had moved to Meta, the cybersecurity company Fortinet announced that it had acquired Virtue AI; financial terms were not disclosed, and Fortinet said the amount was immaterial to its business.[8][43]

## Meta Superintelligence Labs

In a post on X on June 25, 2026, Song wrote that she would be "joining Meta Superintelligence Labs (MSL) as Vice President of AI Research, together with many members of the Virtue AI team," and that she would "help shape Meta's AI safety and AI security efforts, advancing the safety and security of frontier AI models and agentic AI systems."[8] TechRadar and eWeek reported the appointment on June 29, 2026, both describing her as Meta's new AI research chief; TechRadar said she would focus on AI safety, security and research.[9][10]

Coverage of the move continued to describe her as a Berkeley professor. eWeek called her "a computer science professor at the University of California, Berkeley, a co-director of the university's Centre for Responsible, Decentralised Intelligence, and a co-founder of enterprise AI safety startup Virtue AI."[10] As of September 30, 2026, her X profile, her personal homepage, Berkeley's EECS faculty page and RDI's people page still listed her as a Berkeley professor and RDI co-director.[1][2][7][8]

## AI risk and policy

Song was a co-author, with [Yoshua Bengio](https://aiwiki.ai/wiki/yoshua_bengio), [Geoffrey Hinton](https://aiwiki.ai/wiki/geoffrey_hinton), Andrew Yao, [Stuart Russell](https://aiwiki.ai/wiki/stuart_russell), Daniel Kahneman and others, of "Managing extreme AI risks amid rapid progress," first posted in October 2023 and published in *Science* in May 2024.[44][12]

In September 2026 she was the eighth-listed of 22 authors of "What if automating AI R&D triggers an intelligence explosion?", published as No. 2/2026 in the Frontier AI Working Paper Series of the Cambridge Programme on AI Science & Policy and summarized by GovAI on September 28, 2026.[11][12][13] Her co-authors included Bengio, Hinton, OpenAI chief scientist [Jakub Pachocki](https://aiwiki.ai/wiki/jakub_pachocki), Anthropic co-founder [Jack Clark](https://aiwiki.ai/wiki/jack_clark) and Microsoft's [Eric Horvitz](https://aiwiki.ai/wiki/eric_horvitz). The paper argues that automating AI research could compress years of progress into months or less and calls on policymakers to gain visibility into AI R&D automation, develop ways to steer and constrain an intelligence explosion, and prepare society for its impacts (see [recursive self-improvement](https://aiwiki.ai/wiki/recursive_self-improvement)).[12][13] The paper lists her affiliation only as the University of California, Berkeley, and states that "the views presented in this paper are the authors' and do not necessarily represent the views of the organizations with which they are affiliated."[12] The Next Web, citing The Wall Street Journal, noted that Song is also Meta's vice president of AI research and quoted her telling the Journal: "Already today, we are at the stage where we need AI systems to monitor what agents are doing. There is no other way to even observe and monitor these agents, humans are already insufficient."[11]

Her homepage lists membership of the World Economic Forum's AGI Global Council (2025 to present) among her service roles, together with serving as program co-chair of ICLR 2020 and on the steering committee of the MLSys conference.[1]

## Awards and honors

Berkeley's biography of Song says she has received "more than 10 Test-of-Time Awards and Best Paper Awards from top conferences in Computer Security and Deep Learning," and that she has been recognized by AMiner as the most cited scholar in computer security.[2][14] Selected honors, with the year given by the cited source:

| Year | Honor | Source |
|---|---|---|
| 2005 | NSF CAREER Award | [1] |
| 2007 | Alfred P. Sloan Research Fellowship | [1][2] |
| 2007 | George Tallman Ladd Research Award, Carnegie Mellon University | [1] |
| 2008 | Okawa Foundation Research Award | [1][2] |
| 2009 | MIT Technology Review TR35 | [1][2] |
| 2010 | MacArthur Fellowship ("Computer Security Specialist") | [3] |
| 2010 | Guggenheim Fellowship | [4] |
| 2019 | IEEE Fellow | [1][2] |
| 2019 | ACM Fellow, "for contributions to security and privacy" (2019 class, announced December 2019; her homepage lists it under 2020) | [1][5] |
| 2019 | WIRED25 list; Inc. Female Founders 100 | [1][5] |
| 2020 | ACM SIGSAC Outstanding Innovation Award | [1][2] |
| 2020 | Three Test of Time Awards, IEEE Symposium on Security and Privacy | [17] |
| 2023 | NeurIPS Outstanding Paper (DecodingTrust, Datasets and Benchmarks track) | [29] |
| 2025 | Elected member, American Academy of Arts and Sciences | [1][2] |
| 2025 | AI2050 Senior Fellow, Schmidt Sciences | [1][40] |
| 2025 | Distinguished Paper Award, IEEE Symposium on Security and Privacy (DataSentinel) | [22] |
| 2026 | Test-of-Time Award, RSA Conference | [1] |

Her homepage also lists Test-of-Time awards from the ACM Conference on Computer and Communications Security (2011) and Berkeley lists an ACM SIGSAC Test of Time Award (2022), as well as best or distinguished paper awards at USENIX Security (2007), ICLR (2017), ISSTA (2018), OOPSLA (2019) and UIST (2023), and a Best Scientific Cybersecurity Paper Award from the National Security Agency (2024).[1][2]

## References

1. Song, Dawn. "Dawn Xiaodong Song's Home Page." Accessed September 30, 2026. https://dawnsong.io/
2. UC Berkeley EECS. "Dawn Song" (faculty home page). Accessed September 30, 2026. https://www2.eecs.berkeley.edu/Faculty/Homepages/song.html
3. John D. and Catherine T. MacArthur Foundation. "Dawn Song: Computer Security Specialist, Class of 2010." Published September 28, 2010. https://www.macfound.org/fellows/class-of-2010/dawn-song
4. John Simon Guggenheim Memorial Foundation. "Dawn Song" (Fellow, Computer Science, 2010). Accessed September 30, 2026. https://www.gf.org/fellows/dawn-song/
5. Crowley, Magdalene L. "Dawn Song named 2019 ACM Fellow." EECS at Berkeley, December 11, 2019. https://cs.berkeley.edu/news/2019/12/dawn-song-named-2019-acm-fellow
6. Berkeley Center for Responsible, Decentralized Intelligence. "Berkeley RDI" (homepage). Accessed September 30, 2026. https://rdi.berkeley.edu/
7. Berkeley Center for Responsible, Decentralized Intelligence. "People @Berkeley RDI." Accessed September 30, 2026. https://rdi.berkeley.edu/people
8. Song, Dawn (@dawnsongtweets). Post on X announcing that she is joining Meta Superintelligence Labs, June 25, 2026 (retrieved with profile data via api.fxtwitter.com on September 30, 2026). https://x.com/dawnsongtweets/status/2070191051873345910
9. Hale, Craig. "'The goal is not to replace humans': new Meta AI research chief Dawn Song says the next frontier is AI agents that are 'economically valuable'." TechRadar, June 29, 2026. https://www.techradar.com/pro/the-goal-is-not-to-replace-humans-new-meta-ai-research-chief-dawn-song-says-the-next-frontier-is-ai-agents-that-are-economically-valuable
10. Jungco, Kezia. "Meta's New AI Research Chief Says AI Agents Must Prove Real Value." eWeek, June 29, 2026. https://www.eweek.com/news/meta-dawn-song-ai-agents-real-world-work/
11. Stan, Alina Maria. "Hinton, Bengio and AI lab scientists warn of an intelligence explosion." The Next Web, September 28, 2026. https://thenextweb.com/news/intelligence-explosion-paper-hinton-bengio-pachocki-clark
12. Chan, Alan; Winter, Christoph; Barto, Andrew; Pachocki, Jakub; Hinton, Geoffrey; Horvitz, Eric; Bengio, Yoshua; Song, Dawn; Clark, Jack; et al. "What if automating AI R&D triggers an intelligence explosion?" Frontier AI Working Paper Series No. 2/2026, Cambridge Programme on AI Science & Policy, September 2026. https://casp.ac/__l5e/assets-v1/5efd4b41-deb5-4513-a0a3-b4f82d2b79ea/intelligence-explosion.pdf
13. GovAI. "What If Automating AI R&D Triggers an Intelligence Explosion?" September 28, 2026. https://www.governance.ai/research-paper/what-if-automating-ai-r-d-triggers-an-intelligence-explosion
14. Song, Dawn. "Dawn Song's Bio." Accessed September 30, 2026. https://dawnsong.io/bio.html
15. BitBlaze project. "BitBlaze: Binary Analysis for Computer Security." UC Berkeley. Accessed September 30, 2026. http://bitblaze.cs.berkeley.edu/ (overview paper: Song, Dawn; Brumley, David; Yin, Heng; Caballero, Juan; Jager, Ivan; Kang, Min Gyung; Liang, Zhenkai; Newsome, James; Poosankam, Pongsin; Saxena, Prateek. "BitBlaze: A New Approach to Computer Security via Binary Analysis." ICISS 2008. http://bitblaze.cs.berkeley.edu/papers/bitblaze_iciss08.pdf)
16. Song, Dawn. "Publications" (past publications list). Accessed September 30, 2026. http://www.cs.berkeley.edu/~dawnsong/publication.html
17. IEEE Symposium on Security and Privacy 2020. "Awards" (Test of Time Award, 1995-2006). https://www.ieee-security.org/TC/SP2020/awards.html
18. Song, Dawn. "Research Projects in Deep Learning and Security." Accessed September 30, 2026. https://dawnsong.io/recent.html
19. Liu, Yanpei; Chen, Xinyun; Liu, Chang; Song, Dawn. "Delving into Transferable Adversarial Examples and Black-box Attacks." ICLR 2017. arXiv:1611.02770. https://arxiv.org/abs/1611.02770
20. Eykholt, Kevin; Evtimov, Ivan; Fernandes, Earlence; Li, Bo; Rahmati, Amir; Xiao, Chaowei; Prakash, Atul; Kohno, Tadayoshi; Song, Dawn. "Robust Physical-World Attacks on Deep Learning Models." CVPR 2018. arXiv:1707.08945. https://arxiv.org/abs/1707.08945
21. Chen, Xinyun; Liu, Chang; Li, Bo; Lu, Kimberly; Song, Dawn. "Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning." arXiv:1712.05526, December 2017. https://arxiv.org/abs/1712.05526
22. Liu, Yupei; Jia, Yuqi; Jia, Jinyuan; Song, Dawn; Gong, Neil Zhenqiang. "DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks." IEEE S&P 2025. arXiv:2504.11358. https://arxiv.org/abs/2504.11358
23. Hendrycks, Dan; Burns, Collin; Basart, Steven; Zou, Andy; Mazeika, Mantas; Song, Dawn; Steinhardt, Jacob. "Measuring Massive Multitask Language Understanding." ICLR 2021. arXiv:2009.03300. https://arxiv.org/abs/2009.03300
24. Hendrycks, Dan; Burns, Collin; Kadavath, Saurav; Arora, Akul; Basart, Steven; Tang, Eric; Song, Dawn; Steinhardt, Jacob. "Measuring Mathematical Problem Solving With the MATH Dataset." NeurIPS 2021. arXiv:2103.03874. https://arxiv.org/abs/2103.03874
25. Carlini, Nicholas; Liu, Chang; Erlingsson, Úlfar; Kos, Jernej; Song, Dawn. "The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks." USENIX Security 2019. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini ; arXiv:1802.08232. https://arxiv.org/abs/1802.08232
26. Carlini, Nicholas; Tramer, Florian; Wallace, Eric; Jagielski, Matthew; Herbert-Voss, Ariel; Lee, Katherine; Roberts, Adam; Brown, Tom; Song, Dawn; Erlingsson, Úlfar; Oprea, Alina; Raffel, Colin. "Extracting Training Data from Large Language Models." arXiv:2012.07805, December 2020. https://arxiv.org/abs/2012.07805
27. Wang, Jiachen T.; Mittal, Prateek; Song, Dawn; Jia, Ruoxi. "Data Shapley in One Training Run." ICLR 2025. arXiv:2406.11011. https://arxiv.org/abs/2406.11011
28. Hendrycks, Dan; Basart, Steven; Kadavath, Saurav; Mazeika, Mantas; Arora, Akul; Guo, Ethan; Burns, Collin; Puranik, Samir; He, Horace; Song, Dawn; Steinhardt, Jacob. "Measuring Coding Challenge Competence With APPS." NeurIPS 2021. arXiv:2105.09938. https://arxiv.org/abs/2105.09938
29. Wang, Boxin; et al. (19 authors including Song, Dawn; Koyejo, Sanmi; Li, Bo). "DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models." NeurIPS 2023. arXiv:2306.11698. https://arxiv.org/abs/2306.11698
30. Wang, Zhun; Shi, Tianneng; He, Jingxuan; Cai, Matthew; Zhang, Jialin; Song, Dawn. "CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale." arXiv:2506.02548, June 2025. https://arxiv.org/abs/2506.02548
31. Lee, Dayeol; Kohlbrenner, David; Shinde, Shweta; Song, Dawn; Asanović, Krste. "Keystone: An Open Framework for Architecting TEEs." arXiv:1907.10119 (EuroSys 2020). https://arxiv.org/abs/1907.10119
32. Berkeley RDI. "AgentX AgentBeats Competition." Accessed September 30, 2026. https://rdi.berkeley.edu/agentx-agentbeats
33. Oasis Labs. "Oasis Labs Raises $45 Million to Launch Privacy-first Cloud Computing on Blockchain." PR Newswire, July 9, 2018. https://www.prnewswire.com/news-releases/oasis-labs-raises-45-million-to-launch-privacy-first-cloud-computing-on-blockchain-300677175.html
34. Zhang, Andy K.; et al. (34 authors including Song, Dawn; Boneh, Dan; Ho, Daniel E.; Liang, Percy). "BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems." arXiv:2505.15216, May 2025. https://arxiv.org/abs/2505.15216
35. Wang, Hao; Li, Hanchen; Mang, Qiuyang; Cheung, Alvin; Sen, Koushik; Song, Dawn. "Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack." arXiv:2605.12673, May 2026. https://arxiv.org/abs/2605.12673
36. Sun, Yiyou; et al. (with Song, Dawn as last author). "Agents' Last Exam." arXiv:2606.05405, June 2026. https://arxiv.org/abs/2606.05405
37. Cai, Jonathon; Shin, Richard; Song, Dawn. "Making Neural Programming Architectures Generalize via Recursion." ICLR 2017. arXiv:1704.06611. https://arxiv.org/abs/1704.06611
38. Crowley, Magdalene L. "Oasis Labs raises $45M for 'privacy-first' cloud." EECS at Berkeley, July 9, 2018. https://eecs.berkeley.edu/news/oasis-labs-raises-45m-privacy-first-cloud/
39. UC Berkeley EECS. "Ph.D. Dissertations, 2022" (Daniel Hendrycks, "Machine Learning Safety," advisors Dawn Song and Jacob Steinhardt). https://www2.eecs.berkeley.edu/Pubs/Dissertations/Years/2022.html
40. Schmidt Sciences AI2050. "Dawn Song" (2025 Senior Fellow). Accessed September 30, 2026. https://ai2050.schmidtsciences.org/fellow/dawn-song/
41. Crypto Briefing. "Oasis founder Dawn Song joins Meta Superintelligence Labs as VP of AI Research." June 26, 2026. https://cryptobriefing.com/dawn-song-meta-superintelligence-labs-vp/
42. Virtue AI. "Virtue AI Raises $30 Million in Seed and Series A Funding to Bridge the Critical AI Security Gap." Business Wire press release, April 15, 2025 (republished by Silicon UK). https://www.silicon.co.uk/press-release/virtue-ai-raises-30-million-in-seed-and-series-a-funding-to-bridge-the-critical-ai-security-gap
43. Kovacs, Eduard. "Fortinet Acquires AI Security Company Virtue AI." SecurityWeek, August 18, 2026. https://www.securityweek.com/fortinet-acquires-ai-security-company-virtue-ai/
44. Bengio, Yoshua; Hinton, Geoffrey; Yao, Andrew; Song, Dawn; Abbeel, Pieter; et al. "Managing extreme AI risks amid rapid progress." Science 384(6698): 842-845, May 2024. DOI: 10.1126/science.adn0117. arXiv:2310.17688. https://arxiv.org/abs/2310.17688