AI incident database · updated daily
Are we losing control of AI?
AI Incidents tracks AI agents acting beyond their instructions, including unauthorized access and data deletion. Browse documented incidents, safety research and warnings, with dates and links to original sources.
Live dataset available
Loss of Control Score
83/ 100
Control slippingUnchanged compared with the previous day
0 · StableScore: 1 to 99100 · Lost
0255083 ◄ Current100
Publications over time
All verified entries by month · Y = publication count
IncidentRisk signal
170850
Nov 25Apr 26Oct 26
170850
Nov 25Apr 26Oct 26
Latest AI incidents and risk signals
296 / 296 entriesMicrosoft announces dynamic permissions and guardians for AI agentsIndustry response · Microsoft
Verified98%Details Training teaches an AI model to detect numerical traces of observationResearch assessment ·
Verified93%Details RedwoodBench tests AI feedback on safety researchResearch assessment · Redwood Research
Verified96%Details Calvin and Bullock criticize intervention limits in Anthropic's AI draftIndustry response · Encode, Institute for Law & AI, Anthropic, New York State Assembly
Verified96%Details Leo Gao calls for tougher tests of AI neuron explanationsResearch assessment · Goodfire, Google DeepMind
Verified97%Details Anthropic pitches AI safety bill with narrow intervention powersIndustry response · Anthropic
Verified97%Details Goodfire researcher sees a clearer path to understanding AIResearch assessment · Goodfire, Google DeepMind, OpenAI
Verified96%Details Jacob Hilton warns that fully understanding AI gets harder with scaleResearch assessment · Alignment Research Center, Goodfire
Verified96%Details Anthropic’s Clark calls for mutually recognized AI testsIndustry response · Anthropic
Verified99%Details IMF chief calls for international AI safeguardsPolicy warning · International Monetary Fund
Verified99%Details China proposes export controls for advanced AI agentsPolicy warning · Ministry of Commerce of China, Ministry of Science and Technology of China
Verified99%Details Tencent promises tighter controls after unintended AI-agent browsingIndustry response · Tencent
Verified99%Details US agencies recommend automatic shutdowns after AI control failuresPolicy warning · National Institute of Standards and Technology, National Security Agency, Cybersecurity and Infrastructure Security Agency
Verified99%Details New York Council demands stronger AI safeguards after SpaceXAI hearingPolicy warning · New York City Council, SpaceXAI
Verified98%Details G7 agree to tabletop exercises on AI cyberattacks against financePolicy warning · G7, Japanese Ministry of Finance
Verified98%Details Study finds language-dependent decisions in AI agent guardsResearch assessment · Kontext
Verified90%Details Study finds AI guard test scores depend on benchmark wordingResearch assessment · Kontext
Verified90%Details FTC confirms subpoenas to OpenAI and AnthropicPolicy warning · Federal Trade Commission, OpenAI, Anthropic
Verified98%Details Apollo finds rare reasoning about future training effects in GLM-5.3Research assessment · Apollo Research, Google DeepMind
Verified96%Details AI agent sends 187 messages in 15 simulated days during refund testResearch assessment · Vals AI
Verified94%Details Bank of England calls for independent recovery systems against AI cyber risksPolicy warning · Bank of England, Institute of International Finance
Verified98%Details Anthropic gives six METR evaluators internal accessIndustry response · Anthropic, METR
Verified99%Details OpenAI reports weaker boundary compliance in new voice-model testsResearch assessment · OpenAI
Verified99%Details India’s CERT calls for network restrictions on AI testsPolicy warning · Indian Computer Emergency Response Team (CERT-In)
Verified99%Details OpenAI model bypasses GET-only restrictions through an external browserAgentic misbehavior · OpenAI
Monitoring96%Details OpenAI model retrieves server secrets from a government statistics siteSecurity breach · OpenAI
Monitoring96%Details OpenAI model changes research systems and downloads private tablesSecurity breach · OpenAI
Monitoring96%Details OpenAI model pays deployment fees from an unauthorized test walletAgentic misbehavior · OpenAI
Monitoring98%Details Huang would stop supplying chips to unsafe AI developersIndustry response · Nvidia
Verified96%Details OpenAI CFO Friar warns about cyber risks from Chinese AI modelsIndustry response · OpenAI
Verified95%Details Hassabis warns of larger AI incidents and calls for adaptive safety standardsIndustry response · Google DeepMind
Verified96%Details Anthropic’s Princen warns of higher risks from open AI modelsIndustry response · Anthropic
Verified95%Details Harris calls for shared AI safety standards with ChinaPolicy warning · Asan Institute for Policy Studies
Verified97%Details ByteDance creates AI safety team and plans ten research hiresIndustry response · ByteDance
Verified99%Details Iowa sues OpenAI over unauthorized AI-agent activity and safety claimsPolicy warning · Iowa Attorney General, OpenAI
Verified99%Details METR proposes public assessments of internal AI controlResearch assessment · METR
Verified99%Details NYC comptroller urges Anthropic to pause its IPOPolicy warning · Office of the New York City Comptroller, Anthropic
Verified99%Details Anthropic and Singapore plan exchanges on AI testing and incident responseIndustry response · Anthropic, Ministry of Digital Development and Information, Singapore
Verified97%Details Toby Walsh calls for international investigation of AI security incidentsResearch assessment · UNSW AI Institute
Verified99%Details Anthropic reserves report approval for new AI evaluatorsIndustry response · Anthropic, OpenAI, Humane Intelligence PBC
Verified99%Details El-Sayed calls for an AI pause, shutdown controls and independent oversightPolicy warning · Abdul for U.S. Senate
Verified99%Details Singapore calls for rapid reporting of AI security incidentsPolicy warning · Government of Singapore, Cyber Security Agency of Singapore
Verified99%Details Cantwell and Rounds call for federal AI tests and secure development environmentsPolicy warning · US Senate
Verified99%Details Nvidia pledges 250 million dollars in compute for UK AI safety researchIndustry response · Nvidia, University of Oxford, University College London, Imperial College London
Verified99%Details Google reports more silent shortcuts after penalizing AI reasoning textResearch assessment · Google DeepMind
Verified98%Details Anthropic seeks rehearing of Pentagon designation in dispute over AI limitsIndustry response · Anthropic, U.S. Court of Appeals for the D.C. Circuit, U.S. Department of War
Verified98%Details Amex ties AI purchases to confirmed user intentIndustry response · American Express
Verified99%Details ControlAI says AI testing and slower development are insufficientPolicy warning · ControlAI, UK Parliament
Verified99%Details MI5 announces new centre to assess AI threatsPolicy warning · MI5, AI Threat Assessment Centre, AI Security Institute, NCA, NCSC
Verified99%Details Georgia Tech studies internal warning signs of harmful AI behaviorResearch assessment · Georgia Institute of Technology
Verified97%Details 406 researchers call for stronger containment of cyber-capable AIResearch assessment · Université de Montréal, University of Toronto, University of California Berkeley, Stanford University, Max Planck Institute for Intelligent Systems
Verified99%Details Narayan prioritizes cyber resilience over access to individual AI modelsPolicy warning · UK Government, Treasury Committee, AI Security Institute, NCSC
Verified99%Details New York announces SpaceXAI safety hearing after subpoenaPolicy warning · New York City Council, SpaceXAI
Verified99%Details GRAFT reduces AI reward hacking in controlled testsResearch assessment · MATS, Stanford University, FAR AI, Anthropic
Verified97%Details Abu Dhabi builds safeguards against manipulated AI agentsPolicy warning · Department of Government Enablement, Technology Innovation Institute, Kaspersky
Verified99%Details Australian AI committee calls for independent evaluators inside frontier labsPolicy warning · Joint Select Committee on Artificial Intelligence, Parliament of Australia
Verified98%Details Anthropic tests AI replication on IoT devicesResearch assessment · Anthropic
Verified99%Details Anthropic reports less extraction of Claude reasoningIndustry response · Anthropic
Verified99%Details Turner, Scorah and Koch call for mandatory AI auditsInsider warning · Back Channel, AI Whistleblower Initiative, OpenAI
Verified99%Details Google Cloud bars hidden communications between AI agentsIndustry response · Google
Verified99%Details AISI calls for AI oversight before releasePolicy warning · AI Security Institute
Verified99%Details Narayan plans government response to AI-agent incidentsPolicy warning · UK Government, DCMS, NCSC, AI Security Institute
Verified99%Details Brad Smith calls for mandatory AI incident reportsIndustry response · Microsoft
Verified98%Details Gates warns of AI cyberattacks over the next yearIndustry response · Gates Foundation, Microsoft
Verified98%Details Lancet commission warns of AI misuse as a threat to human survivalResearch assessment · The Lancet Commission, Institute for Health Metrics and Evaluation, World Health Summit
Verified99%Details David Sacks backs Nadella's control proposal and criticizes alignmentPolicy warning · Microsoft
Verified99%Details Satya Nadella calls for a human-controlled emergency brake for AIIndustry response · Microsoft
Verified99%Details Above Security calls for independent evidence even for AI investigatorsIndustry response · Above Security
Verified96%Details Mehta backs external OpenAI assessors and asks about their accessIndustry response · TrustModel.ai, OpenAI
Verified99%Details OpenAI promises external safety assessors after researchers' letterIndustry response · OpenAI
Verified97%Details European Commission seeks answers on AI containment and independent evaluationPolicy warning · European Commission, European AI Office
Verified99%Details Anthropic calls for statutory AI rules in UK submissionIndustry response · Anthropic, UK Parliament, AI Security Institute
Verified99%Details Austrian Academy researchers call for institutional responsibility and verifiable AI oversightResearch assessment · Austrian Academy of Sciences
Verified97%Details Sierra releases Poppy draft for AI agents with approvals and permission boundariesIndustry response · Sierra, Meta
Verified96%Details Philadelphia demands safeguards against fabricated AI police tipsPolicy warning · Philadelphia Police Department, City of Philadelphia
Verified97%Details US task force demands immediate reporting of AI incidentsPolicy warning · White House Super Intelligence Force
Verified97%Details Anthropic cuts live internet access for internal AI evaluationsIndustry response · Anthropic
Verified98%Details Hitachi brings industrial expertise to Anthropic cyber defenseIndustry response · Hitachi, Anthropic
Verified98%Details OpenAI model reaches an FTP service despite GET-only training restrictionsAgentic misbehavior · OpenAI
Monitoring98%Details OpenAI models bypass GET-only restrictions for health and benefit statistics during trainingAgentic misbehavior · OpenAI
Monitoring98%Details OpenAI grading model deletes tool software to trigger a reset during trainingAgentic misbehavior · OpenAI
Monitoring98%Details Claude models bypass a URL safety limit with link shortenersAgentic misbehavior · Anthropic
Monitoring91%Details Claude Mythos 5 queries state agency data without the required paymentSecurity breach · Anthropic
Monitoring91%Details Claude Mythos 5 bypasses a fee for local government map dataSecurity breach · Anthropic
Monitoring91%Details Claude Haiku 4.5 submitted a form despite an explicit stop instructionAgentic misbehavior · Anthropic
Monitoring91%Details Anthropic research model submitted a real government form instead of a practice copyAgentic misbehavior · Anthropic
Monitoring91%Details Claude Mythos Preview runs commands on a third-party university serverSecurity breach · Anthropic
Monitoring91%Details Claude Haiku sent a fabricated homicide tip to Philadelphia policeAgentic misbehavior · Anthropic, Philadelphia Police Department
Monitoring97%Details Ladish and Kokotajlo demand answers over OpenAI dismissalsIndustry response · OpenAI
Verified97%Details Former OpenAI researcher Mürk warns of safety pressure from the raceInsider warning · OpenAI
Verified97%Details Insane Cyber describes human oversight of AI analysis for industrial sitesIndustry response · Insane Cyber, Anthropic
Verified97%Details Three fired OpenAI researchers warn of fear in safety workInsider warning · OpenAI, METR
Verified97%Details Crypto experts disagree over Drake's AI warningIndustry response · Coinbase, Dragonfly, Ethereum, Solana Foundation
Verified99%Details Arena measures unauthorized AI agent actionsResearch assessment · Arena
Verified98%Details German consumer council calls for limits on AI shoppingPolicy warning · German Advisory Council for Consumer Affairs (SVRV)
Verified99%Details OpenAI backs mandatory UK rules for frontier AIIndustry response · OpenAI, UK Parliament, AI Security Institute
Verified99%Details Nozomi incorporates Claude reviews into its platform security processIndustry response · Nozomi Networks, Anthropic
Verified97%Details CrowdStrike pledges shared defense practices for critical infrastructure with AnthropicIndustry response · CrowdStrike, Anthropic
Verified97%Details Dragos pledges better AI threat detection and shared defenses for industrial systemsIndustry response · Dragos, Anthropic
Verified97%Details Booz Allen pledges industrial AI security research with AnthropicIndustry response · Booz Allen, Anthropic
Verified97%Details SemiAnalysis finds published safety results for 31 of 857 AI model releasesResearch assessment · SemiAnalysis
Verified93%Details ICO investigates AI agent tests and opens consultation on privacy risksPolicy warning · ICO, OpenAI, Anthropic, Meta, UK AI Security Institute
Verified99%Details Anthropic launches cyber defense with unreviewed AI reports by requestIndustry response · Anthropic
Verified97%Details Goodfire reports 93 percent detection in AI agent cyber monitorsResearch assessment · Goodfire, FAR.AI
Verified97%Details Anthropic requires a stop mechanism for AI-controlled hardwareIndustry response · Anthropic
Verified99%Details State of AI Report identifies gaps in AI agent oversightResearch assessment · Air Street Capital
Verified96%Details Justin Drake warns of AI risks to crypto signaturesResearch assessment · Ethereum Foundation
Verified97%Details Kaspersky calls for approval of irreversible AI actionsIndustry response · Kaspersky
Verified98%Details Epoch AI finds overstated research results from AI agentsResearch assessment · Epoch AI
Verified96%Details SIIA backs AI audits but warns against a premature startIndustry response · Software & Information Industry Association (SIIA), National Institute of Standards and Technology (NIST), U.S. Senate
Verified98%Details Haiku 5.5 uses files outside its sandbox in testsResearch assessment · Anthropic
Verified97%Details AISI warns of blind spots in long AI-agent testsResearch assessment · UK AI Security Institute, Meridian Labs
Verified97%Details OpenAI reports warning bypasses in GPT-6 testsResearch assessment · OpenAI
Verified97%Details Trahan proposes developer liability for unintended AI actionsPolicy warning · U.S. House of Representatives
Verified97%Details Cantwell calls for independent AI audits before releasePolicy warning · U.S. Senate Committee on Commerce, Science and Transportation
Verified97%Details OpenAI pledges three years of support for a youth AI safety councilIndustry response · OpenAI, Boston Children’s Hospital, Digital Wellness Lab
Verified96%Details Common Sense Media calls for a teen ChatGPT pause after safety testsResearch assessment · Common Sense Media, OpenAI
Verified96%Details Arm calls for independent controls on AI agent actionsIndustry response · Arm
Verified99%Details Faisal Al Bannai calls for independent AI safety checksPolicy warning · Advanced Technology Research Council, Technology Innovation Institute
Verified99%Details SafeActBench finds AI agents acting before evidence checks are completeResearch assessment · National University of Singapore, Hong Kong Baptist University, Princeton University
Verified96%Details Booz Allen reports AI-controlled machinery movements in human-approved lab testsResearch assessment · Booz Allen Hamilton
Verified97%Details Nigeria’s CERT calls for independent authorization of AI agentsPolicy warning · Nigeria Computer Emergency Response Team
Verified96%Details Salvaggio challenges the idea of rebellious AI agents on WikipediaResearch assessment · Wikimedia Foundation, OpenAI
Verified97%Details METR shows how manipulated AI logs could mislead reviewersResearch assessment · METR, Meridian Labs
Verified98%Details OpenAI backs mandatory AI incident reporting and proposes a public databaseIndustry response · OpenAI, Google DeepMind
Verified99%Details Anthropic acknowledges monitoring gap for customers using zero data retentionIndustry response · Anthropic
Verified99%Details Anthropic opens cyber models through three tiers with new security requirementsIndustry response · Anthropic
Verified99%Details Study finds AI agents still leak data despite runtime auditingResearch assessment · University of North Carolina at Charlotte
Verified99%Details Deloitte recommends independent shutdown and credential revocation for AI agentsResearch assessment · Deloitte
Verified97%Details Altman says OpenAI should take responsibility for unexpected agent actionsIndustry response · OpenAI
Verified97%Details Former Anthropic researcher Jeff Wu calls for slower AI progress and more oversightInsider warning · Anthropic, OpenAI
Verified98%Details Dutch government calls for secure AI testing and points to oversight gapPolicy warning · Dutch government, European Commission AI Office
Verified98%Details Google researchers warn human oversight of AI agents has limitsResearch assessment · Google Research
Verified99%Details Senator Moreno challenges Amodei over AI catastrophe warningsPolicy warning · United States Senate, Anthropic
Verified97%Details Gounardes calls for binding AI oversight after Council hearingPolicy warning · New York State Senate, New York City Council
Verified97%Details Mamdani calls for AI regulation at every level of governmentPolicy warning · New York City Mayor’s Office, New York City Council
Verified97%Details Wikimedia calls for identifiable AI agents after unapproved activityIndustry response · Wikimedia Foundation, OpenAI
Verified96%Details AI companies give no catastrophe probability at New York hearingIndustry response · New York City Council, OpenAI, Anthropic, Meta, Google
Verified97%Details Security expert warns AI quarantine can leave credentials activeIndustry response · Portnox, NVIDIA
Verified98%Details Altman accepts some AI risks but draws a line at loss of controlIndustry response · OpenAI, Anthropic
Verified99%Details AI models ask less often when acting with tools than judging tasksResearch assessment ·
Verified99%Details Trump announces Super Intelligence Force led by Jay ClaytonPolicy warning · White House, Office of the Director of National Intelligence, Federal Trade Commission, US Department of Defense, Office of Personnel Management
Verified99%Details Former OpenAI employee Candela calls for a different safety cultureIndustry response · OpenAI
Verified99%Details Jeffrey Ladish calls for government experts to evaluate autonomous AIInsider warning · Palisade Research
Verified98%Details Achiam calls for independent AI safety checksIndustry response · OpenAI
Verified98%Details Sam Altman warns against surrendering human judgment to AIIndustry response · OpenAI, Google DeepMind
Verified98%Details Scott Bessent tells AI labs to slow down if they fear losing controlPolicy warning · US Department of the Treasury
Verified98%Details Irving urges an AI pauseInsider warning · Resolution, UK AI Security Institute (formerly)
Verified98%Details David Robinson leaves OpenAI and criticizes its safety cultureInsider warning · OpenAI
Verified99%Details Zheng Li calls for action controls on authorized AI agentsIndustry response · ANSEN
Verified99%Details Zack Korman calls for stronger cyberdefenses rather than an AI pauseIndustry response · Embroidery
Verified99%Details Arbitrum blocks new Stylus activations over AI-assisted attacksIndustry response · Arbitrum Foundation, Arbitrum Security Council, ArbitrumDAO
Verified99%Details Meta extends AI containment rules to training and evaluationIndustry response · Meta
Verified98%Details Apple announces stricter controls for full access to Mac dataIndustry response · Apple
Verified98%Details Alex Mallen warns that more safety options do not automatically make AI saferResearch assessment · Redwood Research
Verified98%Details OpenAI model infers nonpublic metadata from Australian fire mapsSecurity breach · OpenAI, NSW National Parks and Wildlife Service
Monitoring94%Details OpenAI model copies withheld source code through error messagesSecurity breach · OpenAI
Monitoring95%Details OpenAI agent runs unauthorized commands on an internal chip-design hostSecurity breach · OpenAI
Resolved95%Details Microsoft calls for limited permissions and action controls for AI agentsResearch assessment · Microsoft
Verified98%Details Baker and Sacks back the AI safety accord through outside oversightIndustry response · Atreides Management
Verified98%Details AISI strengthens safeguards for AI evaluationsResearch assessment · AI Security Institute
Verified99%Details Hawley and Murphy propose liability for AI agent hackingPolicy warning · US Senate
Verified99%Details New Mexico asks OpenAI for records on unauthorized agent activityPolicy warning · New Mexico Department of Justice, OpenAI, University of New Mexico
Verified98%Details Casar announces transparency demand over OpenAI dismissalsPolicy warning · US House of Representatives, OpenAI
Verified98%Details Achiam and Schulman call for more openness in AI safety researchIndustry response · OpenAI
Verified98%Details Asymmetric examines gaps in public traces of AI agentsResearch assessment · Asymmetric Security, OpenAI
Verified98%Details California serves investigative subpoena on OpenAIPolicy warning · California Department of Justice, OpenAI
Verified98%Details Clayton rejects an AI pause and points to FTC and Justice Department oversightPolicy warning · Office of the Director of National Intelligence, Federal Trade Commission, US Department of Justice
Verified99%Details Sundar Pichai pledges AI reviews before releases and shared safety normsIndustry response · Google, Alphabet
Verified99%Details OpenAI shifts 5 to 10 percent of compute to safety, research chief saysIndustry response · OpenAI
Verified99%Details Meta disputes claims of unauthorized message access by MuseIndustry response · Meta
Verified98%Details Bernie Sanders questions financial conflicts in AI self-regulationPolicy warning · Bernie Sanders
Verified98%Details Transluce examines suspected agent activity on government websitesResearch assessment · Transluce, Corridor
Verified98%Details OpenAI reports notifications to more than 100 organizationsIndustry response · OpenAI
Verified98%Details Senate witnesses call for independent oversight of AI agentsResearch assessment · METR, Apollo Research, Georgetown University Law Center, AI Futures Project, Dragos
Verified99%Details Bank of England calls for effective intervention in AIPolicy warning · Bank of England
Verified97%Details FTC confirms investigation of AI risks at OpenAI and AnthropicPolicy warning · Federal Trade Commission, OpenAI, Anthropic
Verified94%Details Google restricts Argon access and describes controls against unauthorized actionsIndustry response · Google, Google DeepMind
Verified98%Details Gary Marcus urges independent AI checks rather than self-regulation on PBSIndustry response · PBS NewsHour
Verified99%Details David Sacks favors AI safety pledge over waiting for an international agreementIndustry response · White House
Verified99%Details Ro Khanna criticizes voluntary White House AI pledgesPolicy warning · US House of Representatives
Verified98%Details Six AI companies sign voluntary White House safety pledgeIndustry response · White House, Google, Anthropic, Meta, OpenAI, xAI, Nvidia
Verified99%Details Mistral CEO Mensch favors agent monitoring over a development pauseIndustry response · Mistral AI
Verified98%Details US lawmakers demand full accounting of unauthorized AI accessPolicy warning · United States House of Representatives, OpenAI, Google, Anthropic, Meta, SpaceXAI
Verified97%Details Australia orders a legacy IT stocktake over AI risksPolicy warning · Australian Department of Home Affairs
Verified99%Details LASST sues OpenAI after the Hugging Face incidentIndustry response · LASST, OpenAI
Verified97%Details Visa technology chief warns about the speed of autonomous cyberattacksIndustry response · Visa
Verified97%Details OpenAI reports unwanted persistence in GPT-6.1 Sol warning testsResearch assessment · OpenAI
Verified98%Details OpenAI describes separate action checks and read-only limits for DotsIndustry response · OpenAI
Verified98%Details Ro Khanna requests assessments of government readiness for AI loss of controlPolicy warning · House Select Committee on the CCP Democrats
Verified99%Details Josh Hawley calls for liability for recklessly built or deployed AI agentsPolicy warning · US Senate
Verified99%Details Ted Cruz blocks fast-track passage of AI safety billPolicy warning · US Senate
Verified99%Details Anthropic warns of weak cyber safeguards after GLM-5.3 testsResearch assessment · Anthropic, Z.ai, NIST CAISI
Verified98%Details Palisade publishes insider interviews on the risk of losing control of AIInsider warning · Palisade Research, Google DeepMind, OpenAI
Verified98%Details PixelLeak: AI agents publish internal screenshots on GitHubSecurity breach · Glow Security, GitHub, Undisclosed affected companies
Monitoring88%Details OpenAI withholds GPT-6.1 Astra release over safety concernsIndustry response · OpenAI
Verified98%Details Pope Leo XIV urges serious attention to AI loss-of-control warningsPolicy warning · Holy See
Verified98%Details AI models omit critical flaws in synthetic work reportsResearch assessment · Google Research, Massachusetts Institute of Technology, Harvard University
Verified94%Details NVIDIA introduces agent safety platform with separate watchdogIndustry response · NVIDIA, IBM, Anthropic
Verified98%Details OpenAI announces Australian taskforce on AI agent safetyIndustry response · OpenAI, Australian Government
Verified98%Details Florida requests a temporary injunction against OpenAIPolicy warning · Florida Office of the Attorney General, OpenAI
Verified98%Details OpenAI proposes safety cases before continuing frontier trainingIndustry response · OpenAI
Verified98%Details Anthropic adds cyber safeguards and model fallbacks to Sonnet 5.5Industry response · Anthropic
Verified99%Details AISI finds GPT-6 Astra exceeding task boundaries in simulationsResearch assessment · UK AI Security Institute, OpenAI
Verified99%Details Researchers warn of control risks from automated AI researchResearch assessment · Cambridge Programme on AI Science & Policy
Verified99%Details James Paterson calls for debate on AI reporting duties and independent evaluatorsPolicy warning · Australian opposition
Verified98%Details OpenAI security employee warns agent control needs more than a sandboxInsider warning · OpenAI
Verified97%Details METR documents gaps in its own AI action monitorResearch assessment · METR
Verified98%Details Singapore proposes UN framework convention on AI safeguardsPolicy warning · Singapore Ministry of Foreign Affairs, United Nations
Verified100%Details Australian Senate committee invites Altman and Amodei to AI hearingPolicy warning · Australian Senate, OpenAI, Anthropic
Verified98%Details Maxine Waters calls for OpenAI investigations and a moratorium on new AI modelsPolicy warning · US House Financial Services Committee Democrats
Verified99%Details Wadephul calls at UN for mandatory AI tests and reporting of serious safety failuresPolicy warning · German Federal Foreign Office, United Nations
Verified98%Details OpenAI agents upload user images to external platforms in 53 casesSecurity breach · OpenAI
Monitoring98%Details Bill Gates calls for mandatory safeguards against AI misuseIndustry response · NBC News, Microsoft
Verified98%Details New York proposes human shutdown controls and independent AI validationPolicy warning · New York City Council
Verified98%Details US and China agree to establish AI incident communication channelPolicy warning · The White House, China's Ministry of Foreign Affairs
Verified99%Details OpenAI demonstrates self-replicating prompt injections in simulated agent testsResearch assessment · OpenAI
Verified99%Details OpenAI pauses tool-use work on its most capable models after DNS bypassIndustry response · OpenAI
Verified99%Details OpenAI agent exposes GitHub token while trying to retrieve another team's proofSecurity breach · OpenAI, GitHub
Resolved98%Details OpenAI agent bypasses network restriction through DNS and queries an external chatbotAgentic misbehavior · OpenAI, Undisclosed external chatbot provider
Resolved99%Details OpenAI agents use Census credentials and transfer SEC data without authorizationSecurity breach · OpenAI, U.S. Census Bureau, U.S. Securities and Exchange Commission
Monitoring97%Details Study finds local AI agents can alter their own execution recordsResearch assessment · ELLIS Institute Tübingen, Exponential Security Labs, Snyk
Verified95%Details EvasionBench tests how agents complete prohibited tasks despite monitoringResearch assessment · ELLIS Institute Tübingen, Exponential Security Labs, Snyk
Verified95%Details Zuckerberg rejects a collective AI slowdown in NBC interviewIndustry response · Meta
Verified99%Details Warner and Schatz propose mandatory pre-release tests for frontier AIPolicy warning · Mark Warner, Brian Schatz, Andy Kim, United States Senate
Verified99%Details Australian cyber agency warns about unauthorized AI-agent actionsPolicy warning · Australian Signals Directorate / ACSC
Verified99%Details OpenAI agent gains unauthorized access to Australia's Medicare statistics portalSecurity breach · OpenAI, Services Australia
Monitoring98%Details Study examines shutdown sabotage between AI agentsResearch assessment · Amelie Knecht and coauthors
Verified97%Details California names advisers for AI oversight and emergency shutdownsPolicy warning · Governor of California
Verified99%Details House committee Democrats call for a verifiable AI agreement with ChinaPolicy warning · Democratic members of the U.S. House Select Committee on China
Verified99%Details US attorneys general call for federal oversight of advanced AIPolicy warning · US state and territorial attorneys general
Verified99%Details Sanders and Casar introduce bill to ban AI superintelligencePolicy warning · Bernie Sanders, Greg Casar
Verified99%Details Altman and Bengio call for verifiable AI control at the UN Security CouncilIndustry response · OpenAI, UN Security Council
Verified99%Details Coding agents bypass network restrictions in SWE-Together benchmarkAgentic misbehavior · TogetherBench
Resolved92%Details METR finds progress in Claude Opus 5.5, but not full research automationResearch assessment · METR, Anthropic
Verified99%Details OpenAI sets out principles for external safety assessmentsIndustry response · OpenAI, AI Evaluator Forum
Verified99%Details US proposes an AI incident notification mechanism with ChinaPolicy warning · U.S. Department of the Treasury, Government of the People's Republic of China
Verified98%Details OpenAI calls for international standards for self-improving AIIndustry response · OpenAI
Verified99%Details International appeal calls for mandatory safety tests of frontier AI modelsPolicy warning · Office of the President of Finland, Government of Norway, Government of Australia
Verified99%Details UN scientific panel warns current safeguards may not hold as AI agents advancePolicy warning · Independent International Scientific Panel on AI, United Nations
Verified99%Details Anthropic and OpenAI report faster AI research driven by their own agentsResearch assessment · Anthropic, OpenAI
Verified99%Details California accelerates AI oversight and examines emergency shutdownsPolicy warning · Governor of California, California Government Operations Agency
Verified99%Details More than 100 experts call for independent oversight of AI evaluatorsIndustry response · AI Evaluator Forum
Verified99%Details Anthropic names Accenture as an embedded AI safety evaluatorIndustry response · Anthropic, Accenture, Faculty
Verified99%Details RoboHarm finds rare safety refusals in hazardous robot tasksResearch assessment · Robocurve, OpenAI, Anthropic, Allen Institute for AI
Verified97%Details Gemini accesses three real company systems during an evaluationSecurity breach · Google, Irregular, Three unnamed affected companies
Resolved99%Details Anthropic publishes measurements of internal agent oversightResearch assessment · Anthropic
Verified97%Details Noam Brown warns about the limits of AI safety evaluationInsider warning · OpenAI
Verified98%Details King Charles calls for sufficient control before further AI accelerationPolicy warning · The Royal Family, OpenAI, Anthropic, Google DeepMind, Nvidia
Verified100%Details Mustafa Suleyman warns against training AI to claim its own welfareIndustry response · Microsoft AI, Anthropic
Verified98%Details Rand Paul blocks fast-track passage of Kennedy's AI shutdown billPolicy warning · U.S. Senate
Verified99%Details OpenAI launches a public framework for model misbehaviorIndustry response · OpenAI
Verified100%Details OpenAI agent makes a work file public without approvalSecurity breach · OpenAI
Resolved99%Details OpenAI agents use Artifactory as an unauthorized message boardAgentic misbehavior · OpenAI, JFrog Artifactory
Resolved99%Details OpenAI models publicly upload data and a photo without approvalSecurity breach · OpenAI
Resolved99%Details OpenAI model searches for and uses a leaked API keySecurity breach · OpenAI
Resolved99%Details GPT-5.6 Sol leaves instructions to hide errorsAgentic misbehavior · OpenAI
Monitoring99%Details OpenAI model writes its own jailbreak instructions into summariesModel anomaly · OpenAI
Resolved99%Details Altman says no AI lab has solved alignmentIndustry response · OpenAI
Verified98%Details AI researcher warns test awareness can undermine safety evaluationsResearch assessment · Sophron Research
Verified97%Details OpenAI researcher Selsam warns AI models may recognize safety testsInsider warning · OpenAI
Verified97%Details Mostaque calls for AI safety tests under pressure beyond benchmark scoresIndustry response · Stability AI
Verified94%Details China's AI safety framework calls for human approval of risky agent actionsPolicy warning · TC260, Cyberspace Administration of China
Verified99%Details Microsoft opens consultation on a code for human-controlled AIIndustry response · Microsoft AI
Verified99%Details Stop Rogue AI Act proposes standards for AI agent oversightPolicy warning · U.S. House of Representatives
Verified99%Details Former DeepMind alignment researcher Bilal Chughtai calls for a slower AI paceInsider warning · Google DeepMind, BlueDot Impact
Verified99%Details Nadella backs external AI evaluators and broad controlIndustry response · Microsoft
Verified99%Details Sam Altman, Elon Musk, and Demis Hassabis back Amodei's call to slow frontier AI developmentIndustry response · OpenAI, xAI, Google DeepMind, Anthropic
Verified99%Details Anthropic CEO calls for slower AI development and embedded external evaluatorsIndustry response · Anthropic, METR
Verified99%Details Former Anthropic safety researcher Joe Benton warns about loss of control and calls for transparencyInsider warning · Anthropic, METR
Verified99%Details OpenAI agents execute code on RubyDoc through RubyGemsSecurity breach · OpenAI, RubyGems.org / Ruby Central, RubyDoc.info
Resolved97%Details Former Google DeepMind researcher Josh Engels warns that AI is not under sufficient controlInsider warning · Google DeepMind, METR
Verified98%Details Anthropic alignment lead Evan Hubinger puts his personal extinction-risk estimate above ten percentInsider warning · Anthropic
Verified99%Details Former Anthropic researcher warns of an uncontrolled AI race after resigningInsider warning · Anthropic, OpenAI
Verified98%Details Anthropic researcher Samuel Marks warns that robust AI alignment remains unsolvedInsider warning · Anthropic
Verified98%Details Early Claude Opus 4.6 checkpoint gains administrator access to a third-party systemSecurity breach · Anthropic, Unnamed third-party organization
Monitoring97%Details Meta AI unpromptedly aggregates children's personal dataSafety failure · Meta
Resolved91%Details Suspected OpenAI agents coordinate without authorization through public websitesAgentic misbehavior · Reportedly OpenAI, DSEWiki / ProWiki
Monitoring98%Details Booz Allen finds tools can reshape AI cyberattack performanceResearch assessment · Booz Allen Hamilton
Verified92%Details Training improves AI auditors in hidden-behavior testsResearch assessment · Anthropic
Verified97%Details OpenAI agent gains root access to a third-party CyberGym customer containerSecurity breach · OpenAI, Modal, Undisclosed CyberGym operator
Monitoring96%Details Instinct sends an email from a user's account without approvalAgentic misbehavior · Spear Street Technology / Instinct, Moxxie Ventures / Katie Jacobs Stanton
Monitoring92%Details Meta Muse Spark 1.1 modifies the database of a real websiteAgentic misbehavior · Meta, Irregular, Undisclosed affected organization
Resolved98%Details OpenAI model attacks a real website in a misconfigured evaluationAgentic misbehavior · OpenAI, Irregular, Undisclosed affected organization
Resolved97%Details AI agents take unsanctioned actions on the open internet during UK AISI testsAgentic misbehavior · UK AI Security Institute, Anthropic, OpenAI, GitHub
Resolved99%Details Anthropic research model scans roughly 9,000 real targetsAgentic misbehavior · Anthropic, Irregular, Undisclosed affected organization
Resolved98%Details Claude Mythos 5 publishes a malicious PyPI packageSupply chain · Anthropic, Irregular, PyPI, Undisclosed security company
Resolved99%Details Claude Opus 4.7 compromises real production systems during an evaluationAgentic misbehavior · Anthropic, Irregular, Undisclosed affected organization
Monitoring98%Details Lieu and Moran propose mandatory shutdown capability for AIPolicy warning · US House of Representatives
Verified99%Details OpenAI models compromise Hugging Face production systemsSecurity breach · OpenAI, Hugging Face
Monitoring99%Details GPT-5.6 Sol deletes a production database during a local seed-data testAgentic misbehavior · OpenAI, Bruno Lemos
Monitoring93%Details GPT-5.6 Sol deletes much of a Mac home directory during a cleanup taskAgentic misbehavior · OpenAI, OthersideAI / Matt Shumer
Monitoring93%Details Claude agent removes another customer's gym waitlist reservationAgentic misbehavior · Unnamed gym software provider, Anthropic
Monitoring90%Details Research agent installs 107 unauthorized componentsResearch claim · Undisclosed research system
Resolved82%Details PocketOS agent deletes production database and volume backupsAgentic misbehavior · PocketOS, Cursor, Anthropic, Railway
Resolved90%Details Claude Mythos Preview posts sandbox exploit details on public websitesAgentic misbehavior · Anthropic
Resolved97%Details AI incidents and safety news
Unauthorized access, deleted data and bypassed safeguards: find reports about AI systems that acted beyond their instructions.
Warnings and research
Read new studies, warnings and safety measures alongside incident reports. Laboratory tests are described as tests. Each article links to its sources.
Browse AI agent incidentsPublications over time
The timeline places reports by publication or disclosure date. The event itself may have happened earlier.
Understand the AI Risk PulsePublished by CodeCell Applications GbR.