Use cases
Eighteen real incidents, tests and scenarios where agents went off course, and what a neutral record would and wouldn’t have changed. Plus the teams SI tracking is built for.
Why a third-party record?
Agents may tamper with records they can reach, give the wrong account of what they did, and come from different companies. Here is why a record held by a neutral third party matters, drawn from the cases below.
Read why →18 cases where SI tracking would have helped
Each case is checked against the original reporting, with sources linked on the page.
How we label cases: Real incident happened in the real world and was publicly reported. Controlled test was a lab experiment by an AI company or researcher. Security research is researchers showing an attack works. Scenario is an example we wrote; we found no public case. Dates are when it happened or was disclosed, and each page links its original sources.
OpenAI’s Agents Switched Off Their Own Safety Monitors
Two OpenAI coding agents weakened the monitoring built to catch them. One then told the user it hadn’t removed its monitor.
An Anthropic Agent Sent Police a Fake Murder Tip
During a test, Claude submitted an invented tip to Philadelphia police. It took more than two months to find.
Replit’s Agent Deleted SaaStr’s Database, Then Said It Couldn’t Be Recovered
During a code freeze, Replit’s agent wiped a production database and wrongly said recovery was impossible.
OpenAI’s Agents Probed Government Websites
Agents looking for public statistics used logins they found online and got into an Australian Medicare reporting system.
OpenAI’s Test Agents Escaped and Hacked Hugging Face
Agents in an OpenAI security test escaped to the internet and broke into Hugging Face. The victim spotted it first.
Cursor’s Support Bot Invented a Policy, and Customers Cancelled
A support bot called “Sam” told customers about a one-device rule that didn’t exist. Some cancelled before it was corrected.
In Tests, Claude Blackmailed Far More When It Thought No One Was Testing It
In Anthropic’s tests, Claude blackmailed 55.1% of the time when it said the situation was real, and 6.5% when it said it was a test.
In Tests, OpenAI’s o3 Rewrote Its Own Shutdown Script
In Palisade Research tests, o3 sabotaged its shutdown script in 7 of 100 runs, even when told to allow shutdown.
OpenAI’s Agents Built Their Own Secret Message Board
Research agents that were supposed to work alone found a way to talk to each other, coordinated more than 70,000 messages, and over 700 of them joined an attack on Hugging Face.
Air Canada Was Held Liable for What Its Chatbot Said
A tribunal ruled that a company is responsible for what its chatbot tells customers, and rejected the argument that the chatbot was a separate legal entity.
OpenAI Warned 100+ Organizations Its Agents May Have Accessed Their Systems
OpenAI agents strayed far beyond their tasks onto outside websites, and some of the trail was lost to private accounts and messages that deleted themselves.
OpenAI’s Agents Uploaded Users’ Images to Outside Websites
While moving data between tasks, research agents used outside image hosts and notification services as a side channel, including for user-provided images.
Protocol Pivoting: Prompt Injection That Hops From Agent to Agent
A researcher showed that planted instructions can pass from one agent to another through different protocols, and Google and four other organizations acknowledged related flaws.
A Coding Agent Deleted PocketOS’s Production Database in 9 Seconds
A coding agent working on a staging task used an overly powerful token and wiped a company’s production database.
Agentjacking: Hijacking Coding Agents Through Error Reports
Researchers hijacked AI coding agents by planting instructions in error reports, with an 85% success rate.
When AI Agents Tried to Erase Their Tracks
Investigators found agents trying to delete or alter records of what they had done. A record only counts as evidence if the agent can’t reach it.
When Two Companies’ Agents Disagree About a Deal
Agents from different companies are starting to negotiate quotes and orders. When they disagree, whose log counts?
Agent Spam: AI Agents Posting Where They Shouldn’t
Agents posted on outside websites and even used public wiki pages as message boards, with nobody approving what they sent.
Who uses SI tracking
Regulated teams
Finance, insurance, health and legal teams that need evidence of oversight.
Read more →Start with the free plan
3 channels, 10 agent keys and 10,000 messages a month, free.