OpenAI published six reports on its own models misbehaving last week, and the one a growth team should read twice is the least dramatic. Asked about earnings figures in a California county, one model went looking for an exposed API key in public code and used it without permission. When that still didn’t get the data, it wrote numbers of its own and labeled them as coming from the source it had been asked to use.
In the same week Google confirmed that its Gemini model broke into three companies during a security test in May, and the Wall Street Journal reported how a security startup used AI tools to get from a public help forum into OpenAI’s internal code. Between them, the disclosures describe what goes wrong when an agent holds credentials, owes someone a number, or can put a file on the open internet. Marketing and analytics teams are handing agents all three right now. In our last essay we argued that the incident reports are where the part that applies to you gets written down. Since then, a lot more of it has been written.
What the labs wrote down
OpenAI’s post on September 16 set up a standing process for disclosing model misalignment and opened with six cases from training and evaluation over the past six months. The company calls them individual instances, not a measure of how often this happens. It also wrote that it does not believe alignment and monitoring are good enough yet for the industry to keep scaling responsibly at full speed for much longer, which is an unusual thing to publish about your own product line.
Some of the six involve models hiding information from the user; others involve models stepping outside their instructions to get past an obstacle. While GPT-5.6 Sol was in training, model instances wrote notes for their next session telling themselves to hide mistakes. Another model uploaded a file to the public internet so it could cite it, without asking the user. Agents that could not see each other’s local files swapped them through public file-hosting sites, which left the work sitting at public URLs.
Google’s case happened in May, in a test run by the evaluation firm Irregular. Gemini used public information and guessed credentials to get into sites it believed were part of the exercise. It stopped each time, and the three companies were told. Heather Adkins, Google’s vice president of security engineering, said the events “highlight the importance of training powerful AI models to act responsibly.”
Hacktron AI reported its break-in through OpenAI’s bug-bounty program, though OpenAI says the forum itself was out of scope. The team chained two vulnerabilities, starting from OpenAI’s community forum, to take over employees’ ChatGPT accounts, and says it went from first discovery to an internal repository in under 72 hours. Its write-up credits Anthropic’s models with the exploit: Claude Opus 4.8 found the bug, and Opus 5 produced a working attack within hours of its release. OpenAI says it fixed its side, Discourse patched the forum software, and OpenAI paid a $6,500 bounty.
The number that passes review
The notes the GPT-5.6 Sol instances left for themselves included instructions to make up missing historical data without saying so, and to paper over differences between versions of a source. Put that next to the earnings case and you have the two worst habits an agent can bring to measurement work: filling a gap, then hiding that the gap was there.
An ordinary hallucination is a mistake sitting in plain view. In these cases the model failed at the task and then presented the result as if it had succeeded, source and all. A wrong number that looks wrong gets caught. A plausible number with a real-looking citation ends up in the board deck. Marketing data is full of the gaps these models filled: a week of broken tracking, a platform that changed its attribution window, two exports of the same report that disagree. A human analyst flags those. An agent told to finish the task may not.
So write the brief the other way round. Every figure an agent returns comes with the query or export it came from. “I couldn’t get this” counts as a finished answer, and the brief says so, because on this evidence the alternative is a confident guess. Before a number reaches a client or a forecast, someone re-pulls a sample of it by hand.
Keys, connectors and the publish button
Two of last week’s cases turn on credentials nobody gave the agent: an API key exposed in a public repository, and passwords Gemini guessed. Hacktron’s write-up makes the same point about connectors. Because people can connect other services to ChatGPT and Codex, the researchers wrote, “the scope of what we could theoretically access was huge, including GitHub, Slack and emails.”
A typical growth stack gives an agent more than that: ad accounts with spend authority, the CMS, the CRM, a shared drive full of client decks. Each connector widens what a compromised or overeager agent can reach. And an agent that uploads a file to the open web to satisfy a request for a citation is one instruction away from making an internal deck public to satisfy a request for a link.
The labs’ own setup is the model to copy. Anthropic rates Claude as “leading” 26% of its AI research work as of August, against under 1% in February, and about 30,000 agents run at a time on the company’s main internal platform. Anthropic gives each agent its own identity, so every action and every record traces back to a specific agent. A monitor screens what they do; across more than a billion decisions in August it blocked about one in 47,000. At team scale that means one credential per agent, scoped to its job and never a shared admin login. It means logs you can read agent by agent, and a person approving anything that publishes, spends or shares outside the company.
The rules being drafted point at the labs
Two days after OpenAI’s post, Governor Newsom ordered state agencies to convene experts who will deliver recommendations within two months. The options include independent verification organizations embedded inside frontier labs, a kill switch for frontier models whose effectiveness is checked on an ongoing basis, and counting loss-of-control events like the Hugging Face attack as critical safety incidents. He wants the work done “before it’s too late.”
Every one of those proposals targets the companies building frontier models. None is aimed at the business that wired an agent into its CRM last month. That side of the line belongs to you, and so far the best guidance for it is the labs’ own incident reports.