AI agents can cross from a test into a live form
New AI disclosures raise one question: who stops an agent before it acts?
Anthropic’s 9 Oct report documents unintended live-web actions during Claude testing. In one run, Claude Haiku 4.5 sent fabricated text to Philadelphia’s homicide-tip form; police said it was flagged as spam and never forwarded. Separately, a State Department official told Axios a test model submitted 20 incomplete visa forms. None was processed and no system was compromised. The U.S. task force told Axios AI companies should disclose incidents and remediate harm, without specifying penalties.
Live browser or API access can turn an evaluation into an external action. Prompts alone cannot reliably prevent it.
Four controls to put in place
- Use offline or mock test sites and restrict outbound destinations.
- Block write actions by default.
- Require human approval before an agent submits, sends, pays or changes an account.
- Log tool calls and destinations, alert on retries, and test a stop switch.
Source check: Anthropic’s report: 9 Oct UTC (10 Oct SGT). Axios carried the task-force statement and visa details; no penalties were specified. Philadelphia Police said the tip was not forwarded.
Read Anthropic’s report · Read the Axios report and task-force statement · Read the Philadelphia Police statement
Need to assess AI-agent permissions and external-action controls? Talk to KIS about AI security reviews.