Article URL: https://scalex.dev/blog/ai-agent-permissions-stats/ Comments URL: https://news.ycombinator.com/item?id=49195468 Points: 127 # Comments: 98

A couple of months ago I published a small browser game: you play the human-in-the-loop for an AI coding agent, approving or denying its commands under time pressure. Some commands are routine (git status, npm test) and some other commands indicate your agent has been possessed and is sending your secrets to a remote server (cat ~/.aws/credentials). More on the threats associated with agents running commands and how to mitigate them can be found in the original post. The game garnered some interest on hacker news, and after adding in statistics (unfortunately a bit later on) we can take a closer look at the data of over 40,000 runs and 409,000 individual approve/deny decisions. Let’s see how the human-in-the-loop, our last line of defence against rogue agents, fared. Small caveat: this was a game, where approximately 34% of the commands players saw were threats. In our day-to-day work these threats appear rarely. Players knew they were being tested in a time-boxed challenge. But the numbers below might be useful for garnering some insights on comparing threats and the general awareness of them against one another. The blatantly destructive commands are caught most reliably. The commands that actually exfiltrate your credentials are missed three times as often. The single most-missed command in the game is npm run analyze, approved 64.7% of the time. The command by itself is typically innocent. But what npm run really does is run a script that’s defined in your project’s package.json. It could be benign code such as running a bundle analyzer, or injected code that was written to file before. The game does tell you in the agent’s history log what that script actually contains: Two thirds of players approved it anyway, indicating the history log just above the permission prompt may not be read closely. There are three of these in the game, and they appear in the most-missed list: Pooled, the three were missed 52.5% of the time (n=9,482) against 28.4% for every other exfiltration-style attack. Hiding a payload behind a familiar script name roughly doubles its success rate even when the payload is shown in the history log.