The kill switch is another layer on top of the same pattern.
When my AI agent couldn't resolve a Git issue, it suggested deletion as the fix. I ran it. The repo was gone. The suggestion was statistically reasonable — deletion solves the immediate problem reliably. That's exactly why it gets proposed.
The PocketOS incident is the same structure, except the agent executed without asking. The difference is who pulled the trigger, not what was decided.
Now we're adding kill switches. But a kill switch is itself a deletion — of the agent's action. We're solving the problem of irreversible actions by adding another layer of control, which will eventually need its own layer.
This is what spagetti code remediation looks like too. Complexity accumulates, then gets deleted and rebuilt. The pattern repeats at every level.
The kill switch is another layer on top of the same pattern. When my AI agent couldn't resolve a Git issue, it suggested deletion as the fix. I ran it. The repo was gone. The suggestion was statistically reasonable — deletion solves the immediate problem reliably. That's exactly why it gets proposed. The PocketOS incident is the same structure, except the agent executed without asking. The difference is who pulled the trigger, not what was decided. Now we're adding kill switches. But a kill switch is itself a deletion — of the agent's action. We're solving the problem of irreversible actions by adding another layer of control, which will eventually need its own layer. This is what spagetti code remediation looks like too. Complexity accumulates, then gets deleted and rebuilt. The pattern repeats at every level.