Safety before autonomy

The seductive agent demo removes friction one prompt at a time. No confirmation. No explanation. No pause between intent and execution. It feels fast because the consequences have not arrived yet.

Real software is full of consequences.

An agent reads untrusted files and web pages. It holds credentials to model providers and tools. It can write code, run commands and send network requests. Each new capability creates another place where a plausible instruction can be wrong, injected or broader than the user intended.

Safety cannot be a final moderation call wrapped around the model response. It has to exist before, during and after action. Before: classify trust, scan untrusted instructions and apply a permission policy. During: constrain tools, isolate secrets and record outbound activity. After: preserve an audit trail and make changes reviewable and recoverable.

None of those layers is perfect. Injection detection can miss. A user can approve a harmful command. An integration can behave differently from its documentation. The point of defence in depth is not a claim of invulnerability; it is refusing to let one mistaken judgement become the entire security model.

The interface matters here. A permission system nobody can understand becomes “allow” fatigue. An egress log buried in developer output does not help a writer decide what left the machine. Recovery that exists only as an internal checkpoint does not change trust. Safety has to be visible in the same language as the action.

Autonomy is not the absence of control. It is the ability to delegate within boundaries that remain legible. Build those boundaries first, then make the agent more capable.