AI Agent Governance Framework: Permissions, Logs and Human Approval
Five controls to write down before any agent touches live systems: permissions, approval gates, logging, escalation, rollback.
In this guide
Businesses preparing to let AI agents touch tools, data, or live workflows need operating rules in place before the first agent runs — not after the first incident. This page is the framework: five controls, each written down, each owned by someone.
1. Permissions — what the agent can touch
List every tool, system, and data source the agent can access, and deny everything else by default. Scoped access is the single biggest risk reducer: an agent that can only read the knowledge base and draft replies cannot delete records or email customers, whatever goes wrong in its reasoning.
2. Approval gates — what needs a human
Decide which actions the agent may take autonomously and which require human sign-off. The practical default for SMEs: anything that leaves the building — customer messages, payments, record changes in systems of record — gets a human approval step. Internal drafts and summaries usually do not.
3. Logging — what happened and why
Every agent action should be logged with enough context to reconstruct what it did and what it acted on. Without logs there is no debugging, no audit trail, and no honest answer when someone asks "why did it do that?"
4. Escalation — when the agent must stop
Define the conditions under which the agent hands the work to a person: confidence below a threshold, a customer expressing frustration, anything touching legal, HR, or compliance territory. Escalation rules keep sensitive judgement human without making the agent useless.
5. Rollback — undoing what it did
Before the agent acts on real systems, know how to reverse each action type. If an action cannot be reversed, it belongs behind an approval gate. Rollback thinking also forces the right architecture: agents acting through systems with histories and undo, not raw database access.
Using the framework
Write the five sections for one specific agent use case — a page or two, not a policy binder. Review it with whoever owns the affected workflow, run the agent narrowly for 30–90 days, and revisit the rules with the logs in front of you. This pairs with our guide to when agents are worth building and the broader staff AI policy.
If you want help writing the rules for a real use case, book a free consultation with Blue Canvas.
If this is the kind of work you want help with, we can build and run this for you, or book a free consultation.
Phil Patterson · Founder, Blue Canvas AI
Phil runs Blue Canvas AI, a Derry-based consultancy helping UK and Irish SMEs scope, train for, and implement practical AI workflows.
FAQ
Frequently asked questions
Do small businesses really need agent governance?
If an agent touches live systems or customer data, yes — but proportionate: a page or two covering permissions, approvals, logging, escalation, and rollback for the specific use case, not an enterprise policy binder.
What should always require human approval?
Anything that leaves the building or cannot be undone: customer communications, payments, changes to systems of record, and anything with legal, HR, or compliance weight.
When should the rules be reviewed?
After the first 30 to 90 days of narrow operation, with the logs in front of you — and again whenever the agent gains a tool, a data source, or a new action type.