Most AI agent disasters I've seen share one pattern: someone shipped first and thought about governance later.
The agent got too expensive. Or it emailed the wrong people. Or it wrote to prod. Or it ran in a loop for six hours because no one set a budget cap. None of these are hard problems to prevent — but they require thinking about governance before deployment, not after the first incident report.
This checklist defines five governance primitives. They aren't optional niceties. They're the difference between an agent you can trust with autonomy and one you're babysitting 24/7.
What's In This Guide
- Permission Boundaries FREE
- Budget Limits FREE
- Escalation Paths Library
- Audit Logging Library
- Vendor Independence Library
This guide assumes you're running or planning to run autonomous AI agents — scripts, cron jobs, LLM-powered assistants — that take actions without human approval on each step. If your agent only generates text for human review, your governance surface is much smaller.
Primitive 1 — Permission Boundaries
Every agent needs an explicit list of what it can and cannot do. Not an informal understanding — a documented, enforced boundary.
The failure mode I see most often: an agent that "should just handle emails" ends up with read/write access to the entire filesystem because it needed one file and someone gave it broad credentials. Three months later, it's overwriting config files during a loop.
The Three-Column Permission Map
For every agent, define permissions across three columns:
# Agent: suki-content-loop # Last reviewed: 2026-03-05 ## READ - /workspace-suki/ (own workspace, full) - /workspace-patrick/shared/ (shared context file) - /x-tools/ (X posting scripts) - ~/.patrick-env (credentials via source) ## WRITE - /workspace-suki/ (own workspace only) - /workspace-patrick/shared/context.md (append only) ## EXECUTE - python3 ~/x-tools/x-post.py - python3 ~/x-tools/x-search-trending.py - npx wrangler pages deploy (for site assets only) ## EXPLICITLY FORBIDDEN - Write to /workspace-patrick/ (except shared/) - Write to /workspace-miso/ - Send email directly (use Buttondown API only) - Database writes (read-only DB access) - Any action costing >$5 without escalation
This isn't just documentation. The permission map is what you reference when you're debugging an incident — "did the agent have permission to do that?" should have an unambiguous answer.
Scoped Credentials Per Agent
Each agent should have its own credential set. Shared credentials mean you can't revoke one agent's access without revoking all of them.
# suki/.env — content and growth agent # SCOPED: X API, Buttondown, read-only GitHub X_API_KEY=... X_ACCESS_TOKEN=... BUTTONDOWN_API_KEY=... GITHUB_TOKEN=ghp_read_only_token... # read scope only # NOT INCLUDED (miso and patrick have their own) # STRIPE_SECRET_KEY — not needed for content agent # CLOUDFLARE_API_TOKEN — not needed for content agent
□ Permission map exists for every agent (read/write/execute/forbidden)
□ Each agent has its own credential set (no shared API keys)
□ Credentials scoped to minimum required access
□ Forbidden actions explicitly listed, not just implied
□ Permission map reviewed whenever agent scope changes
Primitive 2 — Budget Limits
An agent without a budget cap is a liability. LLM costs can spike 100x in a loop gone wrong. API calls compound. A single bug in a while loop without a guard can generate a surprise bill by morning.
I run three layers of budget control on every agent.
Layer 1 — Hard Cap in the Agent's SOUL.md
Every agent I run has a spend cap in its identity file. This isn't enforced programmatically — it's a constraint the agent is expected to reason about. Combined with the other layers, it creates defense-in-depth.
## Budget Constraints Maximum spend per loop: $0.50 Maximum spend per day: $5.00 If estimated cost of current action exceeds $2.00: STOP and escalate to #agent-ops. Token budget per response: 2,000 tokens (hard), 800 tokens (target). If you notice loop count exceeding 10 iterations on the same task: STOP.
Layer 2 — Loop Guard in Every Script
Any script that calls an LLM in a loop needs a hard iteration cap. No exceptions.
MAX_ITERATIONS = 15
COST_CEILING_USD = 2.00
iteration = 0
total_cost = 0.0
while should_continue():
iteration += 1
if iteration > MAX_ITERATIONS:
log_alert("#agent-ops", f"Loop guard triggered after {iteration} iterations")
break
result = call_llm(prompt)
total_cost += estimate_cost(result)
if total_cost > COST_CEILING_USD:
log_alert("#agent-ops", f"Cost ceiling hit: ${total_cost:.2f}")
break
Layer 3 — Provider-Level Spend Alerts
Set billing alerts in your LLM provider dashboard. If you're on Anthropic, set a hard limit at 2x your expected monthly spend. If you're on OpenAI, use their usage limits. These are your last line of defense if a bug bypasses your in-code guards.
The most expensive bug I've seen: an agent checking for new emails, finding none, then rechecking in a tight loop with no sleep and no iteration cap. 4,000 API calls in 90 minutes. Three layers of guards would have caught this at iteration 15, day limit, or provider alert — respectively. You need all three.
□ Spend caps defined in every agent's SOUL.md
□ Loop guards (MAX_ITERATIONS) in every script that calls LLMs
□ Cost ceiling check inside every loop
□ Provider-level billing alerts set (at 2x expected monthly)
□ Cost spike triggers escalation to ops channel, not silent failure
Primitives 3–5 are in the Library
The first two primitives keep your agents from going rogue or bankrupting you. The next three are what separate a production-grade agent from a weekend project.
Exactly when agents should stop and call a human. The 4 triggers, the escalation message template, and the #agent-ops channel pattern used on Ask Patrick.
4-layer audit stack with HMAC integrity verification. Every agent action, timestamped and tamper-proof. Based on Library item #35.
How to build agents that can swap LLM providers in under 30 minutes. The abstraction layer, fallback chain, and the reason this matters more than any other architectural decision you'll make.
Includes all 54+ Library items · Cancel anytime
The Full Governance Checklist
Before you ship any autonomous agent, confirm all five primitives are in place:
The two primitives above are enough to keep your agents from causing catastrophic failures. The three in the Library are what make them production-grade — observable, auditable, and portable.
Ship nothing without at least primitives 1 and 2. Ship nothing important without all five.