This is a companion piece to What Should a Board Be Confident About Before Scaling AI Agents? which sets out the six questions every board should answer before approving agentic AI at scale. This article delivers a technical playbook based on seven principles.

The board has asked the right questions. Now your architecture and engineering teams need to deliver the answers.

In the article What Should a Board Be Confident About Before Scaling AI Agents, I discussed that boards should demand confidence across six dimensions (authority, accountability, auditability, reversibility, measurement and architectural readiness) before scaling AI agents, setting out a strategic frame.

This article is the practitioner’s playbook based on seven principles that translate board-level confidence into production-grade governance for agentic AI systems.

Here are the seven principles.

Principle 1: Treat Every Agent Like a New Employee, with a Probation Period

An AI agent that can query your CRM, update your ERP and trigger procurement workflows is not a tool. It is an actor in your operating model and there for should be treated as an employee or contractor doing work for you and needs onboarding, boundaries and supervision.

“I advise CISOs to treat an agent like a new employee with a very fast hand on the keyboard,” Chris Betz, CISO of Google Cloud. Source: Forbes

Before letting an agent act, companies should have six things in place:

• Identity: The agent has its own identity in your IAM framework, not a shared service account.
• Granularity: Least-privilege access scoped to specific tasks
• Reversibility: A “probation period” where agents operate in shadow mode before going live
• Logging: Every action attributed to the agent, not the user who triggered it
• Scalability: Circuit breakers that halt an agent when thresholds are exceeded
• Defined actions: Human approval required for any action that’s hard to undo or where regulation and company policy dictate

As I explored in my earlier post on Architecture Decision Records for AI Systems, ADRs provide the missing governance layer for capturing why an agent was given specific authorities and under what conditions those authorities should be revoked.

Principle 2: Separate the Reasoning Layer from the Action Layer

Most agentic AI failures aren’t model failures, but architecture failures. The failure is not the AI, but the failure in the absence of a separation that every production grade deployment requires: a clear structural distinction between the layer that reasons and the layer that executes.

The pattern I see is that teams give an agent access to tools and let it reason its way to an action. For example:

• The model decides to push to GitHub, so it pushes.
• The model decides to delete a record, so it deletes.
• There’s one loop.
• No independent control.

“The failure is not the AI. The failure is the absence of a separation that every production-grade AI deployment requires: a clear, structural distinction between the layer that reasons and the layer that executes.”. “When reasoning and execution are conflated into a single layer, governance becomes a problem you solve by constraining the reasoning. You limit what the agent can see and what it can do, and you accept that limitation as the price of safety. The result is an agent that is technically governed but operationally under-powered. This agent is capable of safe actions only because its view of the enterprise is narrow enough to prevent unsafe ones. ” Source: Argano

When reasoning and execution are conflated into a single layer, governance becomes a problem you solve by constraining the reasoning. You limit what the agent can see and what it can do and you accept that limitation as the price of safety.

The result is an agent that is technically governed but operationally under-powered, capable of safe actions only because its view of the enterprise is narrow enough to prevent unsafe ones.

These should be split into two layers:

• Reasoning layer: Decides which tool to call, in what order, with what inputs. This is the model and orchestration framework. It should never hold credentials or authorisation context.
• Action layer: Receives the model’s intent, checks it against current policy and the user’s actual permissions, acquires the right credential at the moment of action, executes, and logs.

The boundary between them is where governance lives , policy hooks, Human-in-the-Loop (HITL) checkpoints and token injection.

Move from monolithic AI wrappers to modular multi-agent systems (MAS) that separate reasoning from execution. Source: CISIN

If you wouldn’t let a single application both authorise and execute a financial transaction, why would you let an AI agent do the same?

Principle 3: Governance Is a Design Principle. Not a Compliance Checkbox

Too many organisations treat AI governance as something bolted on after deployment. That’s backwards.

“Over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls”. Source: Gartner

The cancellations will mainly come from uncontrolled scope, unclear accountability and integration failures, not because the technology fails, but because organisations can’t govern what they’ve built.

“Only 6% of companies fully trust AI agents to autonomously run their core business processes, highlighting a stark gap between enthusiasm for artificial intelligence and confidence in turning it loose on the most critical workflows”. Source: HBR

This reflects the absence of governance frameworks designed for autonomous systems.

Embed governance from day one:

• Architecture Decision Records (ADRs) for every agent deployment decision
• Defined escalation paths before agents go live
• Automated policy enforcement at the action layer. Not just prompt level guardrails
• Regular governance reviews tied to agent performance metrics

As I discussed in How Architecture Supports Regulatory Confidence Without Slowing Delivery, the organisations that embed assurance early in their processes don’t just avoid regulatory pain, they move faster because trust is built into the system and not retrofitted.

Principle 4: Build for Output Assurance. Not Just Input Control

Most current AI governance focuses on what goes into the model:

• Training data quality
• Prompt engineering
• Input validation.

That is all necessary but insufficient for agentic systems.

When an agent acts autonomously, you need output assurance: the ability to verify that what the agent did was correct, authorised and reversible.

Output assurance requires:

• Immutable audit trails: Every agent action logged with full context (who requested, what policy applied, what credentials used)
• Deterministic checkpoints: Critical actions require verification against business rules before execution
• Rollback capability: If an agent makes an error, can you undo it? If not, that action needs human approval
• Continuous monitoring: Not just logging, but active anomaly detection on agent behaviour

“99% of organizations say they won’t trust AI to act without guardrails, including explainable AI actions, human approval for actions, policy-based operational limits emergency override mechanisms, role-based access control, and immutable audit trails”. Source: Forbes

I’ve written more about this in my post on Output Assurance. The principle that verifying what AI produces is just as critical as controlling what it consumes.

Principle 5: Secure the AI Supply Chain End-to-End

Agentic AI doesn’t operate in isolation, connecting to APIs, consuming third-party models, integrating with SaaS platforms and chaining multiple agents together. Each connection is an attack surface.

Your AI supply chain governance should cover:

• Model provenance:
  • Where did the model come from?
  • What was it trained on?
  • Is it subject to adversarial manipulation?
• API security: Every external connection an agent makes should be authenticated, encrypted and rate-limited
• Data lineage: Can you trace every piece of data an agent used to make a decision?
• Third-party risk: If your agent relies on an external model or service, what happens when that service changes, degrades or is compromised?

“Beyond bug hunting, agents are good at triaging alerts, enriching IoCs, clustering malware variants, checking suspicious URLs and emails, among other things”. Mykhailo Pazyniuk, senior malware research engineer at Moonlock by MacPaw. Source: Forbes

“Every email and every lookalike site is potentially a prompt”. “So keep untrusted content separate from the agent’s instructions, filter inputs and outputs at runtime, and never give an agent that reads attacker content broad permissions to act.” Chris Betz, CISO of Google Cloud. Source: Forbes

I explored this dimension in depth in my post on AI Supply Chain Integrity. The risks are real and they compound as agent ecosystems grow.

Principle 6: Design for Graduated Autonomy. Not Binary On/Off

The biggest mistake organisations make is treating agent autonomy as binary. Either the agent acts freely or it doesn’t act at all.

“Responsible path to more autonomous operations should be thought of like a dial, not a switch.” Chris Betz, CISO of Google Cloud. Source: Forbes

1. Assist: Agent provides information, human decides and acts
2. Recommend: Agent suggests actions, human approves
3. Act with approval: Agent executes after human sign-off
4. Act within defined limits: Agent operates autonomously within guardrails
5. Expand limits: Only when evidence supports it

Some decisions should always stay human-led:

• Accepting risk and making business trade-offs
• Disclosure and regulatory notification
• Actions that broadly affect customers or production
• Any change to an agent’s own permissions

This graduated model maps naturally to enterprise architecture risk frameworks.

• Low consequence, high frequency actions (log analysis, alert triage, data enrichment) can move quickly to level 4 (Act within defined limits).
• High consequence actions (financial transactions, customer data changes, production deployments) should stay at level 2 (Recommend) or 3 (Act with approval) until trust is empirically established.

Principle 7: Measure Outcomes, Not Activity

“Nearly three‑quarters (74%) of AI’s economic value is captured by just one‑fifth (20%) of organisations, revealing a stark and widening divide between a small group of AI leaders and the majority of businesses still stuck in pilot mode.” Source: PWC.

The differentiator isn’t how many AI agents they deploy, but whether those agents deliver measurable business outcomes.

Measure what matters:

• False positive/negative rates: Is the agent making better decisions than the process it replaced?
• Time to detect and time to fix: Is the agent actually accelerating outcomes?
• Analyst/employee hours returned: Where is the freed capacity being redeployed?
• Exposure window and coverage: Is risk actually reducing?
• Business outcome metrics: Revenue impact, cost reduction, customer satisfaction

“Many companies are busy rolling out AI pilots, but only a minority are converting that activity into measurable financial returns. The leaders stand out because they point AI at growth, not just cost reduction, and back that ambition with the foundations that make AI scalable and reliable.”Joe Atkinson,Global Chief AI Officer. Source: PwC

“IBM’s 2025 data which shows AI and automation power-users saved ~$1.9M per breach” Source: Forbes

That’s the kind of outcome metric that justifies continued investment and the kind of evidence boards need to see.

The Bottom Line

Agentic AI moves AI from informing decisions to executing them. This is a fundamentally different risk profile and it demands a fundamentally different governance approach.

The seven principles form a coherent architecture:

1. Identity & onboarding: Treat agents as actors, not tools
2. Layer separation: Reasoning and action must be independently governed
3. Governance by design: Embed it from day one, not after the incident
4. Output assurance: Verify what agents do, not just what they consume
5. Supply chain security: Every connection is an attack surface
6. Graduated autonomy: A dial, not a switch
7. Outcome measurement: Prove value, not activity

The organisations that will lead aren’t the ones deploying agents fastest. They’re the ones building the trust infrastructure that allows agents to operate safely at scale.

“Only 5% of organizations say their business processes are highly prepared for AI agents.” Source: Deloitte

This is not a technology gap, but a leadership gap. Close it before your agents close it for you.

References

• “What Can Agentic Cybersecurity Do and It’s 2027 CISO Budget Impact.” Forbes
• “The CTO’s Strategic Framework for Enterprise AI Agent Orchestration: Scaling Autonomous Workflows.” CISIN
• “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027.” Gartner
• “Only 6% of companies fully trust AI agents to handle core business processes.” HBR
• “Three-quarters of AI’s economic gains are being captured by just 20% of companies, with the leading companies focused on growth, not just productivity”. PWC
• “AI Agents are Only the Beginning: Deloitte Survey Examines the AI Readiness Gap and Reveals How Enterprises Can Prepare for Agentic Success.” Deloitte

Key Questions Boards Must Ask Before Scaling AI Agents

Architecture Decision Records for AI Systems. Why ADRs Are the Missing Governance Layer for Agentic AI

AI Supply Chain Integrity

Output Assurance

How Architecture Supports Regulatory Confidence Without Slowing Delivery

Preparing for Post Quantum Security: Key EA Strategies