A question boards should be asking at the moment is: “Should we be scaling AI agents?”
The honest answer is yes, but not until you can answer six questions with confidence.
Agentic AI (systems that don’t just recommend but act) are the most significant shift in enterprise technology since cloud computing. Gartner is currently projecting that 40% of enterprise application will embed task specific AI agents by the end of 2026. Source: Gartner
IDC forcasts “By 2029, IDC, projects 1.15 billion active agents, 40 times the 2025 baseline, executing 217 billion actions per day. Source: IDC
The momentum on scaling AI Agents is real, but so is the gap between ambition and readiness. This is evidenced in Deloitte’ s 2026 State of AI in the Enterprise report: “Business and IT leaders report AI agents are scaling faster than their guardrails”.
Challenges to scale: Only 5% of organizations say their business processes are highly prepared for AI agents, and just 15% have scaled orchestrated, cross-functional multi-agent adoption. Even among the more mature AI adopters, challenges persist: Fewer than half (46%) of scaled adopters believe their business processes are prepared for agentic adoption. Source: Deloitte
Nearly three‑quarters (74%) of AI’s economic value is captured by just one‑fifth (20%) of organisations, revealing a stark and widening divide between a small group of AI leaders and the majority of businesses still stuck in pilot mode. Source: PWC
The differentiator in all of these isn’t technology. There are plenty of AI “solutions” available and more being developed at an fast rate. The differentiator is governance, trust and operational readiness which are board level concerns.
The Risk Has Changed Shape
As AI has evolved, the model has shifted. Traditional AI gave you a recommendation and a human decided what to do with it. However Agentic AI can now break that chain. An agent that can query your CRM, update your ERP, trigger procurement workflows and escalate customer issues automatically. This is not a tool, but an actor in your operating model and one that can make decisions and execute them, sometimes in milliseconds.
The principle of HITL and HOTL (Human in the Loop, Human on the Loop) should still be observed as best practice and an absolute must where high impact decisions are made. Regulations and standards also require this.
Only 6% of companies fully trust AI agents to handle core business processes. Source: HBR
The trust deficit isn’t irrational; it reflects a real governance gap. Organisations lack frameworks to oversee systems that act autonomously. The problem isn’t the technology; it’s that boards haven’t built the guardrails to govern what they’re deploying
Six Questions Every Board Should Be Able to Answer
Before approving the next wave of agentic AI investment, your board should have clear, confident answers to these questions:
1. Do We Know What Our Agents Are Authorised to Do?
Every AI agent must have a defined scope of authority:
- What data it can access?
- What actions it can execute?
- What decisions require human review?
If you can’t document this for each agent on a single page, you don’t have governance, you have hope.
“I advise CISOs to treat an agent like a new employee with a very fast hand on the keyboard. Before letting an agent act, companies should have six things in place. identity, granularity, reversibility. Logging, scalability (circuit breakers), and defined actions.” Chris Betz, CISO of Google Cloud. Source: Forbes
Google Cloud’s own governance guidance reinforces this:
Part of mitigating shadow agent risk is embracing strong AI governance protocols. For agent security to be effective, security and business leaders should govern and deploy their agentic tools with the same rigor applied to human-managed accounts, cloud infrastructure, and enterprise software. Source: Google
The direction is clear:
- Agents get identities
- Identities get governed
- Governance gets audited
Board question: “For every AI agent we’ve deployed, can you show me a one page authority document that defines what it can access, what it can do, and where it must stop and ask a human?”
2. Can We Trace What an Agent Did and Why?
Traditional software leaves a clean audit trail with a linear sequence and straighforward evidence; a user logged in, clicked a button and submitted a form.
AI agents operate differenty; an agent receives a prompt, reasons across multiple data sources, selects tools, chains decisions together and executes (in seconds to milliseconds and across multiple systems. The path from input to action is neither linear nor self-documenting.
AI governance and accountability matter because the regulatory, financial, and reputational consequences of ungoverned AI are no longer hypothetical. According to Grant Thornton’s 2026 AI Impact Survey of 950 business leaders, 78% lack strong confidence they could pass an independent AI governance audit within 90 days. That gap between deployment and defensibility is exactly what the rest of this guide addresses. Source: Dataiku
Most organizations it surveyed said they couldn’t verify what their AI agents actually do across business systems, even though more than a third had already deployed agents inside finance and accounting functions. Source: IDC
The risk is that AI Agents are making decisions on revenue recognition, customer pricing and regulatory reporting have no reconstructable evidence chain.
Global governance is converging on traceability. Different regions use different mechanisms, but the direction is consistent. For high-impact AI, you need documentation, auditability and proof that you can investigate and correct issues. Source: IBM
What the audit trail must capture:
- The prompt: What instruction triggered the agent
- The reasoning path: Which data sources were consulted, which tools were invoked and in what sequence
- The decision: What the agent chose to do and at what confidence level
- The authority: Whose credentials authorised the action and what policy applied
- The outcome: What changed in the target system and whether it was reversible
When an auditor asks how your organization governs AI access to sensitive data, the answer they are looking for is not a policy document. It is evidence. Evidence that access controls were technically enforced. Evidence that every data access event was attributed to a responsible individual. Evidence that the policies your documentation describes were actually operating as described and that the audit log is the record of that operation, not a post-hoc narrative. Source: Kiteworks
Board question: “If an AI agent made a decision last Tuesday that affected a customer, a financial position, or a regulatory obligation — can you show me the complete chain? What data did it use? What policy applied? Whose credentials were involved? What was the outcome?”
3. Can We Stop an Agent Immediately If Something Goes Wrong?
Every agent needs a kill or off switch. Not a “we’ll review it in the next sprint” process. A real time circuit breaker that halts execution when thresholds are breached.
When an agent goes wrong, the consequences are immediate and sometimes irreversible.
Enterprises need this now because agentic AI has quietly crossed a threshold: agents no longer just draft an email or summarize a document; they send it, update the database, or push code to production on their own. Source: SoluLab
The major AI companies now explicitly recommend Human-in-the-Loop (HITL) controls for regulated and high impact scenarios. This isn’t optional guidance but the emerging industry standard.
Regulatory pressure is accelerating HITL adoption: the EU AI Act, effective August 2024, requires human oversight for high-risk AI systems covering an estimated 85,000 enterprises operating in the EU, and similar mandates are expanding in financial services, healthcare, and public sector worldwide (European Commission, 2025). Source: Stealth Agents
Board question: “If an agent starts behaving unexpectedly at 2am on a Sunday, who gets alerted, what stops it, and how fast?”
4. Who Is Accountable When an Agent Gets It Wrong?
This is the question most organisations haven’t answered. When an agent makes an error (and it is possible) who owns the consequence:
- The data scientist who trained it?
- The architect who designed the integration?
- The business owner who approved the use case?
- The vendor who supplied the model?
- The board?
“If something goes wrong with our models during training, there’s gonna be some version of that we need to be responsible for,” Altman said. “There’s gonna need to be a liability framework for companies.” OpenAI CEO Sam Altman. Source: Politico
Vendor agreements don’t transfer accountability. You must define who is responsible before deployment, not scramble to assign blame after something breaks.
Your governance framework needs a clear RACI:
- Who designed the guardrails?
- Who approved the use case?
- Who monitors the outputs?
- Who answers to the regulator?
Board question: “For each agent in production, can you name the individual accountable for its decisions and show me they accepted that accountability in writing?”
5. Are We Measuring Outcomes or Just Activity?
The board should be asking:
- What business outcomes have agents delivered?
- What’s the measurable ROI?
- Where is freed capacity being redeployed?
The top-performing organisations aren’t just cutting costs. They are using AI to pursue growth opportunities and reinvent their business model.
Deploying 50 AI agents is not a success metric, knowing how they are managed and governed is.
AI adoption has moved quickly. McKinsey’s 2025 global survey found that 88% of respondents say their organizations regularly use AI tools in at least one business function, up from 78% the previous year, yet only 7% say AI is fully scaled across the organization, with 30% remaining stuck in the piloting phase. Source: TechRadar
Board question: “Beyond the number of agents deployed, what measurable business outcomes have they delivered in the last 90 days and what would happen if we switched them off?”
6. Is Our Architecture Ready, Not Just Our Technology?
When agents fail at scale the root cause is almost never the model, it’s the architecture around it. Agents that both decide and execute in a single loop (with no independent control layer) should be considered as structurally unsafe at enterprise scale.
“While the model is often the first suspect for pilots stalling, the architecture is the more likely culprit. In most cases, a production AI agent should not be one model doing everything. It should be a coordinated system of specialized models, each doing what it is best suited to perform”. Source: Forbes
Board question: “Can your architecture team draw the boundary between where an agent thinks and where it acts and show the independent control layer between them?”
The Board’s Role
The boards role is about asking the right questions and demanding clear answers (not being technical). The board needs to understand:
- Authority: What agents are allowed to do
- Accountability: Who owns the consequences
- Auditability: Whether actions can be traced and explained
- Reversibility: Whether mistakes can be undone
- Measurement: Whether outcomes justify the investment
The organisations capturing value from agentic the ones whose boards insisted on these foundations before scaling.
What Comes Next
If your board is confident on these six questions, the next step is ensuring your architecture and engineering teams have the technical playbook to deliver.
In the companion piece to this article, I will publish soon “Agentic AI Governance: Seven Principles for Deploying AI Agents That Enterprises Can Actually Trust” I will set out seven architectural principles that translate board-level confidence into production-grade governance.
The board sets the standard and architecture delivers it.
References
- “Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026,” August 2025. Gartner
- “Leading Through the Agentic Deployment Era,” August 2026. Projects 1.15 billion active AI agents by 2029 (40x the 2025 baseline). IDC
- Agent Economics: Adoption and Delivery Model for Agents, Actions and Tokens. IDC
- “Agentic AI Is Scaling Faster Than Guardrails,”. Deloitte
- “AI Agents Are Only the Beginning: Deloitte
- “Why do AI governance and accountability matter for enterprises?”. Dataiku
- “What Can Agentic Cybersecurity Do – And It’s 2027 CISO Budget Impact”. Forbes
- “These 4 AI governance tips help counter shadow agents”. Google
- “AI governance and accountability: best practices for enterprise agents”. Dataiku
- “Action accountability: Why every AI agent needs a trace layer”. IBM
- “How to make AI agent reasoning visible and auditable”. Kore.ai
- “Proving AI Governance to Auditors: What Documentation You Actually Need”. Kiteworks
- “Human in the Loop AI Operations Statistics 2026”. Stealth Agents
- “AI Agent Governance in 2026: Why Enterprises Need Kill-Switches for Autonomous Agents”. SoluLab
- “AI safety framework floated by Democratic duo”. Politico
- “Moving AI from pilot to production starts with the data”. TechRadar
- “The Single-Model Trap That’s Stalling Enterprise AI”. Forbes
Leave a Reply